Readable storage medium, vehicle, and image-based distance measurement method and system

Abstract

The present application relates to the field of intelligent driving, and provides an image-based distance measurement method, wherein an image is collected by a camera. The method comprises: recognizing a vehicle target box on the basis of an image collected by a camera; performing fusion weight condition determination on the vehicle target box; in response to the vehicle target box not satisfying a fusion weight condition, on the basis of a ground projection point of the vehicle target box, obtaining a distance between the camera and a vehicle corresponding to the vehicle target box in a world coordinate system; and in response to the vehicle target box satisfying the fusion weight condition, fusing a first distance measurement value and a second distance measurement value to calculate the distance between the camera and the vehicle corresponding to the vehicle target box in the world coordinate system, wherein the first distance measurement value is obtained on the basis of the ground projection point of the vehicle target box, and the second distance measurement value is obtained on the basis of the width and / or height of the vehicle target box, thereby obtaining an accurate and reliable distance measurement result. The present application further provides an image-based distance measurement system, a readable storage medium, and a vehicle.

Description

Readable storage medium, vehicle, image ranging method and system TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent driving, and particularly relates to an image ranging method, an image ranging system, a readable storage medium and a vehicle. BACKGROUND

[0002] With the continuous upgrading and iteration of software and hardware configurations of intelligent systems of automobiles, it is common for automobiles to be equipped with intelligent vehicle lamps with interactive functions. Accurate measurement of the position of a perceived target relative to a vehicle lamp is very critical to the interactive functions of intelligent vehicle lamps, such as the implementation of the ADB shielding function of a matrix LED headlamp on a target vehicle and the projection function of a digital projection headlamp DLP. If accurate perception measurement cannot be achieved, the interaction of intelligent headlamps will be a poor experience. At present, considering cost and practicability, monocular measurement methods are mainly used to achieve the ranging of a perceived target, that is, the position of a target on an image is recognized by using a perception algorithm based on a picture captured by a monocular camera, and the actual spatial coordinate position of the target is mapped according to the imaging principle of the camera.

[0003] Monocular ranging mainly relies on the geometric information of an object in an image to calculate the distance. A commonly used method is triangulation, as shown in FIG. 1. A is the host vehicle, B and C are the front vehicles, P is the camera, f is the focal length of the camera, H is the height of the camera, Z1 and Z2 are the horizontal straight-line distances of the camera from the front vehicles B and C respectively, and y1 and y2 are the projections of the detection frames corresponding to B and C on the image, respectively. According to similar triangles, y1 = f x H / Z1 and y2 = f x H / Z2 can be obtained.

[0004] Converting the above formula, Z1 = f x H / y1 and Z2 = f x H / y2 can be obtained.

[0005] Therefore, the actual spatial distance of an object in an image needs to be calculated by knowing the coordinates of the landing site of the detection frame corresponding to the object. However, the current distance detection method may cause the camera to shake due to the jolting of the vehicle, resulting in poor imaging quality of the target in the camera imaging image. There will be a large error in the measurement result according to the landing site, and thus the vehicle lamp interaction will be wrong.

[0006] SUMMARY

[0007] The technical problem to be solved by the present application is to provide an image ranging method, an image ranging system, a readable storage medium and a vehicle, which can accurately measure a front vehicle target and ensure that the ranging result is reliable and accurate.

[0008] To solve the above technical problem, the first aspect of the present application provides an image ranging method, wherein the image is collected by a camera, and the method comprises:

[0009] acquire an image based on the camera to identify a vehicle target frame;

[0010] perform a fusion weight condition judgment on the vehicle target frame;

[0011] in response to the vehicle target frame not satisfying the fusion weight condition, derive a distance between the camera and a vehicle corresponding to the vehicle target frame in a world coordinate system based on a landing site of the vehicle target frame;

[0012] in response to the vehicle target frame satisfying the fusion weight condition, fuse and calculate a first ranging value and a second ranging value to derive a distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system, the first ranging value being derived based on the landing site of the vehicle target frame, and the second ranging value being derived based on a width and / or a height of the vehicle target frame.

[0013] In some embodiments, the fusion weight condition includes one or more of the following fusion weight conditions:

[0014] a first fusion weight condition: an intersection over union between a tracking frame derived based on historical trajectory information of the vehicle target frame and the vehicle target frame derived at a current time is less than a set threshold value;

[0015] a second fusion weight condition: a difference between the first ranging value and the second ranging value exceeds a set threshold value;

[0016] a third fusion weight condition: a historical coordinate parameter of the landing site derived based on the historical trajectory information of the vehicle target frame is unstable.

[0017] In some embodiments, the fusing and calculating of the first ranging value and the second ranging value to derive the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system includes:

[0018] based on a mean value of the first ranging value and the second ranging value, respectively calculating a fusion weight of the first ranging value and the second ranging value according to a fusion weight formula;

[0019] based on the first ranging value, the second ranging value, and the respective fusion weights thereof, calculating the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system according to a weighted ranging formula.

[0020] In some embodiments, the fusion weight formula is as follows: k1=1-Z / Z0 k2=Z / Z0

[0021] wherein Z is a mean value of the first ranging value and the second ranging value, Z0 is a set distance value of the fusion weight, k1 is a fusion weight of the first ranging value, and k2 is a fusion weight of the second ranging value;

[0022] The weight ranging formula is as follows: Zlast=k1*Z1+k2*Z2

[0023] wherein Zlast is a distance between the camera and a vehicle corresponding to the vehicle target frame in a world coordinate system, Z1 is the first ranging value, and Z2 is the second ranging value.

[0024] In some embodiments, the identifying the vehicle target frame based on the image captured by the camera comprises:

[0025] obtaining attribute information of a vehicle lamp detection frame in the image captured by the camera, wherein the attribute information of the vehicle lamp detection frame at least includes a vehicle lamp width and a position;

[0026] deriving a vehicle target frame corresponding to the vehicle lamp detection frame based on the attribute information of the vehicle lamp detection frame.

[0027] In some embodiments, the deriving the vehicle target frame corresponding to the vehicle lamp detection frame based on the attribute information of the vehicle lamp detection frame comprises:

[0028] determining a vehicle category based on the attribute information of the vehicle lamp detection frame;

[0029] obtaining a width and a height of a corresponding vehicle target frame according to the vehicle category, and deriving coordinate information of the vehicle target frame based on coordinate information of the vehicle lamp detection frame.

[0030] In some embodiments, in a case where there is a delay in a system to which the image ranging method is applied, the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system is predicted based on the delay, and the delay prediction comprises:

[0031] obtaining trajectory information of the vehicle target frame, wherein the trajectory information at least includes a ranging value and a timestamp;

[0032] deriving a ranging value-time linear fitting equation based on the timestamp and the ranging value of the vehicle target frame, and performing parameter stability analysis on the ranging value-time linear fitting equation;

[0033] in response to the parameter stability of the ranging value-time linear fitting equation, calculating a first ranging value at a system delay time according to the ranging value-time linear fitting equation, and predicting the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time based on the first ranging value;

[0034] in response to the parameters of the ranging value-time linear fitting equation being unstable, predicting a distance between the camera and the vehicle corresponding to the vehicle target frame in a world coordinate system at a system delay time according to a ranging value of the vehicle target frame at a current time;

[0035] wherein the system delay time is a time after the current time plus a system delay time.

[0036] In some embodiments, the parameter stability analysis of the ranging value-time linear fitting equation is performed according to a maximum-minimum value difference method.

[0037] In some embodiments, the delay prediction further comprises: performing smoothing processing on a predicted ranging value of the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time.

[0038] In some embodiments, the smoothing processing on the predicted ranging value of the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time comprises:

[0039] obtaining a predicted ranging value at a previous time of the system delay time;

[0040] performing smoothing processing based on the predicted ranging value at the previous time of the system delay time and the predicted ranging value at the system delay time to obtain a predicted smoothed ranging value, and the formula is as follows: smooth:value=Zp_back*s+Zp_cur*(1-s)

[0041] wherein smooth:value is the predicted smoothed ranging value, Zp_back is the predicted ranging value at the previous time of the system delay time, Zp_cur is the predicted ranging value at the system delay time, and s is a scale factor.

[0042] predicting the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time based on the predicted smoothed ranging value.

[0043] In some embodiments, the method further comprises:

[0044] calibrating internal parameters, external parameters, and distortion parameters of the camera,

[0045] correcting the vehicle lamp detection frame or the vehicle target frame based on the internal parameters, the external parameters, and the distortion parameters of the camera.

[0046] In some embodiments, the method further comprises:

[0047] update the vanishing point coordinate parameter according to a vanishing point coordinate set in a set time period, the vanishing point coordinate in the vanishing point coordinate set being obtained based on a lane line calculated by detecting an image collected by the camera in real time in response to a vehicle speed being greater than a set threshold value;

[0048] update the camera extrinsic parameter based on the updated vanishing point coordinate parameter.

[0049] The second aspect of the present application provides an image ranging system, comprising:

[0050] a camera configured to collect images;

[0051] an identification module configured to identify a vehicle target frame based on the images collected by the camera;

[0052] a judgment module configured to perform a fusion weight condition judgment on the vehicle target frame;

[0053] a ranging module configured to select a ranging mode according to the fusion weight condition judgment result to obtain a distance between the camera and a vehicle corresponding to the vehicle target frame in a world coordinate system;

[0054] wherein the ranging module is configured to: in response to the vehicle target frame not satisfying the fusion weight condition, obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system according to a landing site of the vehicle target frame; and in response to the vehicle target frame satisfying the fusion weight condition, fuse and calculate a first ranging value and a second ranging value to obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system, the first ranging value being obtained based on the landing site of the vehicle target frame, and the second ranging value being obtained based on a width and / or a height of the vehicle target frame.

[0055] The third aspect of the present application provides a readable storage medium, instructions of the storage medium being executed by a processor of an electronic device, the electronic device being capable of executing the image ranging method.

[0056] The fourth aspect of the present application provides a vehicle comprising the image ranging system and / or the readable storage medium.

[0057] Through the above method, the present application has the following beneficial effects:

[0058] The application obtains a vehicle target frame by acquiring a camera-acquired image, judges the vehicle target frame by a fusion weight condition, selects a fusion weight, and when the vehicle target frame does not satisfy the fusion weight condition, it means that the vehicle target frame recognized in the camera-acquired image is relatively accurate, and the landing point can be directly used for ranging; when the vehicle target frame satisfies the fusion weight condition, it means that the imaging quality of the camera-acquired image is poor, and the result obtained by relying on the recognized vehicle target frame or the landing point ranging is unreliable, at this time, the landing point ranging result and the ranging result based on the width and / or height of the vehicle target frame are fused and calculated to obtain a relatively accurate and reliable ranging result, so that accurate ranging can be performed in a complex situation with poor imaging quality. BRIEF DESCRIPTION OF DRAWINGS

[0059] Fig. 1 is a principle diagram of triangulation;

[0060] Fig. 2 is a step diagram of the image ranging method of the application;

[0061] Fig. 3 is a principle diagram of camera pinhole imaging;

[0062] Fig. 4 is a frame schematic diagram of the image ranging system of the application.

[0063] BRIEF DESCRIPTION OF DRAWINGS 100: camera; 200: recognition module; 300: judgment module; 400: ranging module. DETAILED DESCRIPTION

[0064] The specific embodiments of the application are described in detail below with reference to the accompanying drawings, and it should be understood that the specific embodiments described herein are only used to illustrate and explain the application, and the protection scope of the application is not limited to the specific embodiments described below.

[0065] The first aspect of the application provides an image ranging method, and the image used for ranging is acquired by a camera. Specifically, referring to Fig. 2, the image ranging method comprises the following steps:

[0066] S100, recognizing a vehicle target frame based on a camera-acquired image.

[0067] S200, judging the vehicle target frame by a fusion weight condition; wherein the fusion weight condition comprises one or more of the following fusion weight conditions:

[0068] The first fusion weight condition is that historical trajectory information of the vehicle target frame is obtained, the historical trajectory information including a timestamp, a frame height, a frame width and a position, so that a tracking algorithm is used to track and predict, the size and position of the tracking frame at the current time are output, and the tracking frame is compared with the vehicle target frame obtained based on the camera captured picture at the current time. When the intersection-over-union of the two vehicle target frames is less than a set threshold, it indicates that there is a large difference. The tracking algorithm is an image recognition algorithm, which can extract and recognize the target features in the captured image to determine whether the targets in the front and rear frames are the same target, so as to track and identify the same target to generate the trajectory information of the target frame corresponding to the target.

[0069] The second fusion weight condition is that a first distance value is obtained based on the landing position of the vehicle target frame at the current time, and a second distance value is obtained based on the width and / or height of the vehicle target frame at the current time. When the difference between the first distance value and the second distance value exceeds a set threshold, it indicates that there is a large difference.

[0070] The third fusion weight condition is that the historical coordinate parameters of the landing position are obtained based on the historical trajectory information of the vehicle target frame. The historical coordinate parameters are stably analyzed. When the parameters change greatly before and after, the historical coordinate parameters are unstable, and there may be a vehicle jolt causing picture shaking. The parameters can be linearly fitted, and the vertical coordinate difference-time linear fitting equation is analyzed for parameter stability according to the maximum and minimum value difference method. The maximum and minimum values in the data used for linear fitting equation are found by the maximum and minimum value difference method. The stability is judged by comparing the absolute value of the difference between the maximum value and the minimum value with the comparison value. The comparison value can be set by referring to the empirical average value of the data. If the absolute value of the difference is less than or equal to the comparison value, it can be determined that the parameter is stable. If the absolute value of the difference is greater than the comparison value, it can be determined that the parameter is unstable.

[0071] S300, in response to the vehicle target frame not meeting the fusion weight condition, the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system can be obtained according to the landing position (lower edge midpoint) of the vehicle target frame.

[0072] S400, in response to the vehicle target frame meeting the fusion weight condition, the first distance value and the second distance value are fused and calculated to obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system. The first distance value is obtained based on the landing position of the vehicle target frame, and the second distance value is obtained based on the width and / or height of the vehicle target frame.

[0073] It should be noted that, in the real world, the width and height of the vehicle target are fixed, and in camera imaging, the width and height of the target frame corresponding to the vehicle target are inversely proportional to the actual distance of the vehicle target from the camera, so that the pixel width and height of the left top and right bottom points of the target image frame can be calculated, and the actual distance of the target from the camera can be calculated according to the pinhole imaging relationship, wherein the second ranging value can be calculated only according to the width of the vehicle target frame; or the second ranging value is calculated only according to the height of the vehicle target frame; or the second ranging value is the average of the value calculated according to the width of the vehicle target frame and the value calculated according to the height of the vehicle target frame.

[0074] After obtaining the vehicle target frame, ranging can be performed according to the landing site (lower edge midpoint) of the vehicle target frame, or ranging can be performed according to the frame height and / or frame width of the vehicle target frame. Due to differences in the imaging quality of the camera collected images, there may be a problem that the ranging result obtained only by the landing site is unreliable. In this regard, the present application performs fusion weight condition judgment on the vehicle target frame to select the fusion weight. When the vehicle target frame does not satisfy the fusion weight condition, it means that the vehicle target frame recognized in the camera collected image is relatively accurate, and the landing site can be directly used for ranging. When the vehicle target frame satisfies the fusion weight condition, it means that the imaging quality of the camera collected image is poor, and the result obtained by relying only on the landing site of the recognized vehicle target frame is unreliable. At this time, the landing site ranging result and the ranging result based on the width and / or height of the vehicle target frame can be fused and calculated to obtain a more accurate and reliable ranging result, so that accurate ranging can be performed in a complex situation with poor imaging quality.

[0075] In some specific embodiments, the fusion calculation of the first ranging value and the second ranging value to obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system includes:

[0076] Based on the mean value of the first ranging value and the second ranging value, the fusion weight of the first ranging value and the second ranging value is calculated according to the fusion weight formula, and the fusion weight formula is as follows: k1=1-Z / Z0 k2=Z / Z0

[0077] Wherein, Z is the mean value of the first ranging value and the second ranging value, k1 is the fusion weight of the first ranging value, k2 is the fusion weight of the second ranging value, and Z0 is the set distance value of the fusion weight; it should be noted that Z0 is considered as a set value for judging the approximate distance between the vehicle target and the camera, and the closer the distance between them is within the set distance value Z0, the more accurate the landing point ranging is, that is, the first ranging value has more fusion weight, and vice versa, the farther the distance between them is, the more accurate the width-height ranging is, that is, the second ranging value has more fusion weight. When the vehicle target frame corresponds to a vehicle that is far away from the camera and greater than Z0, the vehicle target is small, and the error of the landing point ranging result is large at this time, therefore, the first ranging value and the second ranging value are not fused, but only the second ranging value is taken as the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system, that is, when Z is greater than or equal to Z0, k1 can be set to 0 and k2 can be set to 1; preferably, Z0 is 100, that is, the set distance of the fusion weight is 100m;

[0078] Based on the first ranging value, the second ranging value and their respective fusion weights, the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system is calculated according to the weighted ranging formula, which is as follows: Zlast=k1*Z1+k2*Z2

[0079] Wherein, Zlast is the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system, Z1 is the first ranging value, and Z2 is the second ranging value.

[0080] It should be noted that the use environment of the intelligent headlamp is mostly at night, that is, in a weak ambient light scene, the image captured by the camera has poor imaging quality, and the vehicle target frame may not be accurately obtained. In order to improve the accuracy and reliability of the vehicle target frame identification, as a preferred embodiment, the step S100 includes the following steps:

[0081] S101, obtaining attribute information of the vehicle lamp detection frame in the image captured by the camera, wherein the attribute information of the vehicle lamp detection frame at least includes the width and position of the vehicle lamp.

[0082] Specifically, the target feature in the captured image can be extracted and identified, so that the vehicle lamp target can be extracted. The vehicle lamp detection frame is generated based on the vehicle lamp target, and the width of the vehicle lamp detection frame is the width of the vehicle lamp target, and the position of the vehicle lamp detection frame is the coordinate position of the vehicle lamp detection frame in the captured image. The feature point coordinates on the vehicle lamp detection frame can be taken as the position of the vehicle lamp detection frame, for example, the feature point is the center point of the vehicle lamp detection frame or the lower edge midpoint of the vehicle lamp detection frame.

[0083] S102, obtaining the vehicle target frame corresponding to the vehicle lamp detection frame based on the attribute information of the vehicle lamp detection frame.

[0084] Specifically, the conversion relationship between the width of the vehicle lamp and the width of the vehicle body and the relative position relationship between the vehicle lamp and the vehicle body as a whole can be pre-stored, so that a fixed value can be supplemented according to the position of the vehicle lamp detection frame and the width of the vehicle lamp, and then the corresponding vehicle target frame on the collected image can be obtained, which is used to represent the vehicle.

[0085] Through the above image ranging method, in the case that the vehicle is in a weak ambient light environment, the light-emitting characteristics of the vehicle lamp can be used to clearly identify the outline of the vehicle lamp in the collected image. Since the vehicle lamp is part of the vehicle body, the position of the vehicle body can be inferred according to the width and position of the vehicle lamp, that is, the vehicle target frame can be obtained according to the vehicle lamp detection frame, and then image ranging can be realized according to the measurement target frame. Even in a complex scenario with poor imaging quality at night, accurate ranging can be achieved, which provides a more accurate perception measurement result for the shielding and projection of the intelligent vehicle lamp, and provides a stable and accurate interaction effect for the intelligent vehicle lamp.

[0086] It should be noted that the above image ranging method of the present application is based on the width of the vehicle lamp for ranging. The present application can not be limited to ranging in a weak ambient light environment, but can also range in a strong ambient light environment, as long as the vehicle lamp features can be clearly identified in the collected image. However, in the case that the ambient light satisfies the condition that the outline of the vehicle body can be clearly identified in the collected image, the ranging result based on the vehicle target frame obtained from the vehicle body outline is more accurate than the ranging result based on the vehicle target frame obtained from the vehicle lamp detection frame. Therefore, in the preferred embodiment of the image ranging method of the present application, a prerequisite for vehicle lamp detection frame recognition can be set, for example, the ambient light intensity can be detected, and / or the camera image collected image can be analyzed. If the ambient light satisfies the preset intensity, the camera can clearly capture the collected image with the vehicle body outline, and / or the clear vehicle body outline features can be extracted from the camera collected image. That is, instead of ranging based on the vehicle target frame obtained from the vehicle lamp detection frame, the vehicle body outline features in the camera collected image are directly obtained to range based on the vehicle target frame. On the contrary, if the ambient light intensity does not satisfy the condition that the camera can capture the collected image with a clear vehicle body outline, and / or the clear vehicle body outline features cannot be extracted from the camera collected image, the above ranging based on the vehicle target frame obtained from the vehicle lamp detection frame is used.

[0087] In step S102, different vehicle models have different vehicle lights, for example, large vehicles (trucks, buses and large special vehicles, etc.) and ordinary passenger vehicles (sedans, SUVs and MPVs, etc.) have large differences in vehicle width and height, so the corresponding vehicle target frame also has differences. In order to obtain a more accurate ranging structure, the present application determines the vehicle category based on the attribute information of the vehicle light detection frame. Specifically, the vehicle light width and installation position of large vehicles and ordinary passenger vehicles are different. The values of the vehicle light width of most ordinary passenger vehicles and large vehicles can be counted, so as to identify whether the vehicle belongs to a large vehicle or an ordinary passenger vehicle according to the vehicle light width. Then, the ratio of the vehicle light width to the vehicle body width and the ratio of the vehicle light bottom height position to the vehicle bottom position are counted, so as to obtain the relative position relationship and the relative numerical relationship of the frame width and height of the vehicle light detection frame and the vehicle target frame of different categories. Then, after the vehicle category is identified, the preset values (preset position relationship and preset frame width and height relative numerical relationship) corresponding to the category of vehicle are called. Based on the preset values and the attribute information of the vehicle light detection frame, the width and height of the vehicle target frame are obtained, and based on the coordinate information of the vehicle light detection frame, the coordinate information of the vehicle target frame is obtained. In order to improve the accuracy of vehicle category identification, the frame height (vehicle light vertical height) of the vehicle light detection frame can be further identified, and / or the front and rear of the vehicle can be determined based on the features in the camera captured image, and the frame width (vehicle light horizontal width) of the vehicle light detection frame can be used for determination.

[0088] It should be noted that the above vehicle category identification of large vehicles and ordinary passenger vehicles is only a specific embodiment of vehicle category division, and different vehicle models can be further divided, for example, the vehicle category can also be divided into sedans, SUVs and MPVs, etc. Further, the vehicle light data corresponding to different vehicle models can be input into the image recognition algorithm for learning, so that the specific vehicle model corresponding to the vehicle light can be accurately obtained according to the features of the vehicle light identified in the camera captured image, and the vehicle target frame can be obtained according to the vehicle data of the vehicle model. In the process of obtaining the specific vehicle model corresponding to the vehicle light, since there are many vehicle models in practice, there may be vehicle models that cannot be identified. At this time, the vehicle category can be determined based on the attribute information of the vehicle light detection frame, and the vehicle target frame can be obtained according to the preset values corresponding to the category of vehicle.

[0089] It should be noted that the vehicle target frame corresponding to the vehicle light detection frame is obtained based on the attribute information of the vehicle light detection frame. In the case that the vehicle is close to the camera, the left and right frames of the vehicle target frame may exceed the left and right boundaries of the captured image. In the case that this situation is identified, the frame height / width of the vehicle target frame is not used to calculate the ranging value, but the actual distance between the camera and the vehicle is directly obtained according to the landing position of the vehicle target frame, that is, the first ranging value is directly output as the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system.

[0090] In some embodiments, in the case of a system to which the image ranging method is applied, there is a time delay, i.e., there is a time interval between the camera capturing an image and the final output of the actual ranging value calculated, or there is a time interval between the camera capturing an image and the output of a control signal based on the ranging value of the final output, in order to eliminate the influence of the system delay, the present application can also predict the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time. The delay prediction includes:

[0091] Obtaining trajectory information of the vehicle target frame, wherein the trajectory information at least includes a ranging value and a timestamp, and the trajectory information also contains the ranging value of the vehicle target frame obtained based on the camera capturing an image at the current time.

[0092] Based on the timestamp and the ranging value of the vehicle target frame, a ranging value-time linear fitting equation is derived, and parameter stability analysis is performed on the ranging value-time linear fitting equation. Preferably, the most value difference method is used to perform parameter stability analysis on the ranging value-time linear fitting equation. In addition, in order to avoid the time span of the trajectory information being too large and affecting the accuracy of the linear fitting equation, the trajectory information of the vehicle target frame within a preset time is preferably used.

[0093] In response to the parameter stability of the ranging value-time linear fitting equation, a first ranging value at the system delay time is calculated according to the ranging value-time linear fitting equation, and the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time is predicted based on the first ranging value;

[0094] In response to the parameter instability of the ranging value-time linear fitting equation, the actual distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time is predicted according to the ranging value of the vehicle target frame at the current time, i.e., the ranging value of the vehicle target frame at the current time is directly used as the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time.

[0095] Wherein, the system delay time is the time after the current time of the camera capturing an image plus the system delay time.

[0096] In some embodiments, the delay prediction further includes: smoothing the predicted ranging value of the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time, thereby providing a more reliable predicted ranging value, and the smoothing step includes:

[0097] Obtaining a predicted ranging value at a previous time of the system delay time;

[0098] The predicted ranging value is obtained by smoothing the predicted ranging value at the time before the system delay and the predicted ranging value at the system delay time. The formula is as follows: smooth:value=Zp_back*s+Zp_cur*(1-s)

[0099] Where smooth:value is the predicted smooth ranging value, Zp_back is the predicted ranging value at the time before the system delay, Zp_cur is the predicted ranging value at the system delay, and s is the scaling factor, preferably 0.5.

[0100] The system predicts the distance between the camera and the vehicle target box in the world coordinate system at a delayed time based on the predicted smooth distance measurement value.

[0101] Specifically, taking system delay as t and current time as T, the trajectory information of the vehicle target box within 1 second (preset time) of time T is obtained. Based on this trajectory information, a linear fitting equation of ranging value-time is derived for parameter stability analysis, and then the predicted ranging value N at time T+t (system delay time) is obtained. t If the image captured by the camera at the current moment is the first frame, it means that there was no predicted ranging value at the previous moment, and the predicted ranging value N is directly used. t Output: If there exists a predicted ranging value N for time T+t-1 (the time before the system delay) in the previous time step, then... t-1 Then N t With N t-1 Smoothing is performed on Zp_cur and Zp_back respectively to obtain the predicted smoothed ranging value N. t * And predict smoothed ranging value N t * The final output is used to update the predicted distance value N at time T+t. t .

[0102] It should be noted that the conversion of the camera image coordinates to the information in the world coordinate system is converted to the camera coordinate system, and then converted to the image coordinate system, and in the process of converting to the pixel coordinate system, the world coordinate system is the absolute coordinate system of the objective three-dimensional world, which can be represented by (Xw, Yw, Zw). The position of the digital camera and the position of any other object placed in the three-dimensional environment are described by the reference coordinate system. The camera coordinate system is the optical center coordinate system, which takes the optical center of the camera as the coordinate origin, and the X-axis and the Y-axis are parallel to the X-axis and the Y-axis of the image coordinate system, respectively. The optical axis of the camera is the Z-axis, and the coordinate values thereof are represented by (Xc, Yc, Zc). The image coordinate system takes the center of the CCD (charge coupled device) image plane as the coordinate origin, and the X-axis and the Y-axis are parallel to the two perpendicular edges of the image plane, respectively. The coordinate values thereof are represented by (x, y), which represent the position of the pixel in the image in physical units (such as millimeters). The pixel coordinate system takes the top left corner of the image plane as the origin, and the X-axis and the Y-axis are parallel to the X-axis and the Y-axis of the image coordinate system, respectively. The coordinate values thereof are represented by (u, v). The camera calibration formula is as follows:

[0103] wherein [R T] is the external parameter of the camera to be calibrated, that is, a matrix relationship from the actual world coordinate system to the camera coordinate system is solved; [f x , f y , u0, v0] is the internal parameter of the camera to be calibrated, that is, a matrix relationship from the camera coordinate system to the pixel coordinate system is solved, which includes two parts: the camera coordinate system to the image coordinate system, and the image coordinate system to the pixel coordinate system. The internal parameter f x =f / d x , f y =f / d y , wherein f is the physical focal length of the camera, d x , d y represent the physical size of one imaging pixel on the X-axis and Y-axis of the CCD image plane.

[0104] In the above camera calibration process, the external parameter and the internal parameter of the camera can be solved, and in the calibration process of the internal parameter, the distortion parameter (k1, k2, p1, p2, p3) of the camera can also be solved, wherein k1 and k2 are the radial distortion coefficients of the lens, and p1, p2, and p3

[0105] are the tangential distortion coefficients of the lens. In some specific embodiments, the image ranging method of the present application further comprises calibrating the internal parameter, the external parameter, and the distortion parameter of the camera, and correcting the vehicle lamp detection frame or the vehicle target frame identified based on the collected image based on the internal parameter, the external parameter, and the distortion parameter of the camera.

[0106] It should be noted that the camera calibration and the landing position of the vehicle target frame can be used to obtain the world coordinates of the landing position in the world coordinate system, and the specific steps are as follows:

[0107] According to the internal parameters, external parameters and distortion parameters of the camera, a mapping conversion equation between the pixel coordinate system, the image coordinate system, the camera coordinate system and the world coordinate system is obtained; specifically, a rotation matrix R, a translation matrix T, an inverse rotation matrix R-1, C={k1,k2,p1,p2,k3} R -1 =1 / R

[0108] Wherein, the relationship between the camera coordinate system and the world coordinate system is expressed by Euler angles, r is the roll angle, y is the yaw angle, p is the pitch angle, x cam , y cam , z cam are three coordinates of the camera installation point in the world coordinate system.

[0109] The pixel coordinates P img (x img , y img ) of the landing position at the current time are obtained, and the de-distortion algorithm is used to obtain the actual normalized coordinates P img (x norm , y norm , 1) of the point P norm on the CCD, that is, the coordinates in the image coordinate system. The de-distortion algorithm can use the de-distortion algorithm in Matlab software or OpenCv (Open Computer Vision Library, cross-platform computer vision library). The coordinates and the camera internal parameters are processed to obtain the actual imaging pixel coordinates after distortion correction, and the formula is as follows, u=x norm *f x +u0 v=y norm *f y +v0.

[0110] The P norm point in the image coordinate system is counter-rotated to obtain the normalized coordinates P c_norm (x c_norm , y c_norm , 1) in the camera coordinate system, and the formula is as follows:

[0111] Referring to FIG. 3, the camera coordinates P c (x c , y c , z c ) of the target object can be obtained by using the triangular similarity of the camera pinhole imaging, and the formula is as follows: fScale=H / Yc_norm == Z c / 1.0 = X c / X c_norm = Y c / Y c_norm Z c = 1.0 * H / Y c_norm X c = X c_norm * H / Y c_norm Y c = Y c_norm * H / Y c_norm

[0112] wherein, fScale is the distance between the camera and the target object, H is the installation height of the camera from the ground;

[0113] The camera coordinates of the target object are converted into the world coordinate system by the camera translation operation, so as to obtain the world coordinates P corresponding to the landing position of the target frame on the image wrd (X wrd , Y wrd , Z wrd ), the formula is as follows:

[0114] The above fScale is the first ranging value based on the landing position, and according to the obtained world coordinates of the landing position, the world coordinates of the four corner points (the upper left point, the lower left point, the upper right point, and the lower right point) of the vehicle target frame can be directly calculated in combination with the vehicle body size corresponding to the vehicle category; or the fScale obtained in the process of calculating the world coordinates corresponding to the landing position is used as a normalization factor of the target distance to calculate the world coordinates of the four corner points (the upper left point, the lower left point, the upper right point, and the lower right point).

[0115] It should be noted that the image ranging method of the present application is applied to vehicle image ranging, the camera is a vehicle-mounted camera, in order to facilitate the implementation of corresponding control, the world coordinate system can adopt the vehicle body coordinate system, so as to provide stable and accurate interaction effect for intelligent headlamps

[0116] In some specific embodiments, in order to ensure the accuracy and reliability of the ranging value, the image ranging method of the present application further comprises:

[0117] Updating the vanishing point coordinate parameter according to the vanishing point coordinate set in the set time period, the vanishing point coordinates in the vanishing point coordinate set being obtained based on the vehicle lane calculated and obtained based on the image collected by the camera in real time in response to the vehicle speed being greater than the set threshold;

[0118] Updating the camera external parameter based on the updated vanishing point coordinate parameter.

[0119] Specifically, the vehicle speed and lane line where the camera is located are detected in real time; in response to the vehicle speed being greater than a set threshold and the lane line being a straight line, the intersection coordinates of the lane line on the image are the vanishing point coordinates, so that the vanishing point coordinates can be calculated based on the lane line where the camera is located, and the vanishing point coordinates are stored, and the vanishing point coordinates are updated according to the historical data of the vanishing point coordinates; since the relative position of the camera and the vehicle does not change substantially, the vanishing point coordinate data used should include vanishing point data accumulated for more than a preset time (preferably 24 hours), to ensure that there is enough data amount, the historical data is written into a file for saving, the historical data is stored in the system all the time, the historical data can be distributed and counted to obtain the distribution area with the most data, the vanishing point coordinates corresponding to the distribution area are used to update the vanishing point coordinate parameters, based on the updated vanishing point coordinate parameters, the pitch angle in the camera external parameters is updated, and then the accurate and reliable distance measurement value is calculated based on the updated external parameters. In addition, in order to avoid the vanishing point data from being updated too frequently and increasing the processing burden of the system, the vanishing point coordinate update interval time can be set.

[0120] The second aspect of the present application provides an image distance measurement system, referring to FIG. 4, comprising:

[0121] a camera 100 for acquiring images;

[0122] an acquisition module 200 for identifying a vehicle target frame based on the camera acquiring images;

[0123] a judgment module 300 for performing fusion weight condition judgment on the vehicle target frame;

[0124] a distance measurement module 400 for selecting a distance measurement mode according to the fusion weight condition judgment result to obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system;

[0125] The distance measurement module is configured to: in response to the vehicle target frame not satisfying the fusion weight condition, obtaining the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system according to the landing site of the vehicle target frame; in response to the vehicle target frame satisfying the fusion weight condition, obtaining the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system by fusing and calculating the first distance measurement value and the second distance measurement value; the first distance measurement value is obtained based on the landing site of the vehicle target frame, and the second distance measurement value is obtained based on the width and / or height of the vehicle target frame.

[0126] The image ranging system provided in the application can clearly identify the outline of the vehicle lamp in the collected image under the condition that the vehicle is in a weak ambient light environment, and the vehicle lamp is part of the vehicle body, so the position of the vehicle body can be inferred according to the width and position of the vehicle lamp, that is, the vehicle target frame can be obtained according to the vehicle lamp detection frame, and then image ranging is realized according to the measurement target frame, so that accurate ranging can be realized even in a complex scene with poor imaging quality at night; in addition, the vehicle target frame obtained based on the width of the vehicle lamp or the directly obtained vehicle target frame is subjected to fusion weight condition judgment, and when the imaging quality of the camera collected image is poor, a suitable fusion weight is reasonably selected, the landing point ranging result and the ranging result based on the width and / or height of the vehicle target frame are fused and calculated, so that a more accurate and reliable ranging result is obtained, which provides a more accurate perception measurement result for the shielding and projection of the intelligent vehicle lamp, and provides a stable and accurate interaction effect for the intelligent vehicle lamp.

[0127] The third aspect of the application provides a readable storage medium, when the instructions of the storage medium are executed by the processor of the electronic device, the electronic device can execute the image ranging method provided in the first aspect of the application, therefore, has all the beneficial effects thereof, which will not be repeated here.

[0128] The fourth aspect of the application provides a vehicle comprising the image ranging system provided in the second aspect of the application and / or the readable storage medium provided in the third aspect of the application, therefore, has all the beneficial effects thereof, which will not be repeated here.

[0129] The application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce an apparatus for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0130] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction apparatus, which implements the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks.

[0132] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0133] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the processor can execute instructions. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, or other memory technologies, CD-ROM, digital versatile disc (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information for access by a computing device. In no case does the disclosure rely on a transitory medium to provide a sole means to carry out the operations described herein.

[0134] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

[0135] Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible to a computing device. According to the definition provided herein, a computer readable medium does not include transitory media, such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements in the list, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0137] The above merely provides an example of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. A method of image ranging, the image being captured by a camera, characterized in that, The method comprises: identifying a vehicle target frame based on image acquisition by the camera; judging a fusion weight condition for the vehicle target frame; in response to the vehicle target frame not satisfying the fusion weight condition, deriving a distance between the camera and a vehicle corresponding to the vehicle target frame in a world coordinate system based on a landing site of the vehicle target frame; in response to the vehicle target frame satisfying the fusion weight condition, fusing a first ranging value and a second ranging value to derive the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system, the first ranging value being derived based on the landing site of the vehicle target frame, and the second ranging value being derived based on a width and / or a height of the vehicle target frame.

2. The image ranging method of claim 1, wherein, The fusion weight condition comprises one or more of the following fusion weight conditions: a first fusion weight condition: an intersection-over-union of a tracking frame derived based on historical trajectory information of the vehicle target frame and the vehicle target frame derived at a current time is less than a set threshold value; a second fusion weight condition: a difference between the first ranging value and the second ranging value exceeds a set threshold value; a third fusion weight condition: a historical coordinate parameter of the landing site derived based on the historical trajectory information of the vehicle target frame is unstable.

3. The image ranging method of claim 1, wherein, The fusing of the first ranging value and the second ranging value to derive the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system comprises: deriving respective fusion weights of the first ranging value and the second ranging value based on a mean value of the first ranging value and the second ranging value according to a fusion weight formula; deriving the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system based on the first ranging value, the second ranging value, and the respective fusion weights thereof according to a weighted ranging formula.

4. The image ranging method of claim 3, wherein: the fusion weight formula is as follows: k1 = 1 - Z / Z0 k2 = Z / Z0 wherein Z is the mean value of the first ranging value and the second ranging value, Z0 is a set distance value of the fusion weight, k1 is the fusion weight of the first ranging value, and k2 is the fusion weight of the second ranging value; the weighted ranging formula is as follows: Zlast = k1*Z1 + k2*Z2 wherein Zlast is the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system, Z1 is the first ranging value, and Z2 is the second ranging value.

5. The image ranging method of claim 1, wherein, The identifying of the vehicle target frame based on image acquisition by the camera comprises: obtaining attribute information of a vehicle lamp detection frame in the image acquired by the camera, wherein the attribute information of the vehicle lamp detection frame at least comprises a vehicle lamp width and a position; deriving a vehicle target frame corresponding to the vehicle lamp detection frame based on the attribute information of the vehicle lamp detection frame.

6. The image ranging method of claim 5, wherein, The deriving of the vehicle target frame corresponding to the vehicle lamp detection frame based on the attribute information of the vehicle lamp detection frame comprises: determining a vehicle category based on the attribute information of the vehicle lamp detection frame; obtaining a width and a height of a corresponding vehicle target frame according to the vehicle category, and deriving coordinate information of the vehicle target frame based on coordinate information of the vehicle lamp detection frame.

7. The image ranging method of claim 1, wherein, In the system to which the image ranging method is applied, the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system is predicted in the presence of a system delay, and the delay prediction comprises: obtaining trajectory information of the vehicle target frame, wherein the trajectory information at least comprises a ranging value and a timestamp; obtaining a ranging value-time linear fitting equation based on the timestamp and the ranging value of the vehicle target frame, and performing parameter stability analysis on the ranging value-time linear fitting equation; in response to the parameter stability of the ranging value-time linear fitting equation, calculating a first ranging value at the system delay time according to the ranging value-time linear fitting equation, and predicting the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time based on the first ranging value; in response to the parameter instability of the ranging value-time linear fitting equation, predicting the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time according to the ranging value of the vehicle target frame at the current time; wherein the system delay time is the time after the current time plus the system delay time. The parameter stability analysis on the ranging value-time linear fitting equation is performed according to the maximum-minimum difference method.

8. The image ranging method of claim 7, wherein, The delay prediction further comprises smoothing the predicted ranging value of the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time.

9. The image ranging method of claim 7, wherein, The smoothing of the predicted ranging value of the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time comprises:

10. The image ranging method of claim 9, wherein, obtaining the predicted ranging value at the time before the system delay time; performing smoothing based on the predicted ranging value at the time before the system delay time and the predicted ranging value at the system delay time to obtain a predicted smoothed ranging value, and the formula is as follows: smooth:value=Zp_back*s+Zp_cur*(1-s) wherein smooth:value is the predicted smoothed ranging value, Zp_back is the predicted ranging value at the time before the system delay time, Zp_cur is the predicted ranging value at the system delay time, and s is a scale factor; predicting the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system at the system delay time based on the predicted smoothed ranging value. The method further comprises:

11. The image ranging method according to any one of claims 1-10, wherein, calibrating the internal parameters, external parameters and distortion parameters of the camera, correcting the vehicle light detection frame or the vehicle target frame based on the internal parameters, external parameters and distortion parameters of the camera. The method further comprises:

12. The image ranging method of claim 11, wherein, updating the vanishing point coordinate parameter according to a set of vanishing point coordinates within a set time period, wherein the vanishing point coordinates in the set of vanishing point coordinates are obtained based on the vehicle lane line calculated and obtained from the image captured by the camera in real time in response to the vehicle speed being greater than a set threshold value; updating the camera external parameter based on the updated vanishing point coordinate parameter. comprises:

13. An image ranging system characterized by, a camera (100) for capturing images; an identification module (200) for identifying a vehicle target frame based on the images captured by the camera; a judgment module (300) for performing fusion weight condition judgment on the vehicle target frame; ​ The ranging module (400) is configured to: in response to the vehicle target frame not satisfying the fusion weight condition, obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system according to the landing site of the vehicle target frame; and in response to the vehicle target frame satisfying the fusion weight condition, fuse and calculate the first ranging value and the second ranging value to obtain the distance between the camera and the vehicle corresponding to the vehicle target frame in the world coordinate system; the first ranging value is obtained based on the landing site of the vehicle target frame, and the second ranging value is obtained based on the width and / or height of the vehicle target frame. The instructions of the storage medium are executed by the processor of the electronic device, and the electronic device can perform the image ranging method according to any one of claims 1-12.

14. A readable storage medium, characterized by, The image ranging system of claim 13 and / or the readable storage medium of claim 14.

15. A vehicle characterized by comprising: ​

Images