Ranging method and apparatus, and device, vehicle, storage medium and program product

Abstract

A ranging method, comprising: acquiring a first ranging distance between a first target in a first image frame and a vehicle, and a second ranging distance between the first target in a second image frame and the vehicle (S11); on the basis of the first ranging distance and the second ranging distance, determining a ranging confidence state of the first target (S12); when the ranging confidence state of the first target indicates that the second ranging distance is credible, determining a second predicted size of a second target on the basis of a size relationship between the first target and the second target detected in the second image frame and a first predicted size of the first target (S13); and on the basis of the second predicted size and detection box information of the second target in the second image frame, determining a first target ranging distance between the second target in the second image frame and the vehicle (S14). The ranging distance of a second target can be calculated on the basis of a first predicted size of a first target, the ranging of which is relatively credible, thereby reducing the ranging error of the second target. Further provided are a ranging apparatus, a device, a vehicle, a storage medium, and a program product.

Description

Ranging methods, devices, equipment, vehicles, storage media, and software products

[0001] Cross-referencing

[0002] This application claims priority to Chinese Patent Application No. 202510095133.X, filed on January 21, 2025, entitled “Range Measurement Method, Apparatus, Equipment, Vehicle, Storage Medium and Program Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to, but is not limited to, the field of autonomous driving, and includes, for example, a ranging method, apparatus, device, vehicle, storage medium, and program product. Background Technology

[0004] In the field of autonomous driving, charge-coupled devices (CCDs) are the detection sensors used in intelligent driving. The core functions of a vision system built with a CCD include target detection and distance measurement of the detected target.

[0005] Public content

[0006] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0007] This disclosure provides a ranging method, apparatus, device, vehicle, storage medium, and program product that can improve the low ranging accuracy for distant targets.

[0008] The technical solution adopted in the embodiments of this disclosure is as follows:

[0009] A ranging method, comprising:

[0010] The first ranging distance between the first target and the vehicle in the first image frame and the second ranging distance between the first target and the vehicle in the second image frame are obtained; wherein the first image frame and the second image frame are acquired sequentially by the image acquisition device mounted on the vehicle during the vehicle's movement.

[0011] Based on the first ranging distance and the second ranging distance, determine the ranging confidence state of the first target;

[0012] When the ranging confidence state of the first target indicates that the second ranging distance is reliable, the second predicted size of the second target is determined based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target; wherein the second ranging distance is determined based on the first predicted size; and

[0013] Based on the second predicted size and the detection box information of the second target in the second image frame, the first target ranging distance between the second target and the vehicle in the second image frame is determined.

[0014] Based on the aforementioned technical means, a second predicted size of the second target is calculated based on the first predicted size of the first target, which is considered reliable in its ranging confidence state, and the size relationship between the first target and the second target. Then, the ranging distance between the second target and the vehicle is calculated based on the second predicted size. Since the second ranging distance of the first target is reliable, and this second ranging distance is determined based on the first predicted size, the first predicted size can also be considered reliable. Therefore, based on the first predicted size and the size relationship between the first and second targets, a more accurate second predicted size can be calculated. This allows for a more accurate calculation of the ranging distance between the second target and the vehicle, reducing the ranging error of the second target.

[0015] In some implementations, the first target includes a first cone, and the second target includes a second cone. Determining a second predicted size of the second target based on the size relationship between the first target and the second target detected in the second image frame, and a first predicted size of the first target, includes:

[0016] When the size relationship indicates that the first cone and the second cone have the same size specifications, the first predicted size is determined as the second predicted size; wherein the first predicted size and the second predicted size correspond to the same size parameter, which may include a height parameter and / or a width parameter.

[0017] Based on the aforementioned technical means, when the dimensions of the first and second cones are identical, the first predicted dimension is determined as the second predicted dimension. This improves the overall ranging efficiency and ensures timely and accurate completion of the ranging task, while enhancing the ranging confidence level of the first target's ranging distance.

[0018] In some implementations, the detection box information includes the detection box size and the detection box position; based on the second predicted size and the detection box information of the second target in the second image frame, determining the first target ranging distance between the second target and the vehicle in the second image frame includes:

[0019] Based on the intrinsic parameters of the image acquisition device, the second predicted size, and the detection box size of the second target in the second image frame, the longitudinal ranging distance between the second target and the vehicle in the second image frame is determined.

[0020] Based on intrinsic parameters, the detection box position of the second target in the second image frame, and the longitudinal ranging distance, the lateral ranging distance between the second target and the vehicle in the second image frame is determined; and

[0021] The distance to the first target is determined based on the longitudinal and lateral distances.

[0022] Based on the aforementioned technical means, and using the intrinsic parameters of the image acquisition device, the longitudinal and lateral distances between the second target and the vehicle in the second image frame are determined respectively; and the distance to the first target is determined based on the longitudinal and lateral distances. In this way, the accuracy of the lateral distance determined from the longitudinal distance is improved, thus enhancing the accuracy of the distance to the first target, in addition to improving the accuracy of the longitudinal distance value.

[0023] In some embodiments, the first target includes a first cone, and the second target includes a second cone. Determining the distance to the first target based on longitudinal and lateral distances includes:

[0024] Based on the longitudinal and lateral distance measurements, the coordinates of multiple edge points at the bottom of the second cone are determined.

[0025] Based on the coordinates of multiple edge points, determine the coordinates of the center point of the bottom of the second cone; and

[0026] The distance to the first target is determined based on the coordinates of the center point.

[0027] Based on the aforementioned technical means, the coordinates of multiple edge points at the bottom of the second cone are determined based on the longitudinal and lateral distance measurements. Then, based on these edge point coordinates, the coordinates of the center point at the bottom of the second cone are determined. Finally, based on the center point coordinates, the distance to the first target is determined. In this way, the coordinates of the distance between the second cone and the vehicle can be determined based on the coordinates of multiple edge points of the second cone, improving the accuracy of the distance measurement results.

[0028] In some implementations, determining the ranging confidence state of the first target based on the first ranging distance and the second ranging distance includes:

[0029] Based on the difference between the first and second ranging distances, determine the ranging change corresponding to the second image frame; and

[0030] The ranging confidence state of the first target is determined based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame; wherein, the vehicle movement distance corresponding to the second image frame is the movement distance of the vehicle within the acquisition interval between the first image frame and the second image frame.

[0031] Based on the aforementioned technical means, the ranging change corresponding to the second image frame is determined based on the difference between the first and second ranging distances; and the ranging confidence state of the first target is determined based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame. Thus, by comprehensively considering the differences in the ranging distances of the first target in different image frames and the vehicle movement distance within the corresponding image frame acquisition interval, the accuracy of determining the ranging confidence state of the first target can be improved.

[0032] In some implementations, the first image frame is the frame preceding the second image frame; determining the ranging confidence state of the first target based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame includes:

[0033] Determine the ranging change and the corresponding vehicle movement distance for at least one historical image frame; wherein, the ranging change for the historical image frame is determined based on the difference between the first historical ranging distance of the first target in the historical image frame and the second historical ranging distance of the first target in the previous frame of the historical image frame; and the vehicle movement distance for the historical image frame is the vehicle movement distance within the acquisition interval between the historical image frame and the previous frame of the historical image frame; and

[0034] The ranging confidence state of the first target is determined based on the ranging change and the corresponding vehicle movement distance corresponding to multiple target image frames; wherein, the multiple target image frames include at least one historical image frame and a second image frame.

[0035] Based on the aforementioned technical means, the ranging confidence state of the first target is determined according to the ranging changes and corresponding vehicle movement distances corresponding to multiple target image frames; the multiple target image frames include at least one historical image frame and a second image frame. In this way, the ranging changes and corresponding vehicle movement distances corresponding to each target image frame can be comprehensively considered, improving the accuracy of determining the ranging confidence state of the first target.

[0036] In some implementations, the ranging confidence state of the first target is determined based on the ranging changes and corresponding vehicle movement distances corresponding to multiple target image frames, including:

[0037] When the ranging changes and corresponding vehicle movement distances corresponding to multiple target image frames satisfy the target confidence conditions, the ranging confidence state of the first target is determined as the first state characterizing the reliability of the second ranging distance; wherein, the target confidence conditions include at least one of the following:

[0038] The sum of the ranging changes corresponding to each target image frame is less than the first target distance threshold; wherein, the first target distance threshold is determined based on the sum of the vehicle movement distances corresponding to each target image frame.

[0039] The difference between the range change and the corresponding vehicle movement distance for each target image frame is less than the distance difference threshold.

[0040] For each target image frame, the change in distance and the corresponding vehicle movement distance are either both positive or both negative.

[0041] Based on the aforementioned technical means, when the ranging changes and corresponding vehicle movement distances corresponding to multiple target image frames satisfy the target confidence condition, the ranging confidence state of the first target is determined as the first state characterizing the reliability of the second ranging distance. Thus, on the one hand, the reliability of the sum of the ranging changes corresponding to each target image frame can be determined based on the sum of the ranging changes and the sum of the corresponding vehicle movement distances; on the other hand, the reliability of the value of the ranging change in each target image frame can be determined based on the difference between the ranging change and the corresponding vehicle movement distance; furthermore, the rationality of the direction of the ranging change in each target image frame can be determined based on whether the signs of the ranging change and the corresponding vehicle movement distance are consistent. In this way, the accuracy of the ranging confidence state of the first target can be improved, while flexibly adapting to various ranging scenarios and needs.

[0042] In some embodiments, the above ranging method further includes:

[0043] When the ranging confidence state of the first target indicates that the second ranging distance is reliable, the third ranging distance between the first target and the vehicle in the third image frame is determined based on the vehicle's movement distance between the acquisition interval of the third image frame and the second image frame, and the second ranging distance; wherein the second image frame and the third image frame are acquired sequentially by the image acquisition device during the vehicle's movement.

[0044] Based on the aforementioned technical means, when the ranging confidence state of the first target indicates that the second ranging distance is reliable, the third ranging distance between the first target and the vehicle in the third image frame is determined based on the vehicle's movement distance between the acquisition interval of the third image frame and the second image frame, as well as the second ranging distance. The second and third image frames are acquired sequentially by the image acquisition device during the vehicle's movement. In this way, while ensuring the ranging confidence state of the first target is reliable, the workload of ranging the first target in the third image frame acquired after the second image frame is reduced, thereby improving ranging efficiency while enhancing the ranging accuracy of the first target.

[0045] In some implementations, obtaining the second ranging distance between the first target and the vehicle in the second image frame includes:

[0046] Based on the extrinsic parameters of the image acquisition device and the coordinates of the detection box of the first target in the second image frame, the first predicted size is determined; and

[0047] The second ranging distance is determined based on the first predicted size, the detection box size of the first target in the second image frame, and the detection box coordinates of the first target in the second image frame.

[0048] Based on the aforementioned technical means, a first predicted size is determined using the extrinsic parameters of the image acquisition device and the coordinates of the detection box of the first target in the second image frame. Then, a second ranging distance is determined based on the first predicted size, the size of the detection box of the first target in the second image frame, and the coordinates of the detection box of the first target in the second image frame. This improves the accuracy of the obtained second ranging distance.

[0049] In some implementations, the first target includes a first cone, the second target includes a second cone, and the first predicted size is determined based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the first image frame, including:

[0050] Based on the external parameters of the image acquisition device and the coordinates of the detection box of the first cone in the first image frame, the first candidate size of the first cone is determined.

[0051] Determine the aspect ratio of the detection box of the first cone in the second image frame;

[0052] From the preset set of cone sizes, determine the second candidate size corresponding to the aspect ratio of the detection frame; and

[0053] The first predicted size is determined based on the first candidate size and the second candidate size.

[0054] Based on the aforementioned technical means, a first candidate size for the first cone is determined based on the extrinsic parameters of the image acquisition device and the coordinates of the detection box of the first cone in the first image frame; the aspect ratio of the detection box of the first cone in the second image frame is determined; then, a second candidate size corresponding to the aspect ratio of the detection box is determined from a preset set of cone sizes; and a first predicted size is determined based on the first and second candidate sizes. In this way, the rationality and accuracy of the first predicted size of the first cone can be improved by using multiple candidate sizes.

[0055] In some implementations, the first predicted size is determined based on the first candidate size and the second candidate size, including one of the following:

[0056] If the difference between the first candidate size and the second candidate size is greater than the size difference threshold, the second candidate size is determined as the first predicted size.

[0057] If the difference between the first candidate size and the second candidate size is not greater than the size difference threshold, the first candidate size is determined as the first predicted size.

[0058] Based on the aforementioned technical means, a first predicted size is determined from the first and second candidate sizes based on a size difference threshold and the magnitude of the difference between the first and second candidate sizes. Thus, if the difference between the first and second candidate sizes exceeds the size difference threshold, the first candidate size is considered unreliable, and the second candidate size from a more reliable set of preset target sizes is selected as the first predicted size. If the difference is not greater than the size difference threshold, the first candidate size is considered relatively reliable, and the first candidate size determined based on the first image frame, which is more suitable for the scene, is selected as the first predicted size. This improves the accuracy and reasonableness of the first predicted size.

[0059] In some embodiments, the number of first targets is multiple; before determining the ranging confidence state of the first targets based on the first ranging distance and the second ranging distance, the above ranging method further includes:

[0060] Based on the distribution information of each target detected in the second image frame, the layout type of the targets in the second image frame is determined;

[0061] Based on the first ranging distance and the second ranging distance, the ranging confidence state of the first target is determined, including:

[0062] When the layout type is dense arrangement, for each first target, the ranging confidence state of the first target is determined based on the first ranging distance and the second ranging distance corresponding to the first target.

[0063] Based on the aforementioned technical means, the distribution information of each target detected in the second image frame is used to determine the layout type of the targets in the second image frame. Furthermore, in the case of a densely arranged layout, for each first target, the ranging confidence state of the first target is determined based on the corresponding first and second ranging distances. This allows for the analysis of the ranging confidence states of multiple first targets in densely arranged scenarios. Therefore, based on the first predicted size of multiple first targets with reliable ranging, a more accurate second predicted size can be determined, thereby improving the accuracy of ranging the second targets.

[0064] In some implementations, the difference between the ranging change corresponding to the target image frame and the vehicle movement distance corresponding to the target image frame is less than a target difference threshold; the target image frame includes a second image frame or a historical image frame acquired before the second image frame; the ranging change corresponding to the target image frame is determined based on the difference between the first historical ranging distance of the first target in the target image frame and the second historical ranging distance of the first target in the previous frame of the target image frame; and the vehicle movement distance corresponding to the target image frame is the movement distance of the vehicle within the acquisition interval between the target image frame and the previous frame of the target image frame.

[0065] According to the above technical means, the difference between the ranging change corresponding to the target image frame and the vehicle movement distance corresponding to the target image frame is less than the target difference threshold, indicating that the ranging of each first target is more reliable. Therefore, a more accurate second prediction size can be determined based on the first prediction size of multiple first targets with more reliable ranging, so as to improve the accuracy of ranging of the second target.

[0066] In some embodiments, the above ranging method further includes:

[0067] If the ranging confidence state of the first target indicates that the second ranging distance is unreliable, then acquire the third image frame;

[0068] In the case where the third ranging distance between multiple third targets and the vehicle in the third image frame is less than the target distance threshold, the third predicted size of the multiple third targets is determined; and

[0069] Based on the third predicted size and the detection box information of the fourth target in the third image frame, the second target ranging distance between the fourth target and the vehicle in the third image frame is determined.

[0070] Based on the aforementioned technical means, when the ranging confidence state of the first target indicates that the second ranging distance is unreliable, a third image frame is acquired. If the third ranging distance between multiple third targets and the vehicle in the third image frame is less than a target distance threshold, the third predicted size of the multiple third targets is determined. Based on the third predicted size and the detection box information of the fourth target in the third image frame, the second target ranging distance between the fourth target and the vehicle in the third image frame is determined. Thus, predicting the fourth target based on the third targets that are closer to the vehicle can improve the accuracy of ranging the fourth target even when the first target is unreliable.

[0071] In some embodiments, the above ranging method further includes:

[0072] Determine the relative positional relationship between the lane line points and the second target in the second image frame, and the lane distance between the lane line points and the vehicle in the second image frame; and

[0073] Update the distance to the first target based on the relative position and lane distance.

[0074] Based on the aforementioned technical means, the relative positional relationship between the lane line points and the second target in the second image frame, as well as the lane distance between the lane line points and the vehicle in the second image frame, are determined. Then, based on the relative positional relationship and the lane distance, the distance to the first target is updated. In this way, by using the position of the lane line points in the second image frame, the distance measurement error of the second target can be reduced, and the realism and accuracy of the distance measurement of the first target can be improved.

[0075] A ranging device, comprising:

[0076] The first acquisition station is configured to acquire a first ranging distance between a first target and a vehicle in a first image frame, and a second ranging distance between the first target and a vehicle in a second image frame; wherein the first image frame and the second image frame are acquired sequentially by the image acquisition device mounted on the vehicle during the vehicle's movement;

[0077] The first determining station is configured to determine the ranging confidence state of the first target based on the first ranging distance and the second ranging distance;

[0078] The second determining station is configured to, when the ranging confidence state of the first target indicates that the second ranging distance is reliable, determine the second predicted size of the second target based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target; wherein the second ranging distance is determined based on the first predicted size; and

[0079] The third determining station is configured to determine the first target ranging distance between the second target and the vehicle in the second image frame based on the second predicted size and the detection box information of the second target in the second image frame.

[0080] Based on the aforementioned technical means, a second predicted size of the second target is calculated based on the first predicted size of the first target, which is considered reliable in its ranging confidence state, and the size relationship between the first target and the second target. Then, the ranging distance between the second target and the vehicle is calculated based on the second predicted size. Since the second ranging distance of the first target is reliable, and this second ranging distance is determined based on the first predicted size, the first predicted size can also be considered reliable. Therefore, based on the first predicted size and the size relationship between the first and second targets, a more accurate second predicted size can be calculated. This allows for a more accurate calculation of the ranging distance between the second target and the vehicle, reducing the ranging error of the second target.

[0081] An electronic device includes a memory and a processor. The memory stores computer instructions that can be executed on the processor, and the processor executes the instructions to implement the above-described ranging method.

[0082] A vehicle including the aforementioned electronic equipment.

[0083] A computer-readable storage medium storing computer instructions that can be executed by a processor to implement the above-described ranging method.

[0084] A computer program product includes computer instructions that, when executed by a processor, implement some or all of the steps in the above-described ranging method.

[0085] The beneficial effects of the embodiments disclosed herein are as follows:

[0086] Based on the reliable first predicted size and the size relationship between the first target and the second target, a more accurate second predicted size can be calculated. Thus, the distance between the second target and the vehicle can be calculated more accurately based on the second predicted size, reducing the distance measurement error of the second target.

[0087] If the first target includes a first cone, the second target includes a second cone, and the size relationship indicates that the first and second cones have the same size specifications, then the first predicted size is determined as the second predicted size. This improves the overall ranging efficiency and allows for timely and accurate completion of the ranging task, while also enhancing the ranging confidence level of the distance measured from the first target.

[0088] Based on improving the accuracy of longitudinal ranging values, the accuracy of lateral ranging values ​​determined from longitudinal ranging values ​​was also improved, thus increasing the accuracy of the first target ranging distance.

[0089] When the first target includes a first cone and the second target includes a second cone, the coordinates of a distance measurement between the second cone and the vehicle are determined based on the coordinates of multiple edge points of the second cone, thereby improving the accuracy of the distance measurement results.

[0090] By comprehensively considering the difference in the ranging distance of the first target in different image frames and the vehicle movement distance within the corresponding image frame acquisition interval, the accuracy of determining the ranging confidence state of the first target is improved.

[0091] While improving the accuracy of the ranging confidence state of the primary target, it flexibly adapts to various ranging scenarios and needs;

[0092] If the ranging confidence state of the first target is reliable, the ranging workload of the first target in the third image frame acquired after the second image frame is reduced, thereby improving the ranging efficiency while improving the ranging accuracy of the first target.

[0093] When the first objective includes a first cone and the second objective includes a second cone, the rationality and accuracy of the first predicted size of the first cone are improved based on multiple candidate sizes.

[0094] If the difference between the first candidate size and the second candidate size is greater than the size difference threshold, the second candidate size in the preset target size set is determined as the first predicted size; if the difference is not greater than the size difference threshold, the first candidate size determined based on the first image frame that is more suitable for the scene is determined as the first predicted size, which can improve the accuracy and rationality of the first predicted size.

[0095] In scenarios where targets are densely clustered, the ranging confidence state of multiple first targets is analyzed. Based on the first predicted size of multiple first targets with reliable ranging, a more accurate second predicted size can be determined, thereby improving the accuracy of ranging the second targets.

[0096] A more accurate second prediction size is determined based on the first prediction size of multiple first targets with more reliable ranging, so as to improve the accuracy of ranging of the second target;

[0097] In cases where the first target is unreliable, predicting the fourth target based on a third target that is closer to the vehicle can improve the accuracy of ranging the fourth target.

[0098] Based on the position of the lane line points in the second image frame, the ranging error of the second target can be reduced, and the realism and accuracy of the ranging distance of the first target can be improved.

[0099] After reading and understanding the accompanying diagrams and detailed descriptions, the other aspects can be understood. Attached Figure Description

[0100] Figure 1A is a schematic diagram of the implementation process of a ranging method provided in an embodiment of this disclosure;

[0101] Figure 1B is a schematic diagram of an external parameter mesh provided in an embodiment of this disclosure;

[0102] Figure 1C is a schematic diagram of a vehicle coordinate system provided in an embodiment of this disclosure;

[0103] Figure 1D is a schematic diagram of the bounding box coordinates of a target provided in an embodiment of this disclosure;

[0104] Figure 1E is a schematic diagram of a pinhole imaging ranging principle provided in an embodiment of this disclosure;

[0105] Figure 1F is a structural schematic diagram of the bottom edge point of a cone provided in an embodiment of this disclosure;

[0106] Figure 1G is a schematic diagram of a scenario showing the positional relationship between a lane line and a target according to an embodiment of this disclosure;

[0107] Figure 1H is a coordinate diagram illustrating the positional relationship between a lane line and a target according to an embodiment of this disclosure;

[0108] Figure 2 is a schematic diagram of the implementation process of a cone distance measurement method provided in an embodiment of this disclosure;

[0109] Figure 3 is a schematic diagram of the composition structure of a ranging device provided in an embodiment of this disclosure;

[0110] Figure 4 is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0111] The embodiments of this application will be described below with reference to the accompanying drawings and examples. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the embodiments disclosed herein. The embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0112] The illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0113] In the process of implementing the embodiments of this disclosure, it was found that in intelligent driving, for distant targets, due to reasons such as vehicle movement, target obstruction, or small calculation range of calibration extrinsic parameters at a distance, the distance measurement calculation error usually fluctuates greatly, so the distance measurement result is relatively unreliable.

[0114] When autonomous vehicles measure the longitudinal distance to traffic cones using monocular vision, they often rely on pre-calibration and use extrinsic parameters of the vision system for measurement. This method is relatively accurate when the cones are close to the vehicle. However, when the cones are far from the vehicle, due to systematic errors such as road bumps, the ranging accuracy of this method cannot fully meet the needs of practical applications.

[0115] Monocular vision ranging first performs continuous multi-frame measurements, then optimizes the ranging accuracy using a specific filter to obtain the final ranging result. This method is relatively reliable when the display status of nearby targets is clear and stable. However, for distant targets, due to target movement or occlusion, and the smaller calculation range of extrinsic parameters at distances, the calculation error fluctuates significantly, hindering measurement convergence and making the ranging results relatively unreliable.

[0116] Based on this, the present disclosure proposes a ranging method that can be executed by an electronic device. The electronic device refers to a server, laptop, tablet, desktop computer, smart TV, set-top box, mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device), or other device with data processing capabilities. As shown in Figure 1A, the ranging method includes the following steps S11 to S14:

[0117] Step S11: Obtain the first ranging distance between the first target and the vehicle in the first image frame and the second ranging distance between the first target and the vehicle in the second image frame; the first image frame and the second image frame are acquired sequentially by the image acquisition device mounted on the vehicle during the vehicle's movement.

[0118] Here, the image acquisition device can be a device capable of acquiring information and generating images, such as at least one of a camera, infrared camera, thermal imager, depth camera, etc.

[0119] The first image frame is acquired before the second image frame is acquired.

[0120] The first target refers to the detectable targets in the first and second image frames, which may include static objects in the scene where the vehicle is located. For example, targets may include, but are not limited to, obstacles, green facilities, roadside facilities, etc. Among them, obstacles may include, but are not limited to, traffic cones, water-filled barriers, crash barriers, speed bumps, buildings, etc.

[0121] In some implementations, the first target may include a first obstacle, and the second target may include a second obstacle.

[0122] For example, the first objective may include a first cone, and the second objective may include a second cone.

[0123] The distance measurement can include the lateral and longitudinal distances between the target and the vehicle, which are determined by measurement.

[0124] In some implementations, the first ranging distance between the first target and the vehicle in the first image frame and the second ranging distance between the first target and the vehicle in the second image frame can be obtained by acquiring prior parameters, internal parameters of the vision system, and external parameters.

[0125] The prior parameters may include common dimensions of the target, such as, but not limited to, the aspect ratio of a cone, the width and height of a water-filled barrier, the length of a speed bump, and the height of a building. The vision system includes the image acquisition equipment mounted on the vehicle. The internal parameters of the vision system may include the parameters of the image acquisition equipment, such as, but not limited to, focal length, principal point coordinates, and distortion coefficients. The external parameters of the vision system may be parameters that characterize the pose information of the image acquisition equipment, such as rotation matrices and translation vectors.

[0126] In some implementations, the vehicle's internal and external parameters can be calibrated in advance. Based on the calibrated internal and external parameters, a first ranging distance between the first target and the vehicle in the first image frame and a second ranging distance between the first target and the vehicle in the second image frame are obtained. In this way, the obtained calibrated internal parameters can correct distorted images or points, reducing errors; the obtained calibrated external parameters can be used to calculate the grid display in the image frame, improving the accuracy of target object position information measurement in the image frame and enhancing the display effect.

[0127] As shown in Figure 1B, the grid 11 calculated using external parameters is displayed in image frame 10. In this way, the position information of the target object in image frame 10, such as the lateral or longitudinal distance between it and the vehicle, can be obtained more accurately based on the grid 11.

[0128] In some implementations, a vehicle coordinate system can be constructed, and the distance between the first target and the vehicle can be determined based on the coordinates of the first target in the vehicle coordinate system.

[0129] For example, as shown in Figure 1C, a vehicle coordinate system (VCS) can be constructed, with the projection point O of the rear axle center of the vehicle onto the ground as the origin of the coordinate system, and the positive directions of the x-axis, y-axis and z-axis of the coordinate system pointing to the front, left and top of the vehicle, respectively.

[0130] Step S12: Determine the ranging confidence state of the first target based on the first ranging distance and the second ranging distance.

[0131] Here, the ranging confidence state can characterize the confidence level of the ranging distance to the first target shown.

[0132] In some implementations, the ranging confidence state of the first target can be determined based on the difference between the first ranging distance and the second ranging distance.

[0133] For example, if the difference between the first ranging distance and the second ranging distance is greater than a preset difference threshold, the ranging confidence state of the first target can be determined as unreliable.

[0134] In some implementations, the ranging confidence state of the first target can be determined based on the difference between the first ranging distance and the second ranging distance, and the vehicle movement distance corresponding to the acquisition interval between the first image frame and the second image frame.

[0135] For example, if the difference between the first ranging distance and the second ranging distance is equal to or approximately equal to the vehicle movement distance corresponding to the acquisition interval between the first image frame and the second image frame, the ranging confidence state of the first target can be determined as reliable.

[0136] In some implementations, the vehicle movement distance corresponding to the acquisition interval between the first and second image frames can be determined based on the vehicle speed and time corresponding to the first and second image frames, respectively.

[0137] In some implementations, the vehicle travel distance can be the product of the average vehicle speed corresponding to the first image frame and the second image frame, and the time corresponding to the acquisition interval between the first image frame and the second image frame.

[0138] For example, the vehicle speed corresponding to the first image frame is speed0, and the vehicle speed corresponding to the second image frame is speed1. The timestamp recorded for the first image frame is time0, and the timestamp recorded for the second image frame is time1. Then, the average speed of the vehicle corresponding to the first image frame and the second image frame is speed0. agv It can be (speed0 + speed1) / 2, where time is the acquisition interval between the first and second image frames. span The time range can be from time1 to time0, and the vehicle's travel distance can be speed. agv *time span .

[0139] Step S13: If the ranging confidence state of the first target indicates that the second ranging distance is reliable, the second predicted size of the second target is determined based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target; the second ranging distance is determined based on the first predicted size.

[0140] Here, the second target is a target detected in the second image frame that is different from the first target.

[0141] The dimensional relationship between the first target and the second target can be considered as a mapping relationship between the dimensions of the first target and the dimensions of the second target.

[0142] For example, the size relationship between the first target and the second target may include, but is not limited to, one of the following: the first target and the second target are exactly the same size, the first target and the second target have partially the same size, or the first target and the second target are proportional to each other.

[0143] The first predicted size can be the actual size of the first target obtained through measurement.

[0144] The second predicted size can be the actual size of the second target obtained through measurement.

[0145] In some implementations, the first predicted size can be determined based on the detection bounding box information of the first target in the first image frame and the position information of the first target in the vehicle coordinate system obtained by external parameter transformation.

[0146] The detection box information may include the bounding box information of the first target in the first image frame; the bounding box information may include the position, size, etc. of the bounding box.

[0147] For example, the bounding box coordinates of the first target in the first image frame can be (M top M left M right M bottom ), where M top M left M right and M bottom These represent the top, left, right, and bottom coordinates of the first target bounding box. Using the intrinsic and extrinsic parameters, and employing Zhang Zhengyou's calibration formula, the corresponding extrinsic transformation matrix H can be obtained during calibration by combining multiple points with known horizontal and vertical VCS coordinates. cam_gnd This is used to convert points in the image coordinate system to VCS coordinates; where H cam_gnd =R*T; R is the rotation matrix, and T is the translation vector. The calibration formula for Zhang Zhengyou can be found in formula (1):

[0148] in, It is an extrinsic parameter matrix. This is the intrinsic parameter matrix.

[0149] For example, it can be achieved through the extrinsic transformation matrix H cam_gnd The coordinates Pt of the points in the image img Convert to coordinates Pt in the VCS coordinate system vcs , coordinate Pt img Convert to coordinates Pt vcs The method is described in formula (2): Pt vcs =H cam_gnd *Pt img(2).

[0150] The point l corresponding to the bottom left corner of the bounding box of the first target b And the corresponding point r in the lower right corner b Coordinates Pt in the first image frame img_lb and Pt img_rb They can be represented as (M) left M bottom ) and (M right M bottom ).

[0151] As shown in Figure 1D, the point l corresponding to the lower left corner of the bounding box of the first target 12 obtained through the extrinsic parameter transformation matrix. b And the point r corresponding to the lower right corner of the bounding box of the first target 12 b The VCS coordinates are Pt vcs_lb (x lb ,y lb ,z lb ) and Pt vcs_rb (x rb ,y rb ,z rb ).

[0152] In some implementations, the detection bounding box information of the target in the first image frame can be determined by the target model based on the first image frame.

[0153] For example, a U-Net neural network (a convolutional neural network for image segmentation) model can be pre-built. The model can obtain the bounding box information of all targets in the image frame and determine the VCS coordinates of all targets based on the bounding box information of all targets.

[0154] Step S14: Based on the second predicted size and the detection box information of the second target in the second image frame, determine the first target ranging distance between the second target and the vehicle in the second image frame.

[0155] Here, the first target ranging distance may include the lateral distance and longitudinal distance between the second target and the vehicle in the second image frame.

[0156] In some implementations, the first target ranging distance between the second target and the vehicle in the second image frame can be determined by utilizing the pinhole imaging principle, based on the detection frame of the second target imaged in the second image frame and the actual size of the second target, i.e., the second predicted size.

[0157] It is understandable that if the second ranging distance is reliable, the first predicted size is reliable. Furthermore, since the second predicted size is determined based on the first predicted size, the second predicted size can also be determined to be reliable. Therefore, the first target ranging distance determined based on the reliable second predicted size is reliable.

[0158] In this embodiment, based on the first predicted size of a first target whose ranging confidence state is credible, and the size relationship between the first target and the second target, the second predicted size of the second target is calculated. Then, the ranging distance between the second target and the vehicle is calculated based on the second predicted size. Since the second ranging distance of the first target is credible, and the second ranging distance of the first target is determined based on the first predicted size, the first predicted size can also be considered credible. Therefore, based on the first predicted size and the size relationship between the first target and the second target, a more accurate second predicted size can be calculated. Thus, the ranging distance between the second target and the vehicle can be calculated more accurately based on the second predicted size, reducing the ranging error of the second target.

[0159] It is understandable that, since the perception accuracy of near targets is usually greater than that of far targets, the ranging method in the above embodiments can improve the situation where the ranging accuracy cannot fully meet the actual application scenario when the target is far away from the vehicle due to systematic errors such as road bumps. That is, in multi-target scenarios, the position of far targets or nearby targets can be estimated from the position of near targets, thereby improving the ranging accuracy of targets near far targets or near near targets.

[0160] In some embodiments, determining the second predicted size of the second target based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target, in step S11 above may include the following step S111:

[0161] Step S111: When the size relationship characterizes the first target and the second target with the same size specifications, the first predicted size is determined as the second predicted size; wherein the first predicted size and the second predicted size correspond to the same size parameter, and the size parameter may include a height parameter and / or a width parameter.

[0162] Here, since the first target and the second target have the same size in the second image frame, and the first predicted size is reliable when the ranging confidence state of the first target represents the reliability of the second ranging distance, the ranging distance of the first target can be determined directly based on the second predicted size which is the same as the first predicted size. This can improve ranging efficiency by improving the ranging confidence state of the first target ranging distance.

[0163] For example, if all targets in the second image frame have the same size, the ranging distance of all targets can be determined based on a predicted size.

[0164] In this embodiment of the disclosure, when the size relationship characterizes the first target and the second target as having the same size specifications, the first predicted size is determined as the second predicted size. This improves the overall ranging efficiency and allows for timely and accurate completion of the ranging task, while also enhancing the ranging confidence level of the first target's ranging distance.

[0165] In some embodiments, the first target may include a first cone, the second target may include a second cone, and step S111 above may include the following step S1111:

[0166] Step S1111: When the size relationship characterizes the first cone and the second cone with the same size specifications, the first predicted size is determined as the second predicted size; wherein the first predicted size and the second predicted size correspond to the same size parameter, and the size parameter may include the height parameter and / or the width parameter.

[0167] In this embodiment, when the dimensions of the first cone and the second cone are identical, the first predicted dimension is determined as the second predicted dimension. This improves the overall ranging efficiency and allows for timely and accurate completion of the ranging task, while also enhancing the ranging confidence level of the first target ranging distance.

[0168] In some embodiments, the detection frame information includes the detection frame size and the detection frame position, and step S14 may include steps S141 to S143:

[0169] Step S141: Based on the intrinsic parameters of the image acquisition device, the second predicted size, and the detection box size of the second target in the second image frame, determine the longitudinal distance between the second target and the vehicle in the second image frame.

[0170] Here, the intrinsic parameters can correspond to the intrinsic parameters in the distance measurement method mentioned above.

[0171] As shown in Figure 1E, taking a cone as the target, the distance relationship between the real cone 21 in the scene and the pixel cone 22 corresponding to cone 21 in the imaging plane can be obtained using the pinhole imaging ranging principle. Where, O C f is the focal point of the image acquisition device. x For pixel cone 22 and focus O C The vertical focal length between them, W world D is the actual width of cone 21. world For cone 21 and focus O C The actual longitudinal distance between them; D is determined using the properties of similar triangles.world The method can be found in formula (3): D world =W world *f x / W img (3);

[0172] Among them, W img This represents the pixel width of the cone.

[0173] That is, the distance D of the cone in the x-axis direction of the vehicle coordinate system is obtained. world The corresponding intrinsic distance measurement value dist in .

[0174] It is understandable that W in Figure 1E world This can be equated to the predicted target size in the ranging method described above, W. img This can be represented by the target's bounding box information. Therefore, based on the target's predicted size and the bounding box information, the actual distance D between the target and the focal point of the image acquisition device can be determined. world This refers to the distance between the target and the vehicle.

[0175] If the second ranging distance is reliable, the first predicted size is reliable. Furthermore, since the second predicted size is determined based on the first predicted size, the second predicted size can also be considered reliable. Therefore, based on the above formula (3), in W... img The detection bounding box information corresponding to the second target, and W world Given a reliable second predicted size for the second target, the determined range for the first target is reliable.

[0176] Step S142: Based on the intrinsic parameters, the detection box position of the second target in the second image frame, and the longitudinal ranging distance, determine the lateral ranging distance between the second target and the vehicle in the second image frame.

[0177] In some implementations, the translation transformation matrix between the CCD center of the image acquisition device and the coordinate system determined based on the vehicle can be determined according to the installation location and internal parameters of the image acquisition device.

[0178] For example, the translation transformation matrix can be represented as T Cam2Vcs Based on the translation transformation matrix, the transformation relationship between any point in the image frame and the coordinate system determined based on the vehicle can be determined.

[0179] For example, the coordinates of a point in an image frame in the image coordinate system can be represented as Pt. img The coordinates after conversion to the VCS coordinate system can be represented as Pt. vcs Pt vcsThe method for determining Pt can be found in formula (4): vcs =Pt img *T Cam2Vcs (4).

[0180] In some implementations, the position of the CCD center in the image frame can be obtained based on the calibration parameters of the image acquisition device, and the position of the target relative to the CCD center can be determined.

[0181] For example, the coordinates of the center of the camera CCD in the image frame are Cam img (x ci ,y ci In the case of ), determine the lateral distance coordinate value y_wid_dist of the target relative to the center of the CCD. img The method can be found in formula (5): y_wid_dist img =x ci -M left (5).

[0182] Using the pinhole imaging ranging principle and the property of similar triangles, the lateral distance y_dist between the target and the vehicle is determined. in The method can be found in formula (6): y_dist in =dist in *y_wid_dist img / f y (6);

[0183] Among them, y_dist in f is the lateral distance measurement between the target and the vehicle, i.e., the distance measurement relative to the vehicle's y-axis direction; y This is the focal length in the height direction.

[0184] Step S143: Determine the first target distance based on the longitudinal and lateral distances.

[0185] In some implementations, the longitudinal ranging distance and / or the lateral ranging distance can be determined as the first target ranging distance, or the first target ranging distance can be calculated based on the longitudinal ranging distance and / or the lateral ranging distance.

[0186] In this embodiment, based on the intrinsic parameters of the image acquisition device, the longitudinal and lateral distances between the second target and the vehicle in the second image frame are determined respectively; and the distance to the first target is determined based on the longitudinal and lateral distances. This improves the accuracy of the longitudinal distance value, enhances the accuracy of the lateral distance value determined from the longitudinal distance value, and improves the accuracy of the distance to the first target.

[0187] It is understandable that by combining intrinsic and extrinsic parameters for distance measurement, the accuracy of distance measurement can be improved by using only extrinsic parameters, which is affected by road bumps, and by using only intrinsic parameters to directly estimate the cone size, which can lead to excessively large jumps in distance measurement results.

[0188] In some embodiments, step S143 may include steps S1431 to S1433:

[0189] Step S1431: Based on the longitudinal and lateral distance measurements, determine the coordinates of multiple edge points at the bottom of the second target.

[0190] Here, the edge point at the bottom of the second target can be a point on the outer ring of the bottom of the second target.

[0191] In some implementations, the first target may include a first cone, the second target may include a second cone, and step S1431 may include the following step S14311:

[0192] Step S14311: Based on the longitudinal and lateral distance measurements, determine the coordinates of multiple edge points at the bottom of the second cone.

[0193] As shown in Figure 1F, taking the second cone as the second target as an example, the longitudinal distance dist can be used for measurement. in Determine the x-axis coordinates of the midpoint A of the bottom edge of the bounding box of the second cone in the second image frame in the vehicle coordinate system, based on the lateral distance y_dist. in Determine the y-axis coordinate of the midpoint of the bottom edge of the second cone, and obtain the VCS coordinate cb of that point. ctr_vcs (dist in ,y_dist in Similarly, the width value of the second predicted dimension of the corresponding second cone is W. world In this case, the coordinates of the lower left point B, lower right point C, and the far radius point D of the second cone's bounding box in the VCS coordinate system can be obtained respectively. b_vcs (dist in ,y_dist in -W world / 2,0), l r_vcs (distin ,y_dist in +W world / 2,0) and ct ctr_vcs (dist in +W world / 2,y_dist in ,0).

[0194] Step S1432: Determine the coordinates of the center point of the bottom of the second target based on the coordinates of multiple edge points.

[0195] In some implementations, the coordinates of the center point of the bottom of a second target can be calculated based on the coordinates of multiple edge points.

[0196] For example, the average of the coordinates of multiple edge points can be used to determine the coordinates of the center point at the bottom of the second target.

[0197] In some implementations, the first target may include a first cone, the second target may include a second cone, and step S1432 may include the following step S14321:

[0198] Step S14321: Determine the coordinates of the center point of the bottom of the second cone based on the coordinates of multiple edge points.

[0199] For example, the average of the coordinates of the four points A, B, C, and D in Figure 1F can be used as the coordinate value of the center point M.

[0200] In some implementations, the center point coordinates of the bottom of the second target can be determined using a filter based on the coordinates of multiple edge points.

[0201] For example, points A, B, C, and D in Figure 1F can be taken as the edge points of the bottom of the second cone in the VCS coordinate system. Using an Extended Kalman Filter (EKF), the coordinates of the filtered edge point Edgs of the second cone in the VCS coordinate system can be obtained. i =(x i ,y i ,z i ), where (i = A, B, C, D). Based on the coordinates of the four filter edge points, the VCS coordinates of the center point M of the second cone are determined by averaging. vcs The method can be found in formula (7):

[0202] Step S1433: Determine the distance to the first target based on the coordinates of the center point.

[0203] Here, the distance to the first target can be determined based on the coordinates of the center point of the second target.

[0204] For example, the ordinate of the center point of the second cone can be determined as the first target ranging distance characterizing the longitudinal ranging distance.

[0205] For example, the x-coordinate of the center point of the second cone can be determined as the first target ranging distance representing the lateral ranging distance.

[0206] In this embodiment, the coordinates of multiple edge points at the bottom of the second target are determined based on the longitudinal and lateral distances; the coordinates of the center point at the bottom of the second target are then determined based on the coordinates of the multiple edge points; and the distance to the first target is determined based on the coordinates of the center point. In this way, the coordinates for measuring the distance between the second target and the vehicle can be determined based on the coordinates of multiple edge points of the second target, improving the accuracy of the distance measurement results.

[0207] Understandably, when the first target includes the first traffic cone and the second target includes the second traffic cone, the coordinates of multiple edge points at the bottom of the second traffic cone are determined based on the longitudinal and lateral distance measurements. Then, based on these edge point coordinates, the coordinates of the center point at the bottom of the second traffic cone are determined. Finally, based on the center point coordinates, the distance to the first target is determined. In this way, the coordinates of the distance between the second traffic cone and the vehicle can be determined based on the coordinates of multiple edge points of the second traffic cone, improving the accuracy of the distance measurement results.

[0208] In some embodiments, step S12 may include steps S121 to S122:

[0209] Step S121: Based on the difference between the first ranging distance and the second ranging distance, determine the ranging change corresponding to the second image frame.

[0210] Here, based on the difference between the first ranging distance and the second ranging distance, the change in distance between the first target and the vehicle within the acquisition interval between the first image frame and the second image frame can be determined, that is, the ranging change corresponding to the second image frame.

[0211] Step S122: Determine the ranging confidence state of the first target based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame; the vehicle movement distance corresponding to the second image frame is the movement distance of the vehicle within the acquisition interval between the first image frame and the second image frame.

[0212] In some implementations, the ranging confidence state of the first target can be determined based on the difference between the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame.

[0213] It is understandable that the change in distance between the first target and the vehicle within the acquisition interval between the first and second image frames is equal to or approximately equal to the distance the vehicle moves within the acquisition interval between the first and second image frames.

[0214] In some implementations, the reliability or unreliability of the ranging result can be determined as the ranging confidence state of the first target.

[0215] In some implementations, a convergence flag can be set to characterize whether the ranging result of the first target has converged; based on whether the ranging result of the first target has converged, the ranging confidence state of the first target can be determined.

[0216] For example, if the convergence flag corresponding to the first target is set to true, it is determined that the ranging result of the first target has converged and the ranging result is reliable; if the convergence flag corresponding to the first target is set to false, it is determined that the ranging result of the first target has not converged and the ranging result is unreliable.

[0217] In this embodiment, the ranging change corresponding to the second image frame is determined based on the difference between the first ranging distance and the second ranging distance; and the ranging confidence state of the first target is determined based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame. Thus, by comprehensively considering the differences in the ranging distances of the first target in different image frames and the vehicle movement distance within the corresponding image frame acquisition interval, the accuracy of determining the ranging confidence state of the first target can be improved.

[0218] In some embodiments, the first image frame is the frame preceding the second image frame, and step S122 may include steps S1221 to S1222:

[0219] Step S1221: Determine the ranging change and the corresponding vehicle movement distance for at least one historical image frame. The ranging change corresponding to the historical image frame is determined based on the difference between the first historical ranging distance of the first target in the historical image frame and the second historical ranging distance of the first target in the previous frame of the historical image frame. The vehicle movement distance corresponding to the historical image frame is the movement distance of the vehicle within the acquisition interval between the historical image frame and the previous frame of the historical image frame.

[0220] Here, the historical image frame may include the first image frame.

[0221] In some implementations, the ranging change in a historical image frame can be determined based on the difference between the center point coordinates of the first target corresponding to the first historical ranging distance in the historical image frame and the center point coordinates of the first target corresponding to the second historical ranging distance in the previous frame of the historical image frame.

[0222] Step S1222: Determine the ranging confidence state of the first target based on the ranging change and the corresponding vehicle movement distance corresponding to the multiple target image frames; the multiple target image frames include at least one historical image frame and a second image frame.

[0223] In some implementations, the ranging confidence state of the first target can be determined based on the difference between the ranging change corresponding to each target image frame and the corresponding vehicle movement distance.

[0224] For example, if the difference between the distance change corresponding to each target image frame and the corresponding vehicle movement distance is not greater than the difference threshold, the confidence state of the target image frame can be determined as reliable; if the ratio between the number of reliable image frames and the total number of image frames in multiple target image frames is not less than the ratio threshold, the distance confidence state of the first target can be determined as reliable.

[0225] In this embodiment of the disclosure, the ranging confidence state of the first target is determined based on the ranging change and the corresponding vehicle movement distance corresponding to multiple target image frames; the multiple target image frames include at least one historical image frame and a second image frame. This allows for comprehensive consideration of the ranging change and the corresponding vehicle movement distance corresponding to each target image frame, improving the accuracy of determining the ranging confidence state of the first target.

[0226] In some embodiments, step S1222 may include step S12221:

[0227] Step S12221: When the ranging change and the corresponding vehicle movement distance corresponding to multiple target image frames meet the target confidence condition, determine the ranging confidence state of the first target as the first state characterizing the credibility of the second ranging distance.

[0228] The target confidence conditions include at least one of the following:

[0229] The sum of the ranging changes corresponding to each target image frame is less than the first target distance threshold; the first target distance threshold is determined based on the sum of the vehicle movement distances corresponding to each target image frame.

[0230] The difference between the range change and the corresponding vehicle movement distance for each target image frame is less than the distance difference threshold.

[0231] For each target image frame, the change in distance and the corresponding vehicle movement distance are either both positive or both negative.

[0232] Here, taking into account factors such as calculation errors, the first target distance threshold can be calculated based on the sum of the vehicle movement distances corresponding to each target image frame.

[0233] For example, the first target distance threshold may include the sum of the vehicle movement distances corresponding to each target image frame, or it may include 50% of the vehicle movement distances corresponding to each target image frame.

[0234] Simultaneously, the vehicle movement distance corresponding to each target image frame corresponds to the distance change corresponding to that target image frame. The difference between the distance change and the corresponding vehicle movement distance for each target image frame can characterize the distance measurement error for each target image frame. If the difference is less than the distance difference threshold, it is equivalent to the distance measurement error being less than the error threshold, meaning the distance measurement result is reliable.

[0235] Understandably, by retaining the signs of the ranging changes and the corresponding vehicle movement distances for each target image frame, the signs can represent the transformation direction for each target image frame. If both the ranging change and the corresponding vehicle movement distance for a target image frame are positive or both are negative, it can be determined that the ranging change and vehicle movement distance of the target image frame are a reasonable match. Therefore, the accuracy of determining the ranging confidence state of the first target can be improved.

[0236] In this embodiment, when the ranging changes and vehicle movement distances corresponding to multiple target image frames satisfy the target confidence condition, the ranging confidence state of the first target is determined as a first state characterizing the confidence of the second ranging distance. Thus, on the one hand, the reliability of the sum of the ranging changes corresponding to each target image frame can be determined based on the sum of the ranging changes and the sum of the vehicle movement distances corresponding to each target image frame; on the other hand, the reliability of the value of the ranging change in each target image frame can be determined based on the difference between the ranging change and the corresponding vehicle movement distance; furthermore, the rationality of the direction of the ranging change in each target image frame can be determined based on whether the signs of the ranging change and the corresponding vehicle movement distance are consistent. Therefore, while improving the accuracy of the ranging confidence state of the first target, it flexibly adapts to various ranging scenarios and requirements.

[0237] In some embodiments, the above ranging method may further include the following step S15:

[0238] Step S15: If the ranging confidence state of the first target indicates that the second ranging distance is reliable, based on the vehicle's movement distance between the acquisition interval of the third image frame and the second image frame, and the second ranging distance, determine the third ranging distance between the first target and the vehicle in the third image frame; the second image frame and the third image frame are acquired sequentially by the image acquisition device during the vehicle's movement.

[0239] Here, if the ranging confidence state of the first target represents the reliability of the second ranging distance, the ranging change between the first ranging distance and the second ranging distance is reliable, that is, the size of the first target and its relative positional relationship with the vehicle are reliable.

[0240] Based on this, when measuring the distance of the first target in the third image frame, which is acquired after the acquisition time of the second image frame, there is no need to recalculate based on the predicted size of the first target. The position of the first target in the third image frame can be determined directly based on the position of the first target in the second image frame and the distance the vehicle moves between the second and third image frames. That is, the third distance between the first target and the vehicle in the third image frame.

[0241] For example, the center point coordinates of the first target in the second image frame can be represented as center. last The acquisition interval between the second and third image frames can be expressed as time. span The average speed of the vehicle during the acquisition interval between the second and third image frames can be expressed as speed. agv The center point coordinates of the first target in the third image frame can be represented as center. output Determine the center output The method can be found in formula (8): center output =center last -speed agv *time span (8).

[0242] In this embodiment, when the ranging confidence state of the first target indicates that the second ranging distance is reliable, the third ranging distance between the first target and the vehicle in the third image frame is determined based on the vehicle's movement distance between the acquisition interval of the third image frame and the second image frame, and the second ranging distance; the second and third image frames are acquired sequentially by the image acquisition device during the vehicle's movement. In this way, when the ranging confidence state of the first target is reliable, the ranging workload for the first target in the third image frame acquired after the second image frame is reduced, thereby improving ranging efficiency while enhancing the ranging accuracy of the first target.

[0243] In some embodiments, obtaining the second ranging distance between the first target and the vehicle in the second image frame in step S11 above may include the following steps S112 to S113:

[0244] Step S112: Determine the first predicted size based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the second image frame.

[0245] Here, the extrinsic parameters correspond to the external parameters in the distance measurement method described above.

[0246] In some implementations, the detection coordinates of the first target in the second image frame can be transformed based on the extrinsic parameters of the image acquisition device to obtain the actual size of the first target in the vehicle-based coordinate system as the first predicted size.

[0247] For example, the difference between the lower right and lower left corners of the detection box of the first target in the second image frame along the y-axis in the vehicle coordinate system can be used to determine the width of the first target. ex .

[0248] Step S113: Determine the second ranging distance based on the first predicted size, the detection box size of the first target in the second image frame, and the detection box coordinates of the first target in the second image frame.

[0249] In some implementations, the longitudinal distance between the first target and the vehicle in the second image frame can be determined based on the intrinsic parameters of the image acquisition device, the first predicted size, and the detection box size of the first target in the second image frame; the lateral distance between the first target and the vehicle in the second image frame can be determined based on the intrinsic parameters, the detection box coordinates of the first target in the second image frame, and the longitudinal distance; and the second distance can be determined based on the longitudinal distance and the lateral distance.

[0250] In this embodiment, a first predicted size is determined based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the second image frame; then, a second ranging distance is determined based on the first predicted size, the detection box size of the first target in the second image frame, and the detection box coordinates of the first target in the second image frame. This improves the accuracy of the obtained second ranging distance.

[0251] In some embodiments, step S112 may include steps S1121 to S1124:

[0252] Step S1121: Based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the first image frame, determine the first candidate size of the first target.

[0253] Here, the first candidate size can be the actual size of the initially determined first target. For example, the width in step S112. ex .

[0254] In some embodiments, the first target includes a first cone, and the second target includes a second cone. Step S1121 may include the following step S11211:

[0255] Step S11211: Based on the external parameters of the image acquisition device and the detection box coordinates of the first cone in the first image frame, determine the first candidate size of the first cone.

[0256] Step S1122: Determine the aspect ratio of the detection box of the first target in the second image frame.

[0257] In some implementations, the pixel value of the detection box size of the first target in the first image frame can be obtained based on the detection box coordinates of the first target in the first image frame; and the aspect ratio of the detection box of the first target in the second image frame can be determined based on the obtained pixel value of the detection box size of the first target in the first image frame.

[0258] In some embodiments, the first target includes a first cone, and the second target includes a second cone. Step S1122 may include the following step S11221:

[0259] Step S11221: Determine the aspect ratio of the detection box of the first cone in the second image frame.

[0260] For example, the pixel value W of the width of the first cone img The coordinates M of the right side of the detection box in the first image frame can be determined by the first cone. right With the left coordinate value M left The difference, i.e., M right -M left Similarly, the pixel value H of the height of the first cone is obtained; img The coordinates M below the detection box in the first image frame can be determined by the first cone. bottom With the coordinate value M above top The difference between them, i.e., M bottom -M top Obtain; the aspect ratio of the detection box of the first cone in the second image frame. img_wh It can be represented as W img / H img .

[0261] Step S1123: Determine the second candidate size corresponding to the aspect ratio of the detection frame from the preset target size set.

[0262] Here, the preset target size set includes the actual size of at least one target, and the ratio between the width and height of at least one target can be obtained based on the actual size of the target.

[0263] In some implementations, a set of targets whose width and height are equal to or approximately equal to the aspect ratio of the detection frame can be determined from a preset set of target sizes as a second candidate size.

[0264] In some embodiments, the first target includes a first cone, and the second target includes a second cone. Step S1123 may include the following step S11231:

[0265] Step S11231: Determine the second candidate size corresponding to the width-to-height ratio of the detection frame from the preset set of cone sizes.

[0266] For example, the preset cone size set includes a first size combination with a width of 20.5cm and a height of 30cm, and a second size combination with a width of 30cm and a height of 47cm. The aspect ratios of the first size combination and the second size combination are 0.68 and 0.64, respectively. When the aspect ratio of the detection frame is 0.65, the width and height corresponding to the second size combination can be used as the second candidate size.

[0267] Step S1124: Determine the first predicted size based on the first candidate size and the second candidate size.

[0268] In some implementations, the first predicted size can be selected from the first candidate size and the second candidate size; or the first predicted size can be calculated based on the first candidate size and the second candidate size.

[0269] In this embodiment, a first candidate size of the first target is determined based on the extrinsic parameters of the image acquisition device and the coordinates of the detection box of the first target in the first image frame; the aspect ratio of the detection box of the first target in the second image frame is determined; then, a second candidate size corresponding to the aspect ratio of the detection box is determined from a preset set of target sizes; and a first predicted size is determined based on the first candidate size and the second candidate size. In this way, the rationality and accuracy of the first predicted size of the first target can be improved by using multiple candidate sizes.

[0270] Understandably, when the first target includes a first cone and the second target includes a second cone, a first candidate size for the first cone is determined based on the extrinsic parameters of the image acquisition device and the coordinates of the detection box of the first cone in the first image frame; the aspect ratio of the detection box of the first cone in the second image frame is determined; then, a second candidate size corresponding to the aspect ratio of the detection box is determined from a preset set of cone sizes; and a first predicted size is determined based on the first and second candidate sizes. In this way, the rationality and accuracy of the first predicted size of the first cone can be improved by using multiple candidate sizes.

[0271] In some embodiments, step S1124 may include one of steps S11241 to S11242:

[0272] Step S11241: If the difference between the first candidate size and the second candidate size is greater than the size difference threshold, the second candidate size is determined as the first predicted size.

[0273] Here, a set of size difference thresholds can be preset. If the difference between the first candidate size and the second candidate size is greater than the size difference threshold, the first candidate size is determined to be unreliable, and the second candidate size in the preset cone size set is used as the standard to improve the accuracy of the first predicted size.

[0274] Step S11242: If the difference between the first candidate size and the second candidate size is not greater than the size difference threshold, the first candidate size is determined as the first predicted size.

[0275] Here, if the difference between the first candidate size and the second candidate size is not greater than the size difference threshold, the first candidate size is determined to be reliable, and the obtained first candidate size is used as the first predicted size to improve the adaptability of the distance calculation based on the first predicted size.

[0276] For example, the aspect ratio of the detection box of the first cone in the second image frame can be used as a reference. img_wh Match the preset set of cone sizes with Ratio img_wh The corresponding actual width and / or height of the cone.

[0277] For example, Ratio img_wh The width in the corresponding second candidate size can be represented as width. world_pre Height can be represented as height world_pre Calculate the width of the first candidate size. ex width in the second candidate size world_pre The absolute value of the difference between them (width) dif , in width dif If the difference is greater than the threshold δ, use width. world_pre As the first predicted size; in width dif If the difference is not greater than the threshold δ, use width. ex As the first predicted size.

[0278] In this embodiment, a first predicted size is determined from the first and second candidate sizes based on a size difference threshold and the magnitude of the difference between the first and second candidate sizes. Thus, if the difference between the first and second candidate sizes is greater than the size difference threshold, the first candidate size is considered unreliable, and the second candidate size from a more reliable set of preset cone sizes is determined as the first predicted size. If the difference is not greater than the size difference threshold, the first candidate size is considered relatively reliable, and the first candidate size determined based on the first image frame, which is more suitable for the scene, is determined as the first predicted size, thereby improving the accuracy and reasonableness of the first predicted size.

[0279] In some embodiments, the number of first targets is multiple, and before step S12 above, the ranging method may further include the following step S16:

[0280] Step S16: Based on the distribution information of each target detected in the second image frame, determine the layout type of the target in the second image frame.

[0281] Here, the layout type of the target can include, but is not limited to, discrete layout, dense row layout, etc.

[0282] In some implementations, distribution information can be determined based on the coordinates of each target detected in the second image frame in the image coordinate system.

[0283] For example, we can first determine whether the total number of targets in the second image frame is greater than N. If the total number of targets is greater than N, we can iterate through all targets in the second image frame and store the coordinates of the midpoint of the bottom edge of the detection box of each target in the image coordinate system. We can then perform bubble sort on the stored target coordinates based on the vertical coordinate values. Based on the sorted target queue, we can sequentially obtain the coordinates P of the midpoint of the bottom edge of the detection box of two adjacent targets. img_ctr_a and P img_ctr_b Calculate the Euclidean distance D between two points. a_b ; Obtain the width information W of the two adjacent targets respectively. img_a and W img_b ;

[0284] The distance D between the two adjacent targets mentioned above a_b The sum of width and width W img_a +W img_b The ratio D a_b / (W img_a +W img_b If the ratio is less than the ratio threshold γ, the two targets are determined to be adjacent and meet the scene characteristics of densely arranged targets; if the ratio is not less than the ratio threshold, the two targets are determined to be in a discrete arrangement scene.

[0285] In some implementations, in the case of a discrete layout scenario, the first target can be determined from all targets based on the range change of each target in each target image frame and the vehicle's movement distance.

[0286] For example, in a scenario where the layout type is discrete arrangement, the center point value, convergence state position, and output VCS coordinates of each target in the vehicle coordinate system can be obtained. The convergence state position can correspond to the ranging confidence state position in the ranging method described above. If the target's VCS coordinates are calculated for the first time, the VCS coordinates are kept as the output coordinates. If the target's VCS coordinates are not calculated for the first time, the output coordinates are determined based on the target's convergence state position.

[0287] If the convergence state bit of the target indicates that the ranging state of the target has converged, the coordinates of the target in the second image frame can be directly output based on the difference between the lateral and longitudinal distances of the vehicle's movement and the VCS coordinates output in the previous frame of the target. At the same time, the convergence state bit is maintained to indicate that the ranging state of the target has converged and that the ranging state of the target is reliable. If the convergence state bit of the target indicates that the ranging state of the target has not converged, it is determined whether the ranging distance between the target and the vehicle in each target image frame meets the target confidence condition. If the target confidence condition is met, the convergence state bit of the target is set to converged. If the target confidence condition is not met, the convergence state bit of the target is set to non-converged, and the ranging distance of the target is calculated based on other targets that meet the target confidence condition.

[0288] In some implementations, a straight line can be constructed based on the position of the center point of the bottom of the cone that satisfies the dense arrangement condition; if the slope of the straight line is less than the slope threshold, the layout type of the cone corresponding to the straight line is determined to be a dense row layout.

[0289] For example, by constructing a fitted line for the traffic cones using the least squares method, if the absolute value of the slope of the fitted line is less than tan10°, the layout type of the traffic cones is determined to be a dense, adjacent layout; if the absolute value of the slope of the fitted line is not less than tan10°, it is determined that there is a possibility of using traffic cones for road closure. In this way, by combining the location of the traffic cones, the accuracy of determining the scene where the traffic cones are located and the accuracy of determining the layout type of the traffic cones can be improved.

[0290] Step S12 above may include the following step S123:

[0291] Step S123: When the layout type is dense arrangement, for each first target, determine the ranging confidence state of the first target based on the first ranging distance and the second ranging distance corresponding to the first target.

[0292] In some implementations, when the layout type is dense arrangement, multiple candidate targets can be determined from the second image frame; among each candidate target, if the difference between the ranging change of each target image frame corresponding to the target and the vehicle movement distance is less than the target difference threshold, the candidate target is determined as the first target; then for each first target, the ranging confidence state of the first target is determined based on the first ranging distance and the second ranging distance corresponding to the first target.

[0293] In this embodiment, based on the distribution information of each target detected in the second image frame, the layout type of the targets in the second image frame is determined; and when the layout type is densely arranged, for each first target, the ranging confidence state of the first target is determined based on the first ranging distance and the second ranging distance corresponding to the first target. In this way, in the scenario of densely arranged targets, the ranging confidence state of multiple first targets can be analyzed, so that a more accurate second predicted size can be determined based on the first predicted size of multiple first targets with reliable ranging, thereby improving the accuracy of ranging the second target.

[0294] In some embodiments, the difference between the ranging change corresponding to the target image frame and the vehicle movement distance corresponding to the target image frame is less than a target difference threshold; the target image frame includes a second image frame or a historical image frame acquired before the second image frame, the ranging change corresponding to the target image frame is determined based on the difference between the first historical ranging distance of the first target in the target image frame and the second historical ranging distance of the first target in the previous frame of the target image frame, and the vehicle movement distance corresponding to the target image frame is the movement distance of the vehicle within the acquisition interval between the target image frame and the previous frame of the target image frame.

[0295] For example, when the layout type is dense arrangement, all targets can be traversed, and the range change D corresponding to each target in each target image frame can be calculated. del And the vehicle movement distance corresponding to each target in each target image frame. cur ; in target D del With move cur If the absolute value of the difference between the two is less than the difference threshold, the target is determined as the first target.

[0296] For example, targets whose difference between the distance change corresponding to each target image frame and the vehicle's movement distance is less than a target difference threshold can be marked as verifiable targets, and their numbers and existing information can be recorded. Based on the coordinates of the verifiable targets in the vehicle-based coordinate system, three targets that are less than 80 meters away from the vehicle and are close to the vehicle (e.g., including the closest) can be selected as the first targets. In this way, since the targets are close to the vehicle, distance measurement based on the first targets at this location is more accurate.

[0297] Understandably, the above ranging method can determine the ranging distance of a target near the vehicle based on the size of the target relative to the vehicle. If the ranging distance of the target near the vehicle is reliable, the ranging distance between the target near the vehicle and the vehicle can be determined based on the mapping relationship between the size of the target near the vehicle and the size of the target far from the vehicle or the target near the vehicle. Based on the target information of the target near the vehicle, the accuracy of ranging between the target far from the vehicle or the target near the vehicle and the vehicle can be improved.

[0298] In this embodiment of the disclosure, the difference between the ranging change corresponding to the target image frame and the vehicle movement distance corresponding to the target image frame is less than the target difference threshold, indicating that the ranging of each first target is more reliable. Thus, a more accurate second prediction size can be determined based on the first prediction size of multiple first targets with more reliable ranging, thereby improving the accuracy of ranging of the second target.

[0299] In some embodiments, the above ranging method may further include the following steps S124 to S126:

[0300] Step S124: If the ranging confidence state of the first target indicates that the second ranging distance is not reliable, acquire the third image frame.

[0301] In some implementations, if the ranging confidence state of the first target indicates that the second ranging distance is unreliable, a third image frame can be acquired after acquiring the second image frame. The third image frame may or may not include the first target.

[0302] Step S125: If the third ranging distance between multiple third targets and the vehicle in the third image frame is less than the target distance threshold, determine the third predicted size of the multiple third targets.

[0303] In some implementations, when the third ranging distance between multiple third targets and the vehicle in the third image frame is less than the target distance threshold, the ranging confidence state of each third target can be determined.

[0304] Given that the ranging confidence state of each third target indicates that the third ranging distance is reliable, the third predicted size of multiple third targets is determined respectively.

[0305] When the ranging confidence state of each third target indicates that the third ranging distance is unreliable, the third candidate size of multiple third targets is determined respectively; the average of the third candidate size of each third target is calculated to obtain a set of third predicted sizes.

[0306] Step S126: Based on the third predicted size and the detection box information of the fourth target in the third image frame, determine the second target ranging distance between the fourth target and the vehicle in the third image frame.

[0307] Here, the second target ranging distance can include the lateral distance and longitudinal distance between the fourth target and the vehicle in the third image frame.

[0308] In some implementations, the second target ranging distance between the fourth target and the vehicle in the third image frame can be determined by utilizing the pinhole imaging principle, based on the detection box of the fourth target imaged in the third image frame and the third prediction size.

[0309] In this embodiment, when the ranging confidence state of the first target indicates that the second ranging distance is unreliable, a third image frame is acquired; when the third ranging distance between multiple third targets and the vehicle in the third image frame is less than a target distance threshold, the third predicted size of the multiple third targets is determined; and based on the third predicted size and the detection box information of the fourth target in the third image frame, the second target ranging distance between the fourth target and the vehicle in the third image frame is determined. Thus, predicting the fourth target based on the third targets that are closer to the vehicle can improve the accuracy of ranging the fourth target even when the first target is unreliable.

[0310] In some embodiments, the above ranging method may further include the following steps S17 to S18:

[0311] Step S17: Determine the relative positional relationship between the lane line points in the second image frame and the second target, as well as the lane distance between the lane line points and the vehicle in the second image frame.

[0312] In some implementations, at least one lane line point in the second image frame that is close to (e.g., the closest) to the second target can be identified, and the lateral relative position between the second target and the lane line point can be determined based on the position of the lane line point in the second image frame. For example, the second target may be located to the left or right of the lane line point, or on the lane line corresponding to the lane line point.

[0313] Step S18: Update the distance to the first target based on the relative position relationship and the lane distance.

[0314] In some implementations, the position of the second target relative to the lane line point can be adjusted based on the relative positional relationship between the lane line point and the second target in the second image frame; the coordinates of the lane line point in the vehicle-based coordinate system can be determined based on the lane distance between the lane line point and the vehicle in the second image frame, and the coordinates of the second target in the vehicle-based coordinate system can be adjusted; the distance of the first target can be updated based on the coordinates of the second target in the vehicle-based coordinate system.

[0315] For example, by combining Figure 1G and Figure 1H, the relative position of the point of lane line 31 in the second image frame 30 corresponding to Figure 1G and the second target 32 ​​can be obtained first, that is, the second target 32 ​​is located to the right of the point of lane line 31. Then, according to the point of lane line 31 in the second image frame 30 corresponding to Figure 1H and the VCS coordinate of the second target 32, the VCS coordinate of the second target 32 ​​is adjusted to obtain the distance of the first target.

[0316] In this embodiment, the relative positional relationship between the lane line points and the second target in the second image frame, as well as the lane distance between the lane line points and the vehicle in the second image frame, are determined. Then, based on the relative positional relationship and the lane distance, the distance to the first target is updated. In this way, by using the position of the lane line points in the second image frame, the distance measurement error of the second target can be reduced, and the realism and accuracy of the distance to the first target can be improved.

[0317] Understandably, updating the distance measurement of the first target based on the relative positional relationship between the lane line point and the second target can improve the authenticity and accuracy of the distance measurement of the first target, improve the situation where inaccurate distance measurement results in special curve scenarios lead to false braking caused by intrusion into the main lane, and improve the situation where high-precision maps cannot reflect the missed braking caused by pulling targets due to real road conditions such as construction or detours.

[0318] This disclosure also proposes a cone ranging method, which can be executed by an electronic device. Taking cone ranging in autonomous driving as an example, as shown in Figure 2, the method includes the following steps S201 to S207:

[0319] Step S201: Obtain external parameters and prior parameters.

[0320] Here, the external parameters can correspond to the external parameters of the vision system in the above ranging method, and the prior parameters can correspond to the common dimensions of the cone in the above ranging method.

[0321] In implementation, a vehicle with autonomous driving capabilities can be deployed, and this vehicle is equipped with a vision system with a monocular CCD.

[0322] Step S202: Model reasoning to obtain information such as cones.

[0323] During implementation, a U-NET neural network model can be constructed to perform batch data annotation and training on data such as cones and lane lines, and obtain the position information of cone and lane line points in the image frame.

[0324] Step S203: External parameter ranging and internal parameter correction.

[0325] Here, extrinsic distance measurement can be the process of converting the coordinates of points in the image into VCS coordinates through the extrinsic transformation matrix in the above distance measurement method, as shown in formula (2); intrinsic parameter correction can be the process of obtaining the coordinate position of the CCD center of the image acquisition device in the image based on the calibrated internal parameters of the image acquisition device in the above distance measurement method, and determining the position of the cone relative to the CCD center, as shown in formula (5), so as to reduce the deviation of the cone coordinates relative to the CCD center image coordinates.

[0326] Step S204, EKF optimization.

[0327] This can be compared to the process in the above ranging method of using a filter to determine the coordinates of the center point of the bottom of the second cone based on the coordinates of multiple edge points. Here, an EKF filter is used to optimize the coordinates of multiple edge points at the bottom of the cone to obtain the coordinates of the center point of the bottom of the cone.

[0328] During implementation, the EKF filter is initialized for the first measurement; for subsequent measurements, EKF optimization is performed normally.

[0329] Step S205: Vehicle speed and inter-frame time verification.

[0330] Here, vehicle speed and inter-frame time verification can be equated to the process in the aforementioned ranging method of determining whether the change in distance measured by the target in each target image frame and the distance the vehicle moves meet the target confidence condition. The inter-frame time can be defined as the acquisition interval between two image frames. Based on the vehicle speed and the inter-frame time between the two image frames, the distance the vehicle moves between those two image frames can be determined.

[0331] Step S206: Genetic optimization to calculate the positions of other cones.

[0332] Here, genetic optimization can be compared to the process in the above ranging method of determining the distance between the second target and the vehicle in the second image frame based on the second predicted size of the first target. That is, using the size gene of the reliable cone, gene expression is performed on other cones to determine the distance between other cones and the vehicle.

[0333] Step S207: Correct the lateral position of the cones based on the lane line information.

[0334] Here, lane line information may include the position information of points on the lane line, the relative position of points on the lane line to cones in the image frame, and the coordinate information of points on the lane line in the VCS coordinate system.

[0335] The lateral position can be represented by the coordinate information of the cone in the y-axis direction in the VCS coordinate system in the above distance measurement method.

[0336] In this embodiment, cone information is obtained through model inference; extrinsic distance measurement is performed on the cones, and the distance measurement results are corrected based on intrinsic parameters; the coordinates are optimized using an EKF filter; after verifying the cone distance measurement results based on vehicle speed and inter-frame time, the positions of other cones are calculated based on the size of a reliable cone; then, the lateral position of the cones is corrected based on lane line information. Thus, on the one hand, processing cone information through a model improves overall processing efficiency; on the other hand, extrinsic distance measurement of the cones, correction of the distance measurement results based on intrinsic parameters, the use of an EKF filter, and correction of cone positions based on lane line information all improve the accuracy of the distance measurement values; furthermore, calculating the position of other cones based on the size of a reliable cone can mitigate the jitter caused by systematic errors in the distance measurement process, thereby improving the accuracy of cone distance measurement.

[0337] This disclosure also proposes a ranging device, as shown in FIG3, the device 300 comprising:

[0338] The first acquisition station 301 is configured to acquire a first ranging distance between a first target and a vehicle in a first image frame, and a second ranging distance between the first target and the vehicle in a second image frame; the first image frame and the second image frame are acquired sequentially by an image acquisition device mounted on the vehicle during the vehicle's movement;

[0339] The first determining station 302 is configured to determine the ranging confidence state of the first target based on the first ranging distance and the second ranging distance;

[0340] The second determining station 303 is configured to, when the ranging confidence state of the first target indicates that the second ranging distance is reliable, determine the second predicted size of the second target based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target; the second ranging distance is determined based on the first predicted size; and

[0341] The third determining station 304 is configured to determine the first target ranging distance between the second target and the vehicle in the second image frame based on the second predicted size and the detection box information of the second target in the second image frame.

[0342] In some embodiments, the first target includes a first cone, the second target includes a second cone, and the second determining station may be configured to: determine the first predicted size as the second predicted size when the size relationship indicates that the first cone and the second cone have the same size specifications; wherein the first predicted size and the second predicted size correspond to the same size parameter, and the size parameter may include a height parameter and / or a width parameter.

[0343] In some embodiments, the detection box information includes the detection box size and the detection box position; the third determining station may also be configured to: determine the longitudinal distance between the second target and the vehicle in the second image frame based on the intrinsic parameters of the image acquisition device, the second predicted size, and the detection box size of the second target in the second image frame; determine the lateral distance between the second target and the vehicle in the second image frame based on the intrinsic parameters, the detection box position of the second target in the second image frame, and the longitudinal distance; and determine the distance between the first target based on the longitudinal distance and the lateral distance.

[0344] In some embodiments, the first target includes a first cone, the second target includes a second cone, and the third determining station may be configured to: determine the coordinates of multiple edge points at the bottom of the second cone based on the longitudinal distance and the lateral distance; determine the coordinates of the center point at the bottom of the second cone based on the multiple edge point coordinates; and determine the distance to the first target based on the center point coordinates.

[0345] In some embodiments, the first determining station may further be configured to: determine the ranging change corresponding to the second image frame based on the difference between the first ranging distance and the second ranging distance; determine the ranging confidence state of the first target based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame; the vehicle movement distance corresponding to the second image frame is the movement distance of the vehicle within the acquisition interval between the first image frame and the second image frame.

[0346] In some embodiments, the first image frame is the previous frame of the second image frame; the first determining station may further be configured to: determine the ranging change and the corresponding vehicle movement distance corresponding to at least one historical image frame; the ranging change corresponding to the historical image frame is determined based on the difference between the first historical ranging distance of the first target in the historical image frame and the second historical ranging distance of the first target in the previous frame of the historical image frame, and the vehicle movement distance corresponding to the historical image frame is the movement distance of the vehicle within the acquisition interval between the historical image frame and the previous frame of the historical image frame; determine the ranging confidence state of the first target based on the ranging change and the corresponding vehicle movement distance corresponding to multiple target image frames; the multiple target image frames include the at least one historical image frame and the second image frame.

[0347] In some embodiments, the first determining station may further be configured to: determine the ranging confidence state of the first target as a first state characterizing the credibility of the second ranging distance when the ranging change and the corresponding vehicle movement distance corresponding to the plurality of target image frames respectively meet the target confidence condition;

[0348] The target confidence condition includes at least one of the following: the sum of the distance changes corresponding to each of the target image frames is less than a first target distance threshold; the first target distance threshold is determined based on the sum of the vehicle movement distances corresponding to each of the target image frames; the difference between the distance change and the corresponding vehicle movement distance for each target image frame is less than a distance difference threshold; for each target image frame, the distance change and the corresponding vehicle movement distance are both positive or both negative.

[0349] In some embodiments, the ranging device may further include a fourth determining station, configured to: determine a third ranging distance between the first target and the vehicle in the third image frame based on the vehicle's movement distance between the acquisition interval of the third image frame and the second image frame, and the second ranging distance, when the ranging confidence state of the first target indicates that the second ranging distance is reliable; the second image frame and the third image frame are acquired sequentially by the image acquisition device during the vehicle's movement.

[0350] In some embodiments, the first acquisition station may further be configured to: determine the first predicted size based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the second image frame; and determine the second ranging distance based on the first predicted size, the detection box size of the first target in the second image frame, and the detection box coordinates of the first target in the second image frame.

[0351] In some embodiments, the first target includes a first cone, the second target includes a second cone, and the first acquisition station may be configured to: determine a first candidate size of the first cone based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first cone in the first image frame; determine the aspect ratio of the detection box of the first cone in the second image frame; determine a second candidate size corresponding to the aspect ratio of the detection box from a preset cone size set; and determine the first predicted size based on the first candidate size and the second candidate size.

[0352] In some embodiments, the first acquisition station may also be configured to: determine the second candidate size as the first predicted size when the difference between the first candidate size and the second candidate size is greater than a size difference threshold; or determine the first candidate size as the first predicted size when the difference between the first candidate size and the second candidate size is not greater than a size difference threshold.

[0353] In some embodiments, the number of the first targets is multiple, and the ranging device may further include a fifth determining station, configured to: before determining the ranging confidence state of the first target based on the first ranging distance and the second ranging distance, determine the layout type of the targets in the second image frame based on the distribution information of each target detected in the second image frame;

[0354] The first determining station can also be configured to: when the layout type is densely arranged, for each first target, determine the ranging confidence state of the first target based on the first ranging distance and the second ranging distance corresponding to the first target.

[0355] In some embodiments, the difference between the ranging change corresponding to the target image frame and the vehicle movement distance corresponding to the target image frame is less than a target difference threshold; the target image frame includes the second image frame or a historical image frame acquired before the second image frame, the ranging change corresponding to the target image frame is determined based on the difference between the first historical ranging distance of the first target in the target image frame and the second historical ranging distance of the first target in the previous frame of the target image frame, and the vehicle movement distance corresponding to the target image frame is the movement distance of the vehicle within the acquisition interval between the target image frame and the previous frame of the target image frame.

[0356] In some embodiments, the ranging device may further include a sixth determining station, configured to: acquire a third image frame when the ranging confidence state of the first target indicates that the second ranging distance is unreliable; determine a third predicted size of the plurality of third targets when the third ranging distance between the vehicle and the plurality of third targets in the third image frame is less than a target distance threshold; and determine a second target ranging distance between the vehicle and the fourth target in the third image frame based on the third predicted size and the detection box information of the fourth target in the third image frame.

[0357] In some embodiments, the third determining station may further be configured to: determine the relative positional relationship between the lane line point in the second image frame and the second target, and the lane distance between the lane line point and the vehicle in the second image frame; and update the first target distance based on the relative positional relationship and the lane distance.

[0358] This disclosure also provides an electronic device, including a memory and a processor. As shown in FIG4, the electronic device 400 includes:

[0359] Memory 401 is configured to store computer instructions that can run on processor 402; and

[0360] Processor 402 is configured to execute instructions stored in memory 401 to implement the above-described ranging method.

[0361] This disclosure also proposes a vehicle including the aforementioned electronic equipment.

[0362] This disclosure also proposes a computer program including computer-readable code, which, when run in a computer device, enables a processor in the computer device to perform some or all of the steps in the above-described ranging method.

[0363] This disclosure also proposes a computer program product, including computer instructions, which, when executed by a processor, implement some or all of the steps in the above-described ranging method.

[0364] This disclosure also proposes a computer-readable storage medium storing computer instructions that can be executed by a processor to implement the above-described ranging method.

[0365] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and devices according to embodiments of this disclosure. Each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer instructions. These computer instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0366] These computer instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0367] These computer instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0368] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0369] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0370] Throughout this specification, the phrase "an embodiment" or "one embodiment" means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above-described embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0371] In this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0372] The devices and methods disclosed in the several embodiments provided in this application can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of stations is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple stations or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or stations can be electrical, mechanical, or other forms.

[0373] The units described above as separate components may or may not be physically separate; the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected to implement the embodiments of this disclosure according to actual needs.

[0374] In addition, each functional unit in the embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0375] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by instruction-related hardware. The aforementioned instructions can be stored in a computer-readable storage medium. When the instructions are executed, they perform the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code / instructions, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0376] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code / instructions, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0377] The above embodiments are merely illustrative of this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.

Claims

1. A distance measurement method, comprising: The first distance between the first target and the vehicle in the first image frame and the second distance between the first target and the vehicle in the second image frame are obtained; wherein the first image frame and the second image frame are acquired sequentially by the image acquisition device mounted on the vehicle during the vehicle's movement. Based on the first ranging distance and the second ranging distance, the ranging confidence state of the first target is determined; When the ranging confidence state of the first target indicates that the second ranging distance is reliable, the second predicted size of the second target is determined based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target; wherein the second ranging distance is determined based on the first predicted size; and Based on the second predicted size and the detection box information of the second target in the second image frame, a first target ranging distance between the second target and the vehicle in the second image frame is determined.

2. The method of claim 1, wherein, The first target includes a first cone, and the second target includes a second cone; determining the second predicted size of the second target based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target, includes: When the size relationship indicates that the first cone and the second cone have the same size specifications, the first predicted size is determined as the second predicted size; wherein the first predicted size and the second predicted size correspond to the same size parameter, the size parameter including the height parameter and / or the width parameter.

3. The method of claim 1, wherein, The detection frame information includes the detection frame size and the detection frame position; The step of determining the first target ranging distance between the second target and the vehicle in the second image frame based on the second predicted size and the detection box information of the second target in the second image frame includes: Based on the intrinsic parameters of the image acquisition device, the second predicted size, and the detection box size of the second target in the second image frame, the longitudinal distance between the second target and the vehicle in the second image frame is determined. Based on the intrinsic parameters, the detection box position of the second target in the second image frame, and the longitudinal ranging distance, the lateral ranging distance between the second target and the vehicle in the second image frame is determined; and The first target distance is determined based on the longitudinal distance and the lateral distance.

4. The method according to claim 3, wherein, The first target includes a first cone, and the second target includes a second cone; determining the distance to the first target based on the longitudinal distance and the lateral distance includes: Based on the longitudinal and lateral distances, the coordinates of multiple edge points at the bottom of the second cone are determined. Based on the coordinates of multiple edge points, determine the coordinates of the center point of the bottom of the second cone; and Based on the coordinates of the center point, the distance to the first target is determined.

5. The method according to claim 1, wherein, Determining the ranging confidence state of the first target based on the first ranging distance and the second ranging distance includes: Based on the difference between the first ranging distance and the second ranging distance, the ranging change corresponding to the second image frame is determined; and Based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame, the ranging confidence state of the first target is determined; wherein, the vehicle movement distance corresponding to the second image frame is the movement distance of the vehicle within the acquisition interval between the first image frame and the second image frame.

6. The method according to claim 5, wherein, The first image frame is the frame preceding the second image frame; determining the ranging confidence state of the first target based on the ranging change corresponding to the second image frame and the vehicle movement distance corresponding to the second image frame includes: Determine the ranging change and the corresponding vehicle movement distance for at least one historical image frame; wherein the ranging change for each historical image frame is determined based on the difference between the first historical ranging distance of the first target in the historical image frame and the second historical ranging distance of the first target in the previous frame of the historical image frame; and the vehicle movement distance for each historical image frame is the movement distance of the vehicle within the acquisition interval between the historical image frame and the previous frame of the historical image frame; and The ranging confidence state of the first target is determined based on the ranging change and the corresponding vehicle movement distance corresponding to multiple target image frames; wherein, the multiple target image frames include the at least one historical image frame and the second image frame.

7. The method of claim 6, wherein, The step of determining the ranging confidence state of the first target based on the ranging changes and corresponding vehicle movement distances corresponding to multiple target image frames includes: If the ranging change and the corresponding vehicle movement distance corresponding to the plurality of target image frames satisfy the target confidence condition, the ranging confidence state of the first target is determined as a first state characterizing the reliability of the second ranging distance; wherein, the target confidence condition includes at least one of the following: The sum of the ranging changes corresponding to each of the target image frames is less than a first target distance threshold; wherein, the first target distance threshold is determined based on the sum of the vehicle movement distances corresponding to each of the target image frames; The difference between the distance change and the corresponding vehicle movement distance for each target image frame is less than the distance difference threshold. For each target image frame, the distance change and the vehicle movement distance corresponding to the target image frame are both positive or both negative.

8. The method according to any one of claims 1 to 7, further comprising: When the ranging confidence state of the first target indicates that the second ranging distance is reliable, the third ranging distance between the first target and the vehicle in the third image frame is determined based on the vehicle's movement distance between the acquisition interval of the third image frame and the second image frame, and the second ranging distance; wherein the second image frame and the third image frame are acquired sequentially by the image acquisition device during the vehicle's movement.

9. The method of any one of claims 1 to 7, wherein, The step of obtaining the second ranging distance between the first target and the vehicle in the second image frame includes: Based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the second image frame, the first predicted size is determined; and The second ranging distance is determined based on the first predicted size, the detection box size of the first target in the second image frame, and the detection box coordinates of the first target in the second image frame.

10. The method of claim 9, wherein, The first target includes a first cone, and the second target includes a second cone; determining the first predicted size based on the extrinsic parameters of the image acquisition device and the detection box coordinates of the first target in the first image frame includes: Based on the external parameters of the image acquisition device and the detection box coordinates of the first cone in the first image frame, the first candidate size of the first cone is determined. Determine the aspect ratio of the detection box of the first cone in the second image frame; From a set of preset cone sizes, determine a second candidate size corresponding to the aspect ratio of the detection frame; and The first predicted size is determined based on the first candidate size and the second candidate size.

11. The method according to claim 10, wherein, Determining the first predicted size based on the first candidate size and the second candidate size includes one of the following: If the difference between the first candidate size and the second candidate size is greater than the size difference threshold, the second candidate size is determined as the first predicted size; If the difference between the first candidate size and the second candidate size is not greater than the size difference threshold, the first candidate size is determined as the first predicted size.

12. The method of any one of claims 1 to 7, wherein, The number of the first targets is multiple; before determining the ranging confidence state of the first targets based on the first ranging distance and the second ranging distance, the method further includes: Based on the distribution information of each target detected in the second image frame, the layout type of the targets in the second image frame is determined; Determining the ranging confidence state of the first target based on the first ranging distance and the second ranging distance includes: When the layout type is dense arrangement, for each first target, the ranging confidence state of the first target is determined based on the first ranging distance and the second ranging distance corresponding to the first target.

13. The method according to claim 12, wherein, The difference between the ranging change corresponding to the target image frame and the vehicle movement distance corresponding to the target image frame is less than the target difference threshold; the target image frame includes the second image frame or a historical image frame acquired before the second image frame, the ranging change corresponding to the target image frame is determined based on the difference between the first historical ranging distance of the first target in the target image frame and the second historical ranging distance of the first target in the previous frame of the target image frame, and the vehicle movement distance corresponding to the target image frame is the movement distance of the vehicle within the acquisition interval between the target image frame and the previous frame of the target image frame.

14. The method of claim 12, further comprising: If the ranging confidence state of the first target indicates that the second ranging distance is unreliable, a third image frame is acquired; If, in the third image frame, the third ranging distance between multiple third targets and the vehicle is less than a target distance threshold, the third predicted size of the multiple third targets is determined; and Based on the third predicted size and the detection box information of the fourth target in the third image frame, the second target ranging distance between the fourth target and the vehicle in the third image frame is determined.

15. The method according to any one of claims 1 to 7, further comprising: Determine the relative positional relationship between the lane line points in the second image frame and the second target, as well as the lane distance between the lane line points and the vehicle in the second image frame; and The first target distance is updated based on the relative position relationship and the lane distance.

16. A ranging device, comprising: The first acquisition station is configured to acquire a first ranging distance between a first target and a vehicle in a first image frame, and a second ranging distance between the first target and the vehicle in a second image frame; wherein the first image frame and the second image frame are acquired sequentially by an image acquisition device mounted on the vehicle during the vehicle's movement; The first determining station is configured to determine the ranging confidence state of the first target based on the first ranging distance and the second ranging distance; The second determining station is configured to, when the ranging confidence state of the first target indicates that the second ranging distance is reliable, determine the second predicted size of the second target based on the size relationship between the first target and the second target detected in the second image frame, and the first predicted size of the first target; wherein the second ranging distance is determined based on the first predicted size; and The third determining station is configured to determine the first target ranging distance between the second target and the vehicle in the second image frame based on the second predicted size and the detection box information of the second target in the second image frame.

17. An electronic device comprising: A memory and a processor, the memory storing computer instructions executable on the processor, the processor executing the instructions to implement the steps of the method according to any one of claims 1 to 15.

18. A vehicle comprising the electronic equipment as claimed in claim 17.

19. A computer readable storage medium storing computer instructions, which can be executed by a processor to implement the method of any one of claims 1 to 15.

20. A computer program product comprising computer instructions, which when executed by a processor implement the method of any one of claims 1 to 15.

Images