A target ranging method, device, equipment and medium based on a cylindrical graph detection frame

Abstract

The present application discloses a target ranging method, device, equipment and medium based on a cylindrical projection detection frame, which relates to the field of computer technology and includes: performing cylindrical projection distortion removal processing on the original image collected by a fish-eye camera to obtain a cylindrical projection image, and obtaining two-dimensional detection frames of each target in the current frame according to the cylindrical projection image; determining two-dimensional prediction frames of each target in the current frame through two-dimensional tracking frames of each target in the previous frame and radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters; determining the two-dimensional prediction frame matched with the two-dimensional detection frame, updating the two-dimensional prediction frame matched with the two-dimensional detection frame to obtain the two-dimensional tracking frame of the target in the current frame, and determining the three-dimensional position information of the target according to the two-dimensional tracking frame of the target, so as to achieve target ranging. The present application improves the ranging accuracy and ranging stability of the target.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a target ranging method, device, equipment and medium based on a cylindrical graph detection frame. Background Art

[0002] The rapid development of computer vision has brought technological innovation to the environmental perception of intelligent vehicles, and its applications in the fields of driving assistance systems and intelligent driving are also becoming increasingly widespread. Due to its characteristics such as low cost, low energy consumption, and low computing power, the pure vision-based surround fisheye system has gradually become the mainstream perception solution for major vehicle manufacturers.

[0003] Although the surround fisheye system has an extremely large viewing angle range, the fisheye camera images have strong non-linear distortion, and the distortion of pixels closer to the edge of the image is greater, which is not conducive to the development of target detection algorithms. A common solution is to convert the fisheye image into other forms of images through calibration, such as converting it into a cylindrical image through a cylindrical graph detection framework. However, the current cylindrical graph detection framework has the following problems when dealing with small targets: 1. Due to the small size of small targets, their representation on the cylindrical graph may not be clear enough or difficult to accurately detect, especially for small targets at a long distance. The number of pixel elements they occupy is small, and a pixel in a long-distance scene can represent a distance of dozens of centimeters, so there must be a ranging accuracy problem when ranging. 2. Environmental robustness problem: Factors such as different lighting conditions and the contrast difference between the target and the background may further affect the ranging effect of small targets, resulting in unstable ranging, and even missed detection and false detection. Therefore, the above problems urgently need to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a target ranging method, device, equipment and medium based on a cylindrical graph detection frame, which improves the ranging accuracy and ranging stability. The specific solutions are as follows:

[0005] In a first aspect, the present application discloses a target ranging method based on a cylindrical graph detection frame, including:

[0006] Performing cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtaining two-dimensional detection frames of each target in the current frame according to the cylindrical projection image;

[0007] Determining two-dimensional prediction frames of each target in the current frame through two-dimensional tracking frames of each target in the previous frame and radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters;

[0008] Determine the two-dimensional prediction box that matches the two-dimensional detection box, update the two-dimensional prediction box that matches the two-dimensional detection box to obtain the two-dimensional tracking box of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking box of the target, so as to realize the ranging of the target.

[0009] Optionally, the determining the two-dimensional prediction boxes of the targets in the current frame by using the two-dimensional tracking boxes of the targets in the previous frame and the radial compensation results of the targets includes:

[0010] Obtain respective shrinkage factors based on the feature information of the two-dimensional detection boxes of the targets and the vehicle body perspective control parameters, and determine the radial compensation results of the targets according to the respective shrinkage factors;

[0011] Determine the target ranging points of the two-dimensional tracking boxes of the targets according to the radial compensation results of the targets, and determine the two-dimensional prediction boxes of the targets in the current frame according to the respective target ranging points and the two-dimensional tracking boxes of the targets in the previous frame.

[0012] Optionally, the determining the radial compensation results of the targets according to the respective shrinkage factors includes:

[0013] Determine the height information of the center points of the detection boxes of the two-dimensional detection boxes of the targets;

[0014] Perform multiplication operations on the height information of the center points of the detection boxes of the two-dimensional detection boxes of the targets and the respective shrinkage factors to obtain the radial compensation results of the targets.

[0015] Optionally, the feature information of the two-dimensional detection box of the target includes the width and height information of the two-dimensional detection box of the target, and the vehicle body perspective control parameter includes the height information of the fisheye camera;

[0016] Correspondingly, the obtaining the respective shrinkage factors based on the feature information of the two-dimensional detection boxes of the targets and the vehicle body perspective control parameters includes:

[0017] Obtain respective shrinkage factors according to the target model and by using the width and height information of the two-dimensional detection boxes of the targets and the height information of the fisheye camera;

[0018] Wherein, in the target model, the shrinkage factor is in an inverse relationship with the height information of the fisheye camera, and the shrinkage factor is in a direct relationship with the target ratio, and the target ratio is the ratio of the width information of the two-dimensional detection box of the target to the height information of the two-dimensional detection box of the target.

[0019] Optionally, the determining the two-dimensional prediction box that matches the two-dimensional detection box includes:

[0020] Construct a cost matrix based on the position information and intersection-over-union information between the two-dimensional detection box and the two-dimensional prediction box, and obtain an optimal assignment scheme based on the cost matrix through the Hungarian algorithm;

[0021] In the optimal assignment scheme, determine whether the cost value between the two-dimensional detection box and the two-dimensional prediction box is not less than a preset threshold. If the cost value is not less than the preset threshold, determine the corresponding two-dimensional prediction box as the two-dimensional prediction box matched with the two-dimensional detection box.

[0022] Optionally, the target ranging method based on the cylindrical diagram detection box further includes:

[0023] For any two-dimensional prediction box in the current frame that has not been matched, determine whether the number of times the two-dimensional prediction box has not been matched satisfies a preset unmatched number threshold;

[0024] If the number of times not being matched satisfies the unmatched number threshold, determine the two-dimensional prediction box as an untrusted two-dimensional prediction box, and directly determine the two-dimensional tracking box according to the two-dimensional detection box.

[0025] Optionally, after determining the three-dimensional position information of the target according to the two-dimensional tracking box of the target, it further includes:

[0026] Perform three-dimensional projection on the two-dimensional tracking boxes of the respective targets in the current frame to obtain three-dimensional detection boxes of the respective targets in the current frame;

[0027] Based on the three-dimensional tracking boxes of the respective targets in the previous frame and the target kinematic equation, obtain three-dimensional prediction boxes of the respective targets in the current frame;

[0028] Determine the three-dimensional prediction box matched with the three-dimensional detection box, update the three-dimensional prediction box matched with the three-dimensional detection box, and obtain the three-dimensional tracking box of the target in the current frame to implement filtering of the three-dimensional position information.

[0029] In a second aspect, the present application discloses a target ranging device based on a cylindrical diagram detection box, including:

[0030] A detection box generation module, configured to perform cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain two-dimensional detection boxes of the respective targets in the current frame according to the cylindrical projection image;

[0031] The prediction box generation module is configured to determine the two-dimensional prediction boxes of the targets in the current frame based on the two-dimensional tracking boxes of the targets in the previous frame and the radial compensation results of the targets; the radial compensation results of the targets are respectively obtained based on the feature information of the two-dimensional detection boxes of the targets in the current frame and the vehicle body perspective control parameters;

[0032] The update ranging module is configured to determine the two-dimensional prediction boxes matched with the two-dimensional detection boxes, update the two-dimensional prediction boxes matched with the two-dimensional detection boxes to obtain the two-dimensional tracking boxes of the targets in the current frame, and determine the three-dimensional position information of the targets according to the two-dimensional tracking boxes of the targets, so as to realize the ranging of the targets.

[0033] In a third aspect, the present application discloses an electronic device, including:

[0034] A memory for storing a computer program;

[0035] A processor for executing the computer program to implement the foregoing disclosed target ranging method based on the cylindrical graph detection box.

[0036] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing disclosed target ranging method based on the cylindrical graph detection box is implemented.

[0037] It can be seen that the present application discloses a target ranging method based on a cylindrical graph detection frame, including: performing cylindrical projection distortion removal processing on the original image collected by the fish-eye camera to obtain a cylindrical projection image, and obtaining the two-dimensional detection frames of each target in the current frame according to the cylindrical projection image; determining the two-dimensional prediction frames of each target in the current frame through the two-dimensional tracking frames of each target in the previous frame and the radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters; determining the two-dimensional prediction frames matched with the two-dimensional detection frames, updating the two-dimensional prediction frames matched with the two-dimensional detection frames to obtain the two-dimensional tracking frames of the targets in the current frame, and determining the three-dimensional position information of the targets according to the two-dimensional tracking frames of the targets to achieve target ranging. In summary, the present application obtains each radial compensation result based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters, and compensates the tracking error in the radial direction of the two-dimensional detection frames of each target based on each radial compensation result, so as to predict the true grounding point of the two-dimensional detection frame of the target, obtain the two-dimensional prediction frame of the target, update the two-dimensional prediction frame, obtain the two-dimensional tracking frame of the target, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve target ranging. In this way, the present application obtains a more accurate grounding point, improves the accuracy of target ranging. In addition, the present application updates the two-dimensional prediction frame, suppresses the detection frame jump noise and the ranging point error noise caused by the change of the self-vehicle perspective, and improves the stability of target ranging. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0039] Figure 1 It is a flowchart of a target ranging method based on a cylindrical graph detection frame disclosed in the present application;

[0040] Figure 2 It is a schematic diagram of a target ranging point disclosed in the present application;

[0041] Figure 3 It is a flowchart of a specific target ranging method based on a cylindrical graph detection frame disclosed in the present application;

[0042] Figure 4 It is a schematic structural diagram of a target ranging device based on a cylindrical graph detection frame disclosed in the present application;

[0043] Figure 5A structural diagram of an electronic device disclosed in this application. Detailed implementation manners

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0045] The current cylindrical diagram detection framework has the following problems when dealing with small targets: 1. Since the size of small targets is small, their representation on the cylindrical diagram may not be clear enough or difficult to accurately detect. Especially for small targets at a long distance, the number of pixel elements they occupy is small, and the distance represented by one pixel in a long-distance scenario can reach dozens of centimeters, resulting in problems with ranging accuracy during ranging. 2. The problem of environmental robustness: Factors such as different lighting conditions and the contrast difference between the target and the background may further affect the ranging effect of small targets, resulting in unstable ranging, and even missed detection and false detection. These problems urgently need to be solved by those skilled in the art.

[0046] Therefore, the embodiment of this application proposes a target ranging scheme based on a cylindrical diagram detection frame, which can improve the target ranging accuracy and ranging stability.

[0047] The embodiment of this application discloses a target ranging method based on a cylindrical diagram detection frame. Refer to Figure 1 As shown, the method includes:

[0048] Step S11: Perform cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain the two-dimensional detection frames of each target in the current frame according to the cylindrical projection image.

[0049] In this embodiment, the original image is collected by a fisheye camera deployed on a vehicle. Since the original image collected by the fisheye camera has distortion, it is necessary to perform cylindrical projection distortion removal processing on the original image to obtain a cylindrical projection image, and perform image enhancement processing, and then load it into a pre-trained YOLO (You Only Look Once) model for inference to obtain the real-time detection result of the fisheye camera at the current timestamp. The attributes of each detected target include the category, confidence, and two-dimensional detection frame information of the target, that is, the central point pixel coordinates in the UV coordinate system and the width and height of the two-dimensional detection frame.

[0050] Step S12: Determine the two-dimensional prediction bounding boxes of the targets in the current frame based on the two-dimensional tracking bounding boxes of the targets in the previous frame and the radial compensation results of the targets; the radial compensation results of the targets are respectively obtained based on the feature information of the two-dimensional detection bounding boxes of the targets in the current frame and the vehicle body perspective control parameters.

[0051] In a real parking scenario, as the vehicle moves, due to the perspective change of the target, there is a difference between the actual center point of the target and the center point of the detection bounding box. Specifically, in the camera coordinate system, the U-direction coordinate of the detection bounding box is the same as the U coordinate of the target, while there is a certain difference between the V-direction coordinate of the detection bounding box and the V coordinate of the real target. There are generally the following two methods in traditional technologies to calculate the position of the target: (1) Use the center point of the detection bounding box as the ranging point to calculate the target position; (2) Use the center point at the bottom of the detection bounding box (i.e., the grounding center point) as the ranging point to calculate the target position. The calculation result using the center point at the bottom of the detection bounding box is slightly smaller than the real value, and the calculation result using the center point of the detection bounding box is slightly larger than the real value with smaller fluctuations and better robustness. In this embodiment, a shrinkage factor is introduced, and the radial compensation results of the targets are calculated based on the shrinkage factor. Then, based on the radial compensation results of the targets, the tracking error of the two-dimensional detection bounding boxes of the targets in the radial direction is compensated, which improves the accuracy of the ranging points of the two-dimensional detection bounding boxes and further improves the accuracy of target ranging.

[0052] Specifically, in this embodiment, the shrinkage factors of the targets are obtained based on the feature information of the two-dimensional detection bounding boxes of the targets and the vehicle body perspective control parameters, and the radial compensation results of the targets are determined according to the shrinkage factors. Then, the ranging points of the two-dimensional tracking bounding boxes of the targets are determined according to the radial compensation results of the targets, and the two-dimensional prediction bounding boxes of the targets in the current frame are determined according to the ranging points of the targets and the two-dimensional tracking bounding boxes of the targets in the previous frame. Among them, when determining the radial compensation results according to the shrinkage factors, the height information of the center point of the detection bounding box of each target is determined, and the height information of the center point of the detection bounding box of each target and the shrinkage factors are respectively multiplied to obtain the radial compensation results. Among them, the feature information of the two-dimensional detection bounding box of the target includes the width and height information of the two-dimensional detection bounding box of the target, and the vehicle body perspective control parameters include the height information of the fisheye camera. Further, based on the width and height information of the two-dimensional detection bounding box of the target and the height information of the fisheye camera, in this embodiment, according to the target model, the shrinkage factors are obtained through the width and height information of the two-dimensional detection bounding box of the target and the height information of the fisheye camera. The target model includes:

[0053] ;

[0054] See Figure 2 as shown, represents the shrinkage factor; represents the weight coefficient; represents the width information of the two-dimensional detection box; represents the height information of the two-dimensional detection box; represents the height information of the fish-eye camera. It can be seen that in the target model, the shrinkage factor is inversely proportional to the height information of the fish-eye camera, and the shrinkage factor is directly proportional to the target ratio. The target ratio is the ratio of the width information of the two-dimensional detection box of the target to the height information of the two-dimensional detection box of the target.

[0055] The two-dimensional prediction boxes of each target in the current frame can be expressed as , , where represents the length of the two-dimensional prediction box, represents the length of the two-dimensional prediction box, represents the width of the two-dimensional prediction box, represents the width of the center point of the two-dimensional detection box, represents the radial compensation result. Further, the two-dimensional prediction boxes of each target can be expressed as , .

[0056] When determining the two-dimensional prediction boxes of each target in the current frame according to the ranging points of each target and the two-dimensional tracking boxes of each target in the previous frame, specifically, it is predicted according to the Kalman filtering method. The prediction formula includes: , represents the state vector corresponding to the two-dimensional prediction box, represents a 2×2 identity matrix, is the two-dimensional tracking box of the target in the previous frame, represents the covariance matrix predicted for the current frame, represents the covariance matrix of the previous frame, is the state transition matrix, set to a 2×2 identity matrix, is the system noise matrix. The noise matrix is used to describe the random noise existing in the dynamic model of the system state, reflecting the uncertainty of the system model. It is usually obtained through experiments or experience. The covariance matrix P is used to describe the uncertainty of the system state estimate. It is continuously updated during the filtering process. With the introduction of new observation values, the covariance matrix will gradually decrease, indicating an increase in the confidence level of the state estimate.

[0057] In summary, in this embodiment, the ranging points of the two-dimensional tracking boxes of each target are optimized according to the shrinkage factor, and more accurate two-dimensional prediction boxes are obtained through the ranging points and the two-dimensional tracking boxes of the previous frame.

[0058] Step S13: Determine the two-dimensional prediction box that matches the two-dimensional detection box, update the two-dimensional prediction box that matches the two-dimensional detection box to obtain the two-dimensional tracking box of the target in the current frame, and determine the three-dimensional position information of the target based on the two-dimensional tracking box of the target, so as to achieve ranging of the target.

[0059] In this embodiment, a cost matrix is constructed based on the position information and intersection over union (IoU) information between the two-dimensional detection box and the two-dimensional prediction box, and the Hungarian algorithm is used to obtain the optimal assignment scheme based on the cost matrix. In the optimal assignment scheme, it is judged whether the cost value between the two-dimensional detection box and the two-dimensional prediction box is not less than a preset threshold. If the cost value is not less than the preset threshold, the corresponding two-dimensional prediction box is determined as the two-dimensional prediction box that matches the two-dimensional detection box.

[0060] Exemplarily, the construction formula of the cost matrix is: , represents the distance between the two-dimensional detection box and the two-dimensional prediction box, and this distance is obtained through the position information. represents the intersection over union of the areas of the two-dimensional detection box and the two-dimensional prediction box. represents the coefficient of the intersection over union, which is used to eliminate the influence caused by the large dimension of the dist and ensure the rationality of the cost matrix. The cost matrix contains the cost values between each two-dimensional detection box and each two-dimensional prediction box. In this embodiment, the optimal assignment scheme is obtained from the cost matrix based on the Hungarian algorithm. Further, it is judged whether the cost value between each two-dimensional detection box and the corresponding two-dimensional prediction box in the optimal assignment scheme is not less than the preset threshold. If the cost value is not less than the preset threshold, the corresponding two-dimensional prediction box is determined as the two-dimensional prediction box that matches the two-dimensional detection box. It should be noted that the preset threshold can be determined based on empirical values.

[0061] Further, the two-dimensional prediction box that matches the two-dimensional detection box is updated according to the Kalman filtering method to obtain the two-dimensional tracking box of the target in the current frame. The update formula includes: , is the Kalman coefficient. is the observation matrix, which can be set as the two-dimensional identity matrix. is the observation noise matrix. is the observation vector. , where represents the abscissa of the ground contact center point of the two-dimensional detection box. represents the ordinate of the ground contact center point of the two-dimensional detection box. is the covariance matrix updated in the current frame.

[0062] In this embodiment, if the cost value between the two-dimensional detection box and the two-dimensional prediction box is less than the preset threshold, it is determined that the two-dimensional detection box does not match the two-dimensional prediction box, and life cycle management is performed on the two-dimensional prediction box. Specifically, based on the number of times the two-dimensional prediction box has not been matched and whether the two-dimensional prediction box in the current frame is matched, it is judged whether the two-dimensional prediction box is determined to be an untrustworthy two-dimensional prediction box according to the preset number-of-unmatched-times threshold. For example, if the two-dimensional prediction box in the current frame is not matched and the two-dimensional prediction box has not been matched 5 times, and the preset number-of-unmatched-times threshold is 3, then the two-dimensional prediction box is determined to be an untrustworthy two-dimensional prediction box, and the two-dimensional detection box is directly determined as the two-dimensional tracking box of the target in the current frame.

[0063] In this embodiment, the three-dimensional position information of the target is determined according to the two-dimensional tracking box of the target to achieve ranging of the target. That is, in this embodiment, the coordinates in the x and y directions in the world coordinate system are obtained by converting the U and V (unit: pixel) coordinates, and the three-dimensional position information of the target is obtained, and the unit is m.

[0064] It can be seen that this application compensates the tracking error in the radial direction of the two-dimensional detection box of each target based on each radial compensation result to obtain a more accurate target ranging point, improves the accuracy of the detection box, and further improves the accuracy of target ranging. Further, this application combines the de-distortion result of the cylindrical diagram and designs a two-dimensional detection box tracking method. The two-dimensional prediction box of the target in the current frame is predicted by the two-dimensional tracking box of the target in the previous frame and the two-dimensional detection box of the target in the current frame, and by updating the two-dimensional prediction box, the noise caused by measurement errors is reduced, the adverse impact on the ranging result due to unstable ranging is suppressed, and the stability of target ranging is improved.

[0065] It can be seen that the present application discloses a target ranging method based on a cylindrical graph detection box, including: performing cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtaining the two-dimensional detection boxes of each target in the current frame according to the cylindrical projection image; determining the two-dimensional prediction boxes of each target in the current frame through the two-dimensional tracking boxes of each target in the previous frame and the radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of the two-dimensional detection boxes of each target in the current frame and the vehicle body perspective control parameters; determining the two-dimensional prediction boxes matched with the two-dimensional detection boxes, updating the two-dimensional prediction boxes matched with the two-dimensional detection boxes to obtain the two-dimensional tracking boxes of the targets in the current frame, and determining the three-dimensional position information of the targets according to the two-dimensional tracking boxes of the targets to achieve target ranging. In summary, the present application obtains each radial compensation result based on the feature information of the two-dimensional detection boxes of each target in the current frame and the vehicle body perspective control parameters, and compensates the tracking error in the radial direction of the two-dimensional detection boxes of each target based on each radial compensation result, so as to predict the true grounding points of the two-dimensional detection boxes of the targets, obtain the two-dimensional prediction boxes of the targets, update the two-dimensional prediction boxes, obtain the two-dimensional tracking boxes of the targets, and determine the three-dimensional position information of the targets according to the two-dimensional tracking boxes of the targets to achieve target ranging. In this way, the present application obtains more accurate grounding points, improves the accuracy of target ranging. In addition, the present application updates the two-dimensional prediction boxes to suppress the detection box jump noise and the ranging point error noise caused by the change of the ego-vehicle perspective, and improves the stability of target ranging.

[0066] Further, in this embodiment, further filtering is performed on the three-dimensional position information of the target to improve the stability of the three-dimensional position information of the target. The embodiment of the present application discloses a specific target ranging method based on a cylindrical graph detection box. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Refer to Figure 3 as shown, specifically including:

[0067] Step S21: Performing three-dimensional projection on the two-dimensional tracking boxes of each of the targets in the current frame to obtain the three-dimensional detection boxes of each of the targets in the current frame.

[0068] Step S22: Obtaining the three-dimensional prediction boxes of each of the targets in the current frame based on the three-dimensional tracking boxes of each of the targets in the previous frame and the target kinematic equation.

[0069] Step S23: Determining the three-dimensional prediction boxes matched with the three-dimensional detection boxes, updating the three-dimensional prediction boxes matched with the three-dimensional detection boxes to obtain the three-dimensional tracking boxes of the targets in the current frame, so as to achieve filtering of the three-dimensional position information.

[0070] That is to say, in this embodiment, the two-dimensional tracking boxes of each of the targets in the current frame are projected three-dimensionally to obtain the three-dimensional detection boxes of each of the targets in the current frame, and the three-dimensional prediction boxes of each of the targets in the current frame are obtained based on the three-dimensional tracking boxes of each of the targets in the previous frame and the target kinematic equation. Then, a cost matrix is constructed between the three-dimensional detection box and the three-dimensional prediction box. The construction method of the cost matrix can be seen in the foregoing disclosed embodiments and will not be elaborated here. Further, based on the Hungarian matching algorithm, an optimal assignment is obtained from the cost matrix, and the three-dimensional prediction box matched with the three-dimensional detection box is determined in combination with the optimal assignment. Then, the three-dimensional prediction box matched with the three-dimensional detection box is updated to obtain the three-dimensional tracking box of the target in the current frame, so as to realize the filtering of the three-dimensional position information.

[0071] Among them, in the three-dimensional prediction box prediction and update stage, the prediction formula and the update formula are the same as the prediction formula and the update formula in the previous content. The difference is that in the three-dimensional prediction, , , and are derived through the target kinematic equation. In the three-dimensional update, , , where represents the time difference between two frames, represents the predicted velocity of the target relative to the ego vehicle in the x direction, represents the predicted velocity of the target relative to the ego vehicle in the y direction, represents the predicted position of the target relative to the ego vehicle in the x direction, represents the predicted position of the target relative to the ego vehicle in the y direction, represents the acceleration of the predicted velocity of the target relative to the ego vehicle in the x direction, represents the acceleration of the predicted velocity of the target relative to the ego vehicle in the y direction, represents the velocity of the ego vehicle in the x direction, represents the velocity of the ego vehicle in the y direction, is the velocity of the target relative to the ego vehicle in the x direction obtained when converting from two dimensions to three dimensions, is the velocity of the target relative to the ego vehicle in the y direction obtained when converting from two dimensions to three dimensions, and are obtained by performing least squares fitting by combining the ranging results of multiple frames of each target.

[0072] In summary, after filtering the two-dimensional prediction box in this application, three-dimensional position information is obtained, and three-dimensional filtering is performed on the three-dimensional position information, further improving the stability of the three-dimensional position information of the target.

[0073] Correspondingly, an embodiment of the present application also discloses a target ranging device based on a cylindrical diagram detection frame. Refer to Figure 4 as shown, the device includes:

[0074] A detection frame generation module, configured to perform cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain two-dimensional detection frames of each target in the current frame according to the cylindrical projection image;

[0075] A prediction frame generation module, configured to determine two-dimensional prediction frames of each target in the current frame through the two-dimensional tracking frames of each target in the previous frame and the radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters;

[0076] An updated ranging module, configured to determine the two-dimensional prediction frame matched with the two-dimensional detection frame, update the two-dimensional prediction frame matched with the two-dimensional detection frame to obtain a two-dimensional tracking frame of the target in the current frame, and determine three-dimensional position information of the target according to the two-dimensional tracking frame of the target, so as to realize ranging of the target.

[0077] Wherein, for the more specific working processes of the above-mentioned modules, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details are not described herein again.

[0078] It can be seen that the present application discloses a target ranging method based on a cylindrical graph detection frame, including: performing cylindrical projection distortion removal processing on the original image collected by a fisheye camera to obtain a cylindrical projection image, and obtaining two-dimensional detection frames of each target in the current frame according to the cylindrical projection image; determining two-dimensional prediction frames of each target in the current frame through two-dimensional tracking frames of each target in the previous frame and radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters; determining the two-dimensional prediction frames matched with the two-dimensional detection frames, updating the two-dimensional prediction frames matched with the two-dimensional detection frames to obtain two-dimensional tracking frames of the targets in the current frame, and determining three-dimensional position information of the targets according to the two-dimensional tracking frames of the targets, so as to realize target ranging. In summary, the present application obtains each radial compensation result based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters, and compensates the tracking error in the radial direction of the two-dimensional detection frames of each target based on each radial compensation result, so as to predict the true grounding point of the two-dimensional detection frame of the target, obtain the two-dimensional prediction frame of the target, update the two-dimensional prediction frame, obtain the two-dimensional tracking frame of the target, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target, so as to realize target ranging. In this way, the present application obtains a more accurate grounding point, improves the accuracy of target ranging. In addition, the present application updates the two-dimensional prediction frame, suppresses the detection frame jump noise and the ranging point error noise caused by the change of the ego-vehicle perspective, and improves the stability of target ranging.

[0079] Furthermore, an embodiment of the present application also provides an electronic device. Figure 5 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure should not be considered as any limitation to the scope of use of the present application.

[0080] Figure 5 It is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the target ranging method based on the cylindrical graph detection frame disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0081] In this embodiment, the power supply 26 is used to provide operating voltages for the various hardware devices on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and specific limitations thereof are not provided herein; the input / output interface 24 is used to obtain external input data or output data to the outside, and the specific interface type thereof can be selected according to specific application requirements, and specific limitations are not provided herein.

[0082] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disk, etc., and the resources stored thereon can include a computer program 221, and the storage method can be transient storage or permanent storage. Among them, in addition to the computer program that can be used to complete the target ranging method based on the cylindrical graph detection frame executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 can further include a computer program that can be used to complete other specific tasks.

[0083] Furthermore, an embodiment of this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the target ranging method based on the cylindrical graph detection frame disclosed above is implemented.

[0084] For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details are not described herein again.

[0085] The various embodiments in this application are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts between the various embodiments, reference can be made to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and reference can be made to the description in the method part for related parts.

[0086] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0087] The steps of the methods or algorithms described in combination with the embodiments disclosed in this document can be implemented directly by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0088] Finally, it should also be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0089] The above has introduced in detail a target ranging method, device, equipment, and storage medium based on a cylindrical graph detection frame provided by this application. Specific examples are used in this document to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A target ranging method based on a cylindrical graph detection frame, characterized in that including: performing cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtaining two-dimensional detection frames of each target in the current frame according to the cylindrical projection image; determining two-dimensional prediction frames of each target in the current frame based on two-dimensional tracking frames of each target in the previous frame and radial compensation results of each target; the radial compensation results of each target are respectively obtained based on feature information of two-dimensional detection frames of each target in the current frame and vehicle body perspective control parameters; determining the two-dimensional prediction frame matched with the two-dimensional detection frame, updating the two-dimensional prediction frame matched with the two-dimensional detection frame to obtain a two-dimensional tracking frame of the target in the current frame, and determining three-dimensional position information of the target according to the two-dimensional tracking frame of the target to realize ranging of the target; wherein, the feature information of the two-dimensional detection frame of the target includes the width and height information of the two-dimensional detection frame of the target, and the vehicle body perspective control parameter includes the height information of the fisheye camera; then the determination process of the radial compensation results of each target includes: obtaining respective shrinkage factors according to the target model and through the width and height information of the two-dimensional detection frames of each target and the height information of the fisheye camera, and determining the radial compensation results of each target according to the respective shrinkage factors; wherein, in the target model, the shrinkage factor and the height information of the fisheye camera are in an inverse ratio relationship, the shrinkage factor and the target ratio are in a direct ratio relationship, and the target ratio is the ratio of the width information of the two-dimensional detection frame of the target to the height information of the two-dimensional detection frame of the target.

2. The target ranging method based on the detection frame of the cylindrical diagram according to claim 1, characterized in that The step of determining two-dimensional prediction frames of each target in the current frame based on two-dimensional tracking frames of each target in the previous frame and radial compensation results of each target includes: determining target ranging points of two-dimensional tracking frames of each target according to the radial compensation results of each target, and determining two-dimensional prediction frames of each target in the current frame according to the respective target ranging points and two-dimensional tracking frames of each target in the previous frame.

3. The target ranging method based on the detection frame of the cylinder graph according to claim 2, wherein, The step of determining the radial compensation results of each target according to the respective shrinkage factors includes: determining height information of the center point of the detection frame of the two-dimensional detection frame of each target; performing multiplication operations on the height information of the center point of the detection frame of the two-dimensional detection frame of each target and the respective shrinkage factors to obtain the radial compensation results of each target.

4. The target ranging method based on the cylindrical graph detection frame according to claim 1, wherein, The step of determining the two-dimensional prediction frame matched with the two-dimensional detection frame includes: constructing a cost matrix according to position information and intersection over union information between the two-dimensional detection frame and the two-dimensional prediction frame, and obtaining an optimal assignment scheme based on the cost matrix through the Hungarian algorithm; in the optimal assignment scheme, determining whether the cost value between the two-dimensional detection frame and the two-dimensional prediction frame is not less than a preset threshold, and if the cost value is not less than the preset threshold, determining the corresponding two-dimensional prediction frame as the two-dimensional prediction frame matched with the two-dimensional detection frame.

5. The target ranging method based on the cylindrical graph detection frame according to claim 1, characterized in that It further includes: for any two-dimensional prediction frame in the current frame that is not matched, determining whether the number of times the two-dimensional prediction frame is not matched satisfies a preset unmatched number threshold; If the number of times of non - matching satisfies the non - matching times threshold, the two - dimensional prediction box is determined as an untrusted two - dimensional prediction box, and the two - dimensional tracking box is directly determined according to the two - dimensional detection box.

6. The target ranging method based on the cylindrical graph detection frame according to any one of claims 1 to 5, characterized in that, After determining the three - dimensional position information of the target according to the two - dimensional tracking box of the target, it further includes: Performing three - dimensional projection on the two - dimensional tracking boxes of each target in the current frame to obtain three - dimensional detection boxes of each target in the current frame; Based on the three - dimensional tracking boxes of each target in the previous frame and the target kinematic equation, obtaining three - dimensional prediction boxes of each target in the current frame; Determining the three - dimensional prediction box matched with the three - dimensional detection box, updating the three - dimensional prediction box matched with the three - dimensional detection box, and obtaining the three - dimensional tracking box of the target in the current frame to implement filtering of the three - dimensional position information.

7. The target ranging method based on the detection frame of the cylindrical diagram according to claim 2, characterized in that The determining of the two - dimensional prediction boxes of each target in the current frame according to each target ranging point and the two - dimensional tracking boxes of each target in the previous frame includes: Predicting the two - dimensional prediction boxes of each target in the current frame according to the Kalman filtering method and prediction formula; wherein, the parameters in the prediction formula include the identity matrix, the two - dimensional tracking box of the target in the previous frame, the covariance matrix in the previous frame, the state transition matrix, and the system noise matrix.

8. A target ranging device based on a cylindrical graph detection frame, characterized in that, It includes: A detection box generation module, configured to perform cylindrical projection distortion removal processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain two - dimensional detection boxes of each target in the current frame according to the cylindrical projection image; A prediction box generation module, configured to determine the two - dimensional prediction boxes of each target in the current frame through the two - dimensional tracking boxes of each target in the previous frame and the radial compensation results of each target; the radial compensation results of each target are respectively obtained based on the feature information of the two - dimensional detection boxes of each target in the current frame and the vehicle body perspective control parameters; An updated ranging module, configured to determine the two - dimensional prediction box matched with the two - dimensional detection box, update the two - dimensional prediction box matched with the two - dimensional detection box, obtain the two - dimensional tracking box of the target in the current frame, and determine the three - dimensional position information of the target according to the two - dimensional tracking box of the target to implement ranging of the target; Wherein, the feature information of the two - dimensional detection box of the target includes the width and height information of the two - dimensional detection box of the target, and the vehicle body perspective control parameters include the height information of the fisheye camera; Then the determination process of the radial compensation results of each target includes: Obtaining each shrinkage factor according to the target model through the width and height information of the two - dimensional detection boxes of each target and the height information of the fisheye camera, and determining the radial compensation results of each target according to each shrinkage factor; Wherein, in the target model, the shrinkage factor and the height information of the fisheye camera are in an inverse relationship, the shrinkage factor and the target ratio are in a direct relationship, and the target ratio is the ratio of the width information of the two - dimensional detection box of the target to the height information of the two - dimensional detection box of the target.

9. An electronic device, characterized in that, It includes: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the target ranging method based on the cylindrical graph detection box according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, it implements the target ranging method based on a cylinder graph detection frame according to any one of claims 1 to 6.

Images