Target ranging method, device and equipment based on cylindrical diagram detection frame and medium

Abstract

The invention discloses a target distance measurement method, device and equipment based on a cylindrical diagram detection frame and a medium, and relates to the technical field of computers, and the method comprises the steps: carrying out the cylindrical projection distortion removal processing of an original image collected by a fisheye camera, obtaining a cylindrical projection image, and obtaining a two-dimensional detection frame of each target of a current frame according to the cylindrical projection image; determining a two-dimensional prediction frame of each target of the current frame through the two-dimensional tracking frame of each target of the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target of the current frame and the vehicle body view angle control parameter; and determining a two-dimensional prediction frame matched with the two-dimensional detection frame, updating the two-dimensional prediction frame matched with the two-dimensional detection frame to obtain a two-dimensional tracking frame of the target of the current frame, and determining three-dimensional position information of the target according to the two-dimensional tracking frame of the target to realize target distance measurement. The distance measurement precision and the distance measurement stability of the target are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a target distance measurement method, device, equipment and medium based on a cylindrical image detection frame. Background Art

[0002] The rapid development of computer vision has brought technological innovation to the environmental perception of smart cars, and its applications in driver assistance systems and intelligent driving are becoming increasingly widespread. The surround-view fisheye system based on pure vision has gradually become the mainstream perception solution of major OEMs due to its low cost, low energy consumption, and low computing power.

[0003] Although the surround fisheye system has a very large viewing angle, the fisheye camera image has strong nonlinear distortion, and the closer the pixel is to the edge of the image, the greater the distortion is, which is not conducive to the development of target detection algorithms. A common solution is to convert the fisheye image into other forms of images through calibration, such as converting it into a cylindrical image through a cylindrical image detection framework. However, the current cylindrical image detection framework has the following problems when processing small targets: 1. Due to the small size of small targets, their representation on the cylindrical image may not be clear enough or difficult to detect accurately, especially for small targets at a distance, which occupy fewer pixel elements. The distance represented by one pixel in a long-distance scene can reach tens of centimeters, and there will inevitably be problems with ranging accuracy when measuring distance. 2. Environmental robustness issues: Different lighting conditions, differences in contrast between the target and the background, and other factors may further affect the ranging effect of small targets, resulting in unstable ranging, or even missed detection and false detection. Therefore, the above problems urgently need to be solved by technical personnel in this field. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a target ranging method, device, equipment and medium based on a cylindrical image detection frame, which improves the ranging accuracy and ranging stability. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a target distance measurement method based on a cylindrical image detection frame, comprising:

[0006] Perform cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image;

[0007] Determine the two-dimensional prediction frame of each target in the current frame by using the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body view angle control parameter;

[0008] Determine the two-dimensional prediction frame that matches the two-dimensional detection frame, update the two-dimensional prediction frame that matches the two-dimensional detection frame, obtain the two-dimensional tracking frame of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve ranging of the target.

[0009] Optionally, determining the two-dimensional prediction frame of each of the targets in the current frame by using the two-dimensional tracking frame of each of the targets in the previous frame and the radial compensation result of each of the targets includes:

[0010] Obtaining each shrinkage factor through the feature information of the two-dimensional detection frame of each target and the vehicle body viewing angle control parameter, and determining the radial compensation result of each target according to each shrinkage factor;

[0011] The target ranging points of the two-dimensional tracking frames of the targets are determined according to the radial compensation results of the targets, and the two-dimensional prediction frames of the targets in the current frame are determined according to the target ranging points and the two-dimensional tracking frames of the targets in the previous frame.

[0012] Optionally, determining the radial compensation result of each of the targets according to each of the shrinkage factors includes:

[0013] Determine the height information of the center point of the two-dimensional detection frame of each target;

[0014] The height information of the center point of the two-dimensional detection frame of each target is multiplied by each of the shrinkage factors to obtain the radial compensation result of each target.

[0015] Optionally, the characteristic information of the two-dimensional detection frame of the target includes width and height information of the two-dimensional detection frame of the target, and the vehicle body viewing angle control parameter includes height information of the fisheye camera;

[0016] Correspondingly, the shrinkage factors are obtained by using the feature information of the two-dimensional detection frames of the targets and the vehicle body viewing angle control parameters, including:

[0017] Obtaining each shrinkage factor according to the target model and through the width and height information of the two-dimensional detection frame of each target and the height information of the fisheye camera;

[0018] Among them, in the target model, the shrinkage factor is inversely proportional to the height information of the fisheye camera, and the shrinkage factor is directly proportional to the target ratio, and the target ratio is the ratio of the width information of the two-dimensional detection frame of the target to the width information of the two-dimensional detection frame of the target.

[0019] Optionally, determining the two-dimensional prediction box that matches the two-dimensional detection box includes:

[0020] Constructing a cost matrix according to the position information and intersection-over-union information between the two-dimensional detection frame and the two-dimensional prediction frame, and obtaining an optimal assignment scheme based on the cost matrix by using the Hungarian algorithm;

[0021] In the optimal assignment scheme, it is determined whether the cost value between the two-dimensional detection box and the two-dimensional prediction box is not less than a preset threshold. If the cost value is not less than the preset threshold, the corresponding two-dimensional prediction box is determined as the two-dimensional prediction box matched by the two-dimensional detection box.

[0022] Optionally, the target distance measurement method based on the cylindrical image detection frame further includes:

[0023] For any unmatched two-dimensional prediction frame of the current frame, determining whether the number of times the two-dimensional prediction frame is unmatched meets a preset unmatched number threshold;

[0024] If the number of unmatched times meets the unmatched times threshold, the two-dimensional prediction frame is determined as an untrustworthy two-dimensional prediction frame, and the two-dimensional tracking frame is directly determined according to the two-dimensional detection frame.

[0025] Optionally, after determining the three-dimensional position information of the target according to the two-dimensional tracking frame of the target, the method further includes:

[0026] Performing three-dimensional projection on the two-dimensional tracking frame of each target in the current frame to obtain a three-dimensional detection frame of each target in the current frame;

[0027] Obtaining a three-dimensional prediction frame of each target in the current frame based on the three-dimensional tracking frame of each target in the previous frame and the target kinematic equation;

[0028] The three-dimensional prediction frame matched by the three-dimensional detection frame is determined, and the three-dimensional prediction frame matched by the three-dimensional detection frame is updated to obtain a three-dimensional tracking frame of the target in the current frame, so as to implement filtering of the three-dimensional position information.

[0029] In a second aspect, the present application discloses a target distance measurement device based on a cylindrical image detection frame, comprising:

[0030] A detection frame generation module is used to perform cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image;

[0031] A prediction frame generation module is used to determine the two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body view control parameter;

[0032] The updating ranging module is used to determine the two-dimensional prediction frame that matches the two-dimensional detection frame, update the two-dimensional prediction frame that matches the two-dimensional detection frame, obtain the two-dimensional tracking frame of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve ranging of the target.

[0033] In a third aspect, the present application discloses an electronic device, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is used to execute the computer program to implement the target ranging method based on the cylindrical image detection frame disclosed above.

[0036] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the target ranging method based on a cylindrical image detection frame disclosed above is implemented.

[0037] It can be seen that the present application discloses a target ranging method based on a cylindrical image detection frame, including: performing cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtaining a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image; determining a two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body perspective control parameter; determining a two-dimensional prediction frame matched by the two-dimensional detection frame, updating the two-dimensional prediction frame matched by the two-dimensional detection frame, obtaining a two-dimensional tracking frame of the target in the current frame, and determining the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve the ranging of the target. In summary, it can be seen that the present application obtains various radial compensation results based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters, and compensates for the radial tracking errors of the two-dimensional detection frames of each target based on the various radial compensation results, so as to predict the actual grounding point of the two-dimensional detection frame of the target, obtain the two-dimensional prediction frame of the target, and update the two-dimensional prediction frame to obtain the two-dimensional tracking frame of the target, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve the ranging of the target. In this way, the present application obtains a more accurate grounding point and improves the accuracy of target ranging. In addition, the present application suppresses the detection frame jump noise and the ranging point error noise caused by the change of the vehicle perspective by updating the two-dimensional prediction frame, thereby improving the stability of target ranging. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flow chart of a target distance measurement method based on a cylindrical image detection frame disclosed in this application;

[0040] Figure 2 A schematic diagram of a target ranging point disclosed in the present application;

[0041] Figure 3 A flow chart of a specific target distance measurement method based on a cylindrical image detection frame disclosed in this application;

[0042] Figure 4 This is a schematic diagram of the structure of a target distance measurement device based on a cylindrical image detection frame disclosed in this application;

[0043] Figure 5This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The current cylindrical image detection framework has the following problems when processing small targets: 1. Due to the small size of small targets, their representation on the cylindrical image may not be clear enough or difficult to detect accurately, especially for small targets at a distance. They occupy fewer pixel elements, and the distance represented by one pixel in a long-distance scene can reach tens of centimeters, which causes distance measurement accuracy problems. 2. Environmental robustness issues: Different lighting conditions, target-background contrast differences and other factors may further affect the ranging effect of small targets, resulting in unstable ranging, or even missed detections and false detections. The above problems urgently need to be solved by those skilled in the art.

[0046] To this end, the embodiment of the present application proposes a target ranging solution based on a cylindrical image detection frame, which can improve the target ranging accuracy and ranging stability.

[0047] The present application discloses a method for measuring target distance based on a cylindrical image detection frame. Figure 1 As shown, the method includes:

[0048] Step S11: performing cylindrical projection dedistortion processing on the original image captured by the fisheye camera to obtain a cylindrical projection image, and obtaining a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image.

[0049] In this embodiment, the original image is collected by a fisheye camera deployed on the vehicle. Since the original image collected by the fisheye camera is distorted, it is necessary to perform cylindrical projection dedistortion processing on the original image to obtain a cylindrical projection image, and perform image enhancement processing, and then load it into a pre-trained YOLO (You Only Look Once) model for inference to obtain the real-time detection result of the fisheye camera at the current timestamp, and the attributes of each detected target, including the target category, confidence and two-dimensional detection box information, that is, the center point pixel coordinates in the UV coordinate system and the width and height of the two-dimensional detection box.

[0050] Step S12: Determine the two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body perspective control parameter.

[0051] In a real parking scene, as the vehicle moves, due to the change in the viewing angle of the target, there is a difference between the actual center point of the target and the center point of the detection frame. Specifically, in the camera coordinate system, the U-direction coordinate of the detection frame is consistent with the U coordinate of the target, while the V-direction coordinate of the detection frame and the V coordinate of the real target are somewhat different. Traditional technology generally has the following two ways to calculate the position of the target: (1) using the center point of the detection frame as the ranging point to calculate the target position; (2) using the bottom center point of the detection frame as the ranging point (that is, the ground center point) to calculate the target position. The calculation result using the bottom center point of the detection frame is slightly smaller than the true value, and the calculation result using the center point of the detection frame is slightly larger than the true value but has less fluctuation and better robustness. This embodiment introduces a shrinkage factor, and calculates the radial compensation result of each target based on the shrinkage factor, and then compensates for the radial tracking error of the two-dimensional detection frame of each target based on the radial compensation result of each target, thereby improving the accuracy of the two-dimensional detection frame ranging point, thereby improving the accuracy of the target ranging.

[0052] Specifically, this embodiment obtains each shrinkage factor through the characteristic information of the two-dimensional detection frame of each target and the vehicle body view angle control parameter, and determines the radial compensation result of each target according to each shrinkage factor, and then determines the target ranging point of the two-dimensional tracking frame of each target according to the radial compensation result of each target, and determines the two-dimensional prediction frame of each target in the current frame according to each target ranging point and the two-dimensional tracking frame of each target in the previous frame. Wherein, when determining each radial compensation result according to each shrinkage factor, the height information of the center point of the detection frame of the two-dimensional detection frame of each target is determined, and the height information of the center point of the detection frame of the two-dimensional detection frame of each target is multiplied by each shrinkage factor respectively to obtain each radial compensation result. Wherein, the characteristic information of the two-dimensional detection frame of the target includes the width and height information of the two-dimensional detection frame of the target, and the vehicle body view angle control parameter includes the height information of the fisheye camera. Further, based on the width and height information of the two-dimensional detection frame of the target and the height information of the fisheye camera, this embodiment obtains each shrinkage factor according to the target model and through the width and height information of the two-dimensional detection frame of each target and the height information of the fisheye camera, and the target model includes:

[0053] ;

[0054] See also Figure 2 As shown, represents the shrinkage factor; represents the weight coefficient; Indicates the width information of the two-dimensional detection frame; Indicates the height information of the two-dimensional detection frame; Represents the height information of the fisheye camera. It can be seen that in the target model, the shrinkage factor is inversely proportional to the height information of the fisheye camera, and the shrinkage factor is directly proportional to the target ratio. The target ratio is the ratio of the width information of the two-dimensional detection frame of the target to the width information of the two-dimensional detection frame of the target.

[0055] The two-dimensional prediction box of each target in the current frame can be expressed as , ,in, represents the length of the two-dimensional prediction box, represents the length of the two-dimensional prediction box, Represents the width of the two-dimensional prediction box, Indicates the width of the center point of the two-dimensional detection box. represents the radial compensation result. Furthermore, the two-dimensional prediction box of each target can be expressed as , .

[0056] The two-dimensional prediction frame of each target in the current frame is determined according to each target ranging point and the two-dimensional tracking frame of each target in the previous frame. Specifically, it is predicted according to the Kalman filter method. The prediction formula includes: , Represents the state vector corresponding to the two-dimensional prediction box, represents the 2*2 identity matrix, is the 2D tracking box of the target in the previous frame, represents the covariance matrix predicted by the current frame, represents the covariance matrix of the previous frame, is the state transfer matrix, set to a 2*2 identity matrix, is the system noise matrix. The noise matrix is ​​used to describe the random noise existing in the dynamic model of the system state, reflecting the uncertainty of the system model. It is usually obtained through experiments or empirical estimates. The covariance matrix P is used to describe the uncertainty of the system state estimate. It is continuously updated during the filtering process. With the introduction of new observations, the covariance matrix will gradually decrease, indicating that the confidence in the state estimate is improved.

[0057] In summary, this embodiment optimizes the target ranging points of the two-dimensional tracking frame of each target according to the shrinkage factor, and obtains a more accurate two-dimensional prediction frame through the target ranging points and the two-dimensional tracking frame of the previous frame.

[0058] Step S13: determine the two-dimensional prediction frame that matches the two-dimensional detection frame, update the two-dimensional prediction frame that matches the two-dimensional detection frame, obtain the two-dimensional tracking frame of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve ranging of the target.

[0059] In this embodiment, a cost matrix is ​​constructed according to the position information and intersection-and-union ratio information between the two-dimensional detection frame and the two-dimensional prediction frame, and an optimal assignment scheme is obtained based on the cost matrix through the Hungarian algorithm. In the optimal assignment scheme, it is determined whether the cost value between the two-dimensional detection frame and the two-dimensional prediction frame is not less than a preset threshold. If the cost value is not less than the preset threshold, the corresponding two-dimensional prediction frame is determined as the two-dimensional prediction frame matched by the two-dimensional detection frame.

[0060] Exemplarily, the cost matrix is ​​constructed as follows: , represents the distance between the two-dimensional detection box and the two-dimensional prediction box, which is obtained through position information, It represents the area intersection ratio of the two-dimensional detection box and the two-dimensional prediction box. The coefficient representing the intersection-over-union ratio is used to eliminate the influence caused by the large dimension of dist and ensure the rationality of the cost matrix. The cost matrix contains the cost values ​​between each two-dimensional detection frame and each two-dimensional prediction frame. In this embodiment, the optimal assignment scheme is obtained from the cost matrix based on the Hungarian algorithm. Further, it is determined whether the cost value of each two-dimensional detection frame and the corresponding two-dimensional prediction frame in the optimal assignment scheme is not less than a preset threshold. If the cost value is not less than the preset threshold, the corresponding two-dimensional prediction frame is determined as the two-dimensional prediction frame matched by the two-dimensional detection frame. It should be pointed out that the preset threshold can be determined based on an empirical value.

[0061] Furthermore, the two-dimensional prediction frame matched by the two-dimensional detection frame is updated according to the Kalman filtering method to obtain the two-dimensional tracking frame of the target in the current frame. The update formula includes: , is the Kalman coefficient, is the observation matrix, which can be set to a two-dimensional identity matrix, is the observation noise matrix, is the observation vector, ,in, Represents the horizontal coordinate of the ground center point of the two-dimensional detection frame, Indicates the vertical coordinate of the ground center point of the two-dimensional detection frame, is the updated covariance matrix of the current frame.

[0062] In this embodiment, if the cost between the two-dimensional detection frame and the two-dimensional prediction frame is less than a preset threshold, it is determined that the two-dimensional detection frame does not match the two-dimensional prediction frame, and the two-dimensional prediction frame is managed in its life cycle. Specifically, based on the number of times the two-dimensional prediction frame has not been matched and whether the two-dimensional prediction frame of the current frame has been matched, a preset unmatched number threshold is used to determine whether the two-dimensional prediction frame is determined as an untrustworthy two-dimensional prediction frame. For example, if the two-dimensional prediction frame of the current frame has not been matched, the two-dimensional prediction frame has not been matched 5 times, and the preset unmatched number threshold is 3, the two-dimensional prediction frame is determined as an untrustworthy two-dimensional prediction frame, and the two-dimensional detection frame is directly determined as the two-dimensional tracking frame of the target of the current frame.

[0063] In this embodiment, the three-dimensional position information of the target is determined according to the two-dimensional tracking frame of the target to achieve the distance measurement of the target, that is, in this embodiment, the U, V (unit is pixel) coordinates are converted to obtain the coordinates in the x, y direction in the world coordinate system to obtain the three-dimensional position information of the target, the unit is m.

[0064] It can be seen that the present application compensates for the radial tracking error of the two-dimensional detection frame of each target based on each radial compensation result to obtain a more accurate target ranging point, thereby improving the accuracy of the detection frame and thus improving the accuracy of the target ranging. Furthermore, the present application designs a two-dimensional detection frame tracking method in combination with the dedistortion result of the cylindrical image, and obtains the two-dimensional prediction frame of the current frame target by predicting the two-dimensional tracking frame of the previous frame target and the two-dimensional detection frame of the current frame target, and by updating the two-dimensional prediction frame, reduces the noise caused by the measurement error, suppresses the adverse effect of the ranging result due to the unstable ranging, and improves the stability of the target ranging.

[0065] It can be seen that the present application discloses a target ranging method based on a cylindrical image detection frame, including: performing cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtaining a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image; determining a two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body perspective control parameter; determining a two-dimensional prediction frame matched by the two-dimensional detection frame, updating the two-dimensional prediction frame matched by the two-dimensional detection frame, obtaining a two-dimensional tracking frame of the target in the current frame, and determining the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve the ranging of the target. In summary, it can be seen that the present application obtains various radial compensation results based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters, and compensates for the radial tracking errors of the two-dimensional detection frames of each target based on the various radial compensation results, so as to predict the actual grounding point of the two-dimensional detection frame of the target, obtain the two-dimensional prediction frame of the target, and update the two-dimensional prediction frame to obtain the two-dimensional tracking frame of the target, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve the ranging of the target. In this way, the present application obtains a more accurate grounding point and improves the accuracy of target ranging. In addition, the present application suppresses the detection frame jump noise and the ranging point error noise caused by the change of the vehicle perspective by updating the two-dimensional prediction frame, thereby improving the stability of target ranging.

[0066] Furthermore, this embodiment further filters the three-dimensional position information of the target to improve the stability of the three-dimensional position information of the target. The embodiment of the present application discloses a specific target distance measurement method based on a cylindrical image detection frame. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Figure 3 As shown, specifically including:

[0067] Step S21: performing three-dimensional projection on the two-dimensional tracking frame of each target in the current frame to obtain a three-dimensional detection frame of each target in the current frame.

[0068] Step S22: obtaining the three-dimensional prediction frames of the targets in the current frame based on the three-dimensional tracking frames of the targets in the previous frame and the target kinematic equations.

[0069] Step S23: determining the three-dimensional prediction frame that matches the three-dimensional detection frame, updating the three-dimensional prediction frame that matches the three-dimensional detection frame, and obtaining the three-dimensional tracking frame of the target in the current frame to implement filtering of the three-dimensional position information.

[0070] That is to say, this embodiment performs three-dimensional projection on the two-dimensional tracking frames of each target in the current frame to obtain the three-dimensional detection frames of each target in the current frame, and obtains the three-dimensional prediction frames of each target in the current frame based on the three-dimensional tracking frames of each target in the previous frame and the target kinematic equation, and then constructs a cost matrix between the three-dimensional detection frame and the three-dimensional prediction frame. The method for constructing the cost matrix is ​​shown in the aforementioned disclosed embodiment and will not be repeated here. Furthermore, based on the Hungarian matching algorithm, the optimal assignment is obtained from the cost matrix, and the three-dimensional prediction frame matched by the three-dimensional detection frame is determined in combination with the optimal assignment, and then the three-dimensional prediction frame matched by the three-dimensional detection frame is updated to obtain the three-dimensional tracking frame of the target in the current frame, so as to achieve filtering of the three-dimensional position information.

[0071] Among them, in the three-dimensional prediction frame prediction and update stage, the prediction formula and update formula are the same as the prediction formula and update formula in the previous content. The difference is that in the three-dimensional prediction, , , and It is derived from the target kinematic equation that when updating in three dimensions, , ,in, Represents the time difference between two frames, It represents the predicted x-direction speed of the target relative to the vehicle. It represents the predicted y-direction speed of the target relative to the vehicle. It represents the predicted x-direction position of the target relative to the vehicle. It indicates the predicted y-direction position of the target relative to the vehicle. represents the predicted acceleration of the target relative to the x-direction velocity of the vehicle, It represents the predicted acceleration of the target relative to the vehicle in the y direction. represents the speed of the vehicle in the x direction, represents the speed of the vehicle in the y direction, is the x-direction velocity of the target relative to the vehicle obtained when converting from 2D to 3D, is the y-direction velocity of the target relative to the vehicle obtained when converting from 2D to 3D, and It is obtained by combining the ranging results of multiple frames of each target and performing least squares fitting.

[0072] From the above, it can be seen that after filtering the two-dimensional prediction frame, the present application obtains three-dimensional position information, and performs three-dimensional filtering on the three-dimensional position information to further improve the stability of the three-dimensional position information of the target.

[0073] Correspondingly, the embodiment of the present application also discloses a target distance measurement device based on a cylindrical image detection frame, see Figure 4 As shown, the device comprises:

[0074] A detection frame generation module is used to perform cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image;

[0075] A prediction frame generation module is used to determine the two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body view control parameter;

[0076] The updating ranging module is used to determine the two-dimensional prediction frame that matches the two-dimensional detection frame, update the two-dimensional prediction frame that matches the two-dimensional detection frame, obtain the two-dimensional tracking frame of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve ranging of the target.

[0077] Among them, for more specific working processes of the above-mentioned modules, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0078] It can be seen that the present application discloses a target ranging method based on a cylindrical image detection frame, including: performing cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtaining a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image; determining a two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body perspective control parameter; determining a two-dimensional prediction frame matched by the two-dimensional detection frame, updating the two-dimensional prediction frame matched by the two-dimensional detection frame, obtaining a two-dimensional tracking frame of the target in the current frame, and determining the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve the ranging of the target. In summary, it can be seen that the present application obtains various radial compensation results based on the feature information of the two-dimensional detection frames of each target in the current frame and the vehicle body perspective control parameters, and compensates for the radial tracking errors of the two-dimensional detection frames of each target based on the various radial compensation results, so as to predict the actual grounding point of the two-dimensional detection frame of the target, obtain the two-dimensional prediction frame of the target, and update the two-dimensional prediction frame to obtain the two-dimensional tracking frame of the target, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve the ranging of the target. In this way, the present application obtains a more accurate grounding point and improves the accuracy of target ranging. In addition, the present application suppresses the detection frame jump noise and the ranging point error noise caused by the change of the vehicle perspective by updating the two-dimensional prediction frame, thereby improving the stability of target ranging.

[0079] Furthermore, an embodiment of the present application also provides an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0080] Figure 5 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input and output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the target ranging method based on the cylindrical image detection frame disclosed in any of the above embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0081] In this embodiment, the power supply 26 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 24 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0082] In addition, the memory 22, as a carrier for storing resources, may be a read-only memory, a random access memory, a disk or an optical disk, etc., and the resources stored thereon may include a computer program 221, and the storage method may be temporary storage or permanent storage. In addition to including a computer program that can be used to complete the target ranging method based on the cylindrical image detection frame performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 221 may further include a computer program that can be used to complete other specific tasks.

[0083] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the target ranging method based on a cylindrical image detection frame disclosed above is implemented.

[0084] For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0085] The various embodiments in this application are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0086] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0087] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0088] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0089] The above is a detailed introduction to a target ranging method, device, equipment, and storage medium based on a cylindrical image detection frame provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A target distance measurement method based on a cylindrical image detection frame, characterized in that: include: Perform cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image; Determine the two-dimensional prediction frame of each target in the current frame by using the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body view angle control parameter; Determine the two-dimensional prediction frame that matches the two-dimensional detection frame, update the two-dimensional prediction frame that matches the two-dimensional detection frame, obtain the two-dimensional tracking frame of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve ranging of the target.

2. The target distance measurement method based on cylindrical image detection frame according to claim 1, characterized in that: The determining of the two-dimensional prediction frame of each of the targets in the current frame by using the two-dimensional tracking frame of each of the targets in the previous frame and the radial compensation result of each of the targets includes: Obtaining each shrinkage factor through feature information of the two-dimensional detection frame of each target and the vehicle body viewing angle control parameter, and determining a radial compensation result of each target according to each shrinkage factor; The target ranging points of the two-dimensional tracking frames of the targets are determined according to the radial compensation results of the targets, and the two-dimensional prediction frames of the targets in the current frame are determined according to the target ranging points and the two-dimensional tracking frames of the targets in the previous frame.

3. The target distance measurement method based on cylindrical image detection frame according to claim 2, characterized in that: Determining the radial compensation result of each of the targets according to each of the shrinkage factors includes: Determine the height information of the center point of the two-dimensional detection frame of each target; The height information of the center point of the two-dimensional detection frame of each target is multiplied by each of the shrinkage factors to obtain the radial compensation result of each target.

4. The target distance measurement method based on cylindrical image detection frame according to claim 3 is characterized in that: The characteristic information of the two-dimensional detection frame of the target includes width and height information of the two-dimensional detection frame of the target, and the vehicle body viewing angle control parameter includes height information of the fisheye camera; Correspondingly, the shrinkage factors are obtained by using the feature information of the two-dimensional detection frames of the targets and the vehicle body viewing angle control parameters, including: Obtaining each shrinkage factor according to the target model and through the width and height information of the two-dimensional detection frame of each target and the height information of the fisheye camera; Among them, in the target model, the shrinkage factor is inversely proportional to the height information of the fisheye camera, and the shrinkage factor is directly proportional to the target ratio, and the target ratio is the ratio of the width information of the two-dimensional detection frame of the target to the width information of the two-dimensional detection frame of the target.

5. The target distance measurement method based on cylindrical image detection frame according to claim 1, characterized in that: The determining the two-dimensional prediction frame that matches the two-dimensional detection frame includes: Constructing a cost matrix according to the position information and intersection-over-union information between the two-dimensional detection frame and the two-dimensional prediction frame, and obtaining an optimal assignment scheme based on the cost matrix by using the Hungarian algorithm; In the optimal assignment scheme, it is determined whether the cost value between the two-dimensional detection box and the two-dimensional prediction box is not less than a preset threshold. If the cost value is not less than the preset threshold, the corresponding two-dimensional prediction box is determined as the two-dimensional prediction box matched by the two-dimensional detection box.

6. The target distance measurement method based on cylindrical image detection frame according to claim 1, characterized in that: Also includes: For any unmatched two-dimensional prediction frame of the current frame, determining whether the number of times the two-dimensional prediction frame is unmatched meets a preset unmatched number threshold; If the number of unmatched times meets the unmatched times threshold, the two-dimensional prediction frame is determined as an untrustworthy two-dimensional prediction frame, and the two-dimensional tracking frame is directly determined according to the two-dimensional detection frame.

7. The target distance measurement method based on cylindrical image detection frame according to any one of claims 1 to 6, characterized in that: After determining the three-dimensional position information of the target according to the two-dimensional tracking frame of the target, the method further includes: Performing three-dimensional projection on the two-dimensional tracking frame of each target in the current frame to obtain a three-dimensional detection frame of each target in the current frame; Obtaining a three-dimensional prediction frame of each target in the current frame based on the three-dimensional tracking frame of each target in the previous frame and the target kinematic equation; The three-dimensional prediction frame matched by the three-dimensional detection frame is determined, and the three-dimensional prediction frame matched by the three-dimensional detection frame is updated to obtain a three-dimensional tracking frame of the target in the current frame, so as to implement filtering of the three-dimensional position information.

8. A target distance measurement device based on a cylindrical image detection frame, characterized in that: include: A detection frame generation module is used to perform cylindrical projection dedistortion processing on the original image collected by the fisheye camera to obtain a cylindrical projection image, and obtain a two-dimensional detection frame of each target in the current frame according to the cylindrical projection image; A prediction frame generation module is used to determine the two-dimensional prediction frame of each target in the current frame through the two-dimensional tracking frame of each target in the previous frame and the radial compensation result of each target; the radial compensation result of each target is obtained based on the feature information of the two-dimensional detection frame of each target in the current frame and the vehicle body view control parameter; The updating ranging module is used to determine the two-dimensional prediction frame that matches the two-dimensional detection frame, update the two-dimensional prediction frame that matches the two-dimensional detection frame, obtain the two-dimensional tracking frame of the target in the current frame, and determine the three-dimensional position information of the target according to the two-dimensional tracking frame of the target to achieve ranging of the target.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the target ranging method based on the cylindrical image detection frame as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the target ranging method based on a cylindrical image detection frame as described in any one of claims 1 to 7 is implemented.

Images