A monocular ranging method for a vehicle in motion
By employing the deep learning YOLOv5 algorithm and CBAM attention mechanism in monocular ranging methods to improve target detection accuracy, and combining K-means++ clustering and EIOU methods, and using Kalman filtering to fuse vehicle speed information, the problem of inaccurate target detection in monocular ranging is solved, and higher accuracy target distance estimation is achieved.
Patent Information
- Application Number
- CN202211544247.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-04
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2042-12-04
AI Technical Summary
Existing monocular ranging methods have low target detection accuracy and do not take into account lens distortion and vehicle pitch motion in actual imaging, resulting in inaccurate ranging accuracy.
The YOLOv5 deep learning object detection algorithm is adopted and the CBAM attention mechanism is added. The K-means++ clustering algorithm and EIOU method are combined to improve the object detection accuracy. The target distance is calculated by an improved data regression ranging model. The Kalman filter algorithm is used to fuse vehicle speed information to optimize the ranging results.
It improves the accuracy and generalization ability of monocular ranging, reduces ranging error, and achieves more accurate target distance estimation.
Smart Images
Figure CN115937312B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of automobile auxiliary driving and automatic driving, and particularly relates to a monocular distance measuring method for vehicle driving. BACKGROUND
[0002] At present, advanced driver assistance system (ADAS) has been widely applied in high-end vehicles. The most important part of ADAS is the estimation of the distance of the front target. There are many ways to estimate the distance of the target, and the method of using a monocular camera to estimate the distance is the most cost-effective choice.
[0003] In the prior art, the biggest problem affecting the accuracy of monocular distance measurement is that the target detection accuracy in vehicle-mounted monocular distance measurement is not high, and the floating of the position and size of the target detection frame leads to inaccurate distance measurement. At the same time, the schemes adopted in the prior art are mostly based on the simplification of pinhole imaging model or lens imaging model, without considering the lens distortion and other optical path errors existing in actual imaging, and also limiting the scene to an ideal scene, such as a flat road surface, ignoring the pitching motion of the vehicle, etc. In actual use, these conditions are difficult to completely meet. SUMMARY
[0004] The purpose of the present application is to overcome the shortcomings of the prior art and provide a monocular distance measuring method for vehicle driving. The method uses deep learning technology to obtain the position and size coordinates of the target, establishes a distance measuring model through an improved data regression method, and adds a Kalman filter fusion algorithm to fuse the measurement data with the state model based on the vehicle speed, thereby reducing the distance measurement error in actual distance measurement.
[0005] The present application is realized by the following technical solutions:
[0006] A monocular distance measuring method for vehicle driving, the steps are as follows:
[0007] Step 1: Obtain the image collected by the vehicle-mounted monocular camera;
[0008] Step 2: input the collected image into a pre-constructed and trained target detection model, and output the position information of the target detection frame in the image from the target detection model;
[0009] Step 3: read the position coordinate values of the target detection frame output in the previous step, and substitute them into the pre-established distance measuring model to calculate the distance value of the target;
[0010] Step 4, the distance value of the target obtained in the above step is substituted into the Kalman filtering algorithm, the Kalman filtering algorithm reads the speed information of the current vehicle, and fuses the distance value of the target, and finally outputs the optimized distance value of the target.
[0011] In the above technical solution, in step 1, the vehicle-mounted monocular camera collects pictures at a fixed frequency, preferably at a frequency of 5fps.
[0012] In the above technical solution, YOLOv5 deep learning target detection algorithm is used as the backbone network of the target detection model, and attention mechanism is added to the YOLOv5 backbone network to improve the stability of the target detection algorithm.
[0013] In the above technical solution, the CBAM attention mechanism is integrated into the YOLOv5 backbone network.
[0014] In the above technical solution, the K-means++ clustering algorithm is used in the target detection model to locate the prior box (anchor box) of the target in the image.
[0015] In the above technical solution, the EIOU method is also used in the target detection model and added to the YOLOv5 backbone network.
[0016] In the above technical solution, the ranging model is a function relationship between independent variables and dependent variables constructed by a large amount of real data through data regression method, wherein the independent variable refers to the target pixel position relationship detected by the target detection model, and the dependent variable refers to the distance from the target.
[0017] In the above technical solution, the ranging model uses the pixel diagonal angle length H of the target identified by the target detection model as the independent variable parameter fitting function mapping relationship, and the pixel diagonal angle length H of the target is calculated according to the following formula 1: wherein x1, x2, y 1、 y2 is the diagonal point coordinate of the target detection frame output by the target detection model:
[0018]
[0019] In the above technical solution, in step 4, the vehicle speed data provided by the vehicle odometer is substituted into the prediction model of Kalman filtering, and in the Kalman filtering prediction model, Kalman filtering realizes prior estimation by using formula (2), and obtains the current noise covariance by using formula (3):
[0020]
[0021]
[0022] wherein, is the optimal estimation of the target distance at the current time, is the optimal estimation of the target distance at the previous time, u t-1 is the system control quantity at the previous time (i.e., the vehicle speed), F represents a state transition matrix from the time t-1 to the time t, and B represents a state transition matrix system control matrix from the time t-1 to the time t, is the prior estimation covariance, i.e., the current noise covariance, P t-1 is the variance of the optimal estimation value at the previous time, and Q is a hyperparameter;
[0023] The second application stage of the Kalman filter is updating, and when the observation data of the next stage is obtained, the Kalman gain in the Kalman filter is updated by using formula (4):
[0024]
[0025] wherein K t represents the Kalman gain at the time t, and R represents the measurement error covariance.
[0026] Finally, the distance data of the target in the image calculated by the target detection model and the ranging model are fused with the optimal estimation value of the vehicle at the previous time, the optimal estimation value of the target distance at the time t is obtained according to formula (5), and the covariance matrix is updated according to formula (6).
[0027]
[0028]
[0029] In the formula, z t represents the distance data of the target in the image calculated by the target detection model and the ranging model.
[0030] The present application has the advantages and beneficial effects that:
[0031] The present application is based on a deep learning target detection algorithm as the core to construct a high-precision target detection framework, and accurately identifies the position information of the target detection object in the image. In order to solve the problem that the generalization and precision of the existing ranging method are not strong, the present application adopts an improved data regression ranging method to establish a ranging model, and adds a Kalman filter fusion algorithm to fuse the measurement data with the state model built based on the vehicle speed, thereby reducing the ranging error generated in the ranging practice. BRIEF DESCRIPTION OF DRAWINGS
[0032] Figure 1 is the flowchart of the monocular ranging method for vehicle driving of the present application.
[0033] Figure 2is a structural schematic diagram of the CBAM attention mechanism used in the present application.
[0034] Figure 3 is a schematic diagram of the Kalman filtering algorithm used in the present application for fusing the vehicle speed information and the predicted target distance information.
[0035] For those skilled in the art, other related drawings can be obtained from the above drawings without creative labor. DETAILED DESCRIPTION
[0036] In order for those skilled in the art to better understand the present application, the technical solutions of the present application will be further described below in conjunction with specific embodiments.
[0037] The monocular ranging method for vehicle driving of the present application first constructs a target detection model for identifying the pixel position information of the target detection object in the image. Specifically:
[0038] The present application adopts the YOLOv5 deep learning target detection algorithm as the backbone network of the target detection model. YOLOv5 (You Only Look Once) is a single-stage target detection algorithm published by Uitralytics LLC company. Compared with YOLOv4, YOLOv5 has the characteristics of smaller mean weight file, shorter training time and faster inference speed, while the average precision of detection is not much reduced.
[0039] The attention mechanism is added to the YOLOv5 backbone network to improve the stability of the target detection algorithm. The CBAM attention mechanism is integrated into the YOLOv5 backbone network in this embodiment to improve the detection effect of the target detection algorithm. CBAM attention mechanism is an attention mechanism module that provides attention maps from channel and spatial dimensions. It mainly includes channel attention module (CAM) and spatial attention module (SAM), which can refine the features extracted by the model and effectively improve the classification effect of the model. The CBAM attention mechanism structure is as shown in Figure 2 .
[0040] The K-means++ clustering algorithm is used to position the target prior box (anchor box) in the image. The advantage of using K-means++ clustering algorithm is to randomly generate clustering centers. This method of randomly generating initial clustering centers ensures the discrete type of initial clustering centers and improves the generation effect of target prior box (anchor box).
[0041] The EIOU method is also used, which is added to the YOLOv5 backbone network to improve the regression accuracy. The penalty term of EIOU is based on the penalty term of CIOU to divide the aspect ratio influencing factor to calculate the length and width of the target frame and the anchor frame respectively. Using the EIOU method can make the convergence speed faster and improve the regression accuracy.
[0042] After identifying the position information of the target detection object in the image through the target detection model, the distance of the target needs to be calculated by using the ranging model. The ranging model is a function relationship between the independent variable (the independent variable refers to the position relationship of the target pixel points detected by the target detection) and the dependent variable (the dependent variable refers to the distance D from the target) by constructing a fitting function through a large amount of real data. At present, in the prior art, the pixel width or height of the target in the image is usually used as the independent variable parameter to fit the function mapping relationship, and the pixel diagonal length (H) of the target identified by the target detection model is used as the parameter fitting function mapping relationship in the present application. Selecting the pixel diagonal length of the target as the mapping parameter instead of selecting the pixel width or height of the target can reduce the error of the pixel. When the mapping relationship of the pixel is established, under the condition that the accuracy of the detection frame is unchanged, extracting more pixel sample numbers in the image as parameters will reduce certain error.
[0043] The target pixel diagonal length calculation method is shown in formula 1, wherein x1, x2, y 1、 y2 is the diagonal point coordinate of the target detection frame output by the target detection model:
[0044]
[0045] In the image ranging task, the farther the target is from the object, the greater the distance value expressed by a single pixel, and any slight pixel error will cause a large error in the ranging result. Therefore, in order to further improve the ranging accuracy, the Kalman filtering optimization method is used to optimize the ranging function curve to make it more close to the real distance curve.
[0046] Kalman filtering is a method of secondary estimation of data, which uses the dynamic estimation of the motion state of the system to fuse the predicted value of the system and the observation data of the observer. Kalman filtering is a continuously iterative loop, and since the calculation of the optimal value at the current time only depends on the state of the previous stage and the uncertainty matrix, the real-time and efficiency of the ranging algorithm are ensured.
[0047] In the example of the present application, the distance data of the target in the image calculated by the target detection model and the ranging model is fused with the vehicle speed data provided by the vehicle odometer. The specific steps are as shown in Figure 3 .
[0048] The vehicle speed data provided by the vehicle odometer is substituted into the prediction model of Kalman filter, in which Kalman filter uses formula (2) to realize prior estimation and uses formula (3) to obtain current noise covariance:
[0049]
[0050]
[0051] wherein, represents the optimal estimation of the target distance at the current time, represents the optimal estimation of the target distance at the previous moment, u t-1 represents the system control amount at the previous moment (i.e., the vehicle speed), F represents the state transition matrix from the t-1 moment to the t moment, B represents the state transition matrix system control matrix from the t-1 moment to the t moment, represents the prior estimation covariance, i.e., the current noise covariance, P t-1 represents the optimal estimation value variance at the previous moment, and Q is a hyperparameter.
[0052] The second application stage of Kalman filter is updating, when the observation data of the next stage is obtained, the Kalman gain in Kalman filter is updated by using formula (4):
[0053]
[0054] wherein, K t represents the Kalman gain at the t moment, and R represents the measurement error covariance.
[0055] Finally, the distance data of the target in the image calculated through the target detection model and the ranging model is fused with the optimal estimation value at the previous moment of the vehicle, the optimal estimation value of the target distance at the t moment is obtained according to formula (5), and the covariance matrix is updated according to formula (6).
[0056]
[0057]
[0058] In the formula, z t represents the distance data of the target in the image calculated through the target detection model and the ranging model.
[0059] On the basis of the above scheme, the monocular ranging method for vehicle driving of the present application has the following specific steps:
[0060] Step 1, acquiring the image collected by the vehicle-mounted monocular camera.
[0061] It should be noted that the present application needs to use vehicle speed to fit the ranging, so it is necessary to collect pictures at a fixed frequency, preferably, 5fps frequency is collected, which is the most optimal combination of ranging effect and operation effect.
[0062] Step 2, input the collected image into the target detection model constructed and trained in advance, and output the position information of the target detection frame in the image from the target detection model.
[0063] Step 3, by reading the position coordinate value of the target detection frame output in the last step, it is substituted into the pre-established ranging model, and the distance value of the target is calculated.
[0064] Step 4, the distance value obtained in the last step is substituted into the Kalman filtering algorithm, the Kalman filtering algorithm reads the speed information of the current car, and fuses with the distance value, and finally outputs the optimized distance value of the target.
[0065] In order to verify the accuracy of the ranging method of the present application, static target objects are set on the road, and identification points are set at positions 10m, 20m, 30m, 40m, 50m, 60m, 70m, 80m, 90m, 100m away from the target object at 5m intervals to confirm the distance of the vehicle from the target. The vehicle maintains a speed of 20km / s to travel towards the target. Then, according to the video provided by the car recorder mounted on the real vehicle, five frames per second are selected as the sampling interval to verify the present application. The following table is the ranging result under the real vehicle experiment.
[0066] Serial number Actual distance / m Measured distance / m Absolute error / m Relative error / % 1 10 10.73 0.73 7.3 2 20 21.02 1.02 5.1 3 30 29.82 0.18 0.6 4 40 42.19 2.19 5.47 5 50 52.32 2.32 4.64 6 60 63.65 3.65 6.08 7 70 72.46 2.46 3.51 8 80 81.98 1.98 2.47 9 90 90.36 0.36 0.4 10 100 101.2 1.2 1.2 Average \ \ 1.61 3.67
[0067] The above is an exemplary description of the present application, it should be noted that without departing from the core of the present application, any simple modification, modification or other equivalent replacement which can not cost creative labor of those skilled in the art falls within the protection scope of the present application.
Claims
1. A monocular ranging method for use in vehicle motion, characterized in that, Includes the following steps: Step 1: Acquire images captured by the vehicle-mounted monocular camera; Step 2: Input the acquired image into the pre-built and trained object detection model, and the object detection model outputs the position information of the object detection box in the image; Step 3: By reading the position coordinates of the target detection box output in the previous step, and substituting them into the pre-established ranging model, the distance value of the target is calculated. Step 4: Substitute the target distance value obtained in the previous step into the Kalman filter algorithm. The Kalman filter algorithm will read the current speed information of the car and fuse it with the target distance value to finally output the optimized target distance value. The ranging model is constructed using a large amount of real data to reflect the functional relationship between independent and dependent variables based on data regression. The independent variable refers to the positional relationship of the target pixels detected by the target detection model, and the dependent variable refers to the distance from the target. The ranging model uses the pixel oblique angle length H of the target identified by the target detection model as the independent variable parameter to fit the mapping relationship of the function. The calculation method of the pixel oblique angle length of the target is shown in Equation 1 below, where The coordinates of the diagonal points of the bounding box output by the object detection model: 。 2. The monocular ranging method for vehicles in motion according to claim 1, characterized in that: In step 1, the vehicle-mounted monocular camera acquires images at a fixed frequency.
3. The monocular ranging method for vehicles in motion according to claim 2, characterized in that: The vehicle-mounted monocular camera captures images at a frequency of 5fps.
4. The monocular ranging method for vehicles in motion according to claim 1, characterized in that: The YOLOv5 deep learning object detection algorithm is used as the backbone network of the object detection model. An attention mechanism is added to the YOLOv5 backbone network to improve the stability of the object detection algorithm.
5. The monocular ranging method for vehicles in motion according to claim 4, characterized in that: The CBAM attention mechanism is integrated into the YOLOv5 backbone network.
6. The monocular ranging method for vehicles in motion according to claim 4, characterized in that: The object detection model uses the K-means++ clustering algorithm to locate the prior bounding boxes of objects in the image.
7. The monocular ranging method for vehicles in motion according to claim 4, characterized in that: The object detection model also employs the EIOU method, which is added to the YOLOv5 backbone network.
8. The monocular ranging method for vehicles in motion according to claim 1, characterized in that: In step 4, the vehicle speed data provided by the vehicle odometer is substituted into the Kalman filter prediction model. In the Kalman filter prediction model, the Kalman filter uses equation (2) to achieve prior estimation and uses equation (3) to obtain the current noise covariance: in, This represents the optimal estimate of the target distance at the current moment. This represents the optimal estimate of the target distance at the previous moment. F represents the system control quantity at the previous moment, and F represents the quantity from... Time's up The state transition matrix at time t, Indicates from Time's up The state transition matrix at any given time is the system control matrix. This represents the prior estimated covariance, i.e., the current noise covariance. Let Q represent the variance of the best estimate at the previous time step, and Q be a hyperparameter. The second application stage of Kalman filtering is the update. When the observation data for the next stage is obtained, the Kalman gain in Kalman filtering is updated using equation (4): in, This represents the Kalman gain at time t. This represents the measurement error covariance; Finally, the distance data of the target in the image calculated by the target detection model and the ranging model are fused with the optimal estimate of the vehicle at the previous moment. The optimal estimate of the target distance at time t is obtained according to Equation (5), and the covariance matrix is updated according to Equation (6). In the formula, This represents the distance data of the target in the image calculated by the target detection model and the ranging model.