Vehicle detection and collision early warning method and device, system and storage medium

Abstract

The invention discloses a vehicle detection and collision early warning method, device and system and a storage medium, and the method comprises the steps: obtaining a YOLOv11 vehicle detection model according to a KITTI data set; the YOLOv11 vehicle detection model is deployed in an embedded device; according to a YOLOv11 vehicle detection model, the actual distance between the front vehicle and the camera is calculated through the monocular vision principle; and comparing the actual distance with the safe distance, and making a decision whether to give an early warning or not. By adopting the technical scheme of the invention, efficient vehicle detection and distance early warning can be realized under the condition of low cost.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned driving technology, and in particular relates to a vehicle detection and collision warning method and device, system, and storage medium. Background Art

[0002] In the field of intelligent transportation systems and autonomous driving, vehicle detection and distance estimation are important links to improve driving safety. In early autonomous driving and ADAS systems, vehicle detection and distance measurement mainly rely on multi-sensor fusion technology, such as LiDAR, millimeter-wave radar, and binocular cameras. These sensors transmit and receive reflected signals to achieve accurate detection and distance calculation of target objects. For example, LiDAR can calculate the distance of the target object by emitting a laser beam and detecting its reflection time; millimeter-wave radar detects the speed and distance of the object by emitting electromagnetic waves. These multi-sensor fusion systems are highly accurate and robust, and can cope with various complex traffic environments, such as different lighting conditions and complex weather environments.

[0003] However, despite the high reliability of these sensor systems, they are expensive, complex, and large in size, which prevents them from being widely used in ordinary passenger cars. In order to reduce system costs and simplify system structure, vehicle detection and distance estimation technology based on monocular vision has gradually become a research hotspot. Monocular vision systems only need to use one camera, combined with image processing algorithms and geometric principles to complete vehicle detection and distance calculation. The monocular camera captures a two-dimensional image, combines the internal and external parameters of the camera and the known size of the object, and uses trigonometric geometry to deduce the distance from the object to the camera.

[0004] Existing vehicle detection systems usually rely on multi-sensor fusion solutions. Although these solutions are highly accurate, they are not suitable for large-scale commercial applications due to their high cost and complexity. On the other hand, although visual detection technology is low-cost and easy to deploy, it still has shortcomings in actual distance estimation and detection accuracy. Although the current vehicle detection system has certain real-time performance and detection accuracy, it has insufficient detection performance when facing long-distance or small-sized vehicle targets. Especially in the case of high-speed driving or long distances, traditional vehicle detection and collision warning systems have difficulty in accurately identifying small targets, resulting in missed detection or detection delays. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a vehicle detection and collision warning method and device, system, and storage medium, which can achieve efficient vehicle detection and distance warning under low-cost conditions.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A vehicle detection and collision warning method, comprising:

[0008] According to the KITTI dataset, the YOLOv11 vehicle detection model is obtained;

[0009] According to the YOLOv11 vehicle detection model, the actual distance between the front vehicle and the camera is calculated through the monocular vision principle;

[0010] Compare the actual distance with the safe distance and decide whether to issue an early warning.

[0011] Preferably, the image data transmitted back by the camera is recognized according to the YOLOv11 vehicle detection model, the recognized vehicle is marked in the image, and the camera is calibrated using the Zhang Zhengyou calibration method to obtain the internal and external parameters of the camera; the midpoint of the bottom edge of the vehicle annotation box is selected as the feature point, the pixel coordinates of the point are converted into world coordinates, and the distance between the camera and the vehicle in front is calculated based on the camera imaging principle.

[0012] As a preference, the safety distance D s The calculation formula is as follows:

[0013] D s =D1+d0+V1(T w +T r )

[0014] Among them, D1 is the distance of the vehicle ahead, the safe parking distance d0, V1 is the speed of the vehicle, T w is the hydraulic brake response time, T r The driver's reaction time.

[0015] The present invention also provides a vehicle detection and collision warning device, comprising:

[0016] The training module is used to obtain the YOLOv11 vehicle detection model based on the KITTI dataset;

[0017] The detection module is used to calculate the actual distance between the front vehicle and the camera through the monocular vision principle based on the YOLOv11 vehicle detection model;

[0018] The early warning module is used to compare the actual distance with the safe distance and make a decision on whether to issue an early warning.

[0019] Preferably, the detection module uses the image data returned by the camera according to the YOLOv11 vehicle detection model to mark the identified vehicle in the image, calibrates the camera using the Zhang Zhengyou calibration method, and then obtains the internal and external parameters of the camera; selects the midpoint of the bottom edge of the vehicle annotation box as the feature point, converts the pixel coordinates of the point into world coordinates, and calculates the distance between the camera and the vehicle in front based on the camera imaging principle.

[0020] As a preference, the safety distance D s The calculation formula is as follows:

[0021] D s =D1+d0+V1(T w +T r )

[0022] Among them, D1 is the distance of the vehicle ahead, the safe parking distance d0, V1 is the speed of the vehicle, T w is the hydraulic brake response time, T r The driver's reaction time.

[0023] An embodiment of the present invention further provides a vehicle detection and collision warning system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a vehicle detection and collision warning method when executed by the processor.

[0024] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the vehicle detection and collision warning method when running.

[0025] The present invention designs a vehicle detection model based on the YOLOv11 algorithm, and its detection accuracy and real-time performance are better than other detection accuracy. According to the monocular vision measurement of the distance to the vehicle in front, when the measured distance is less than the safe distance, an early warning and reminder are given. Under the premise of ensuring high accuracy, real-time vehicle detection and distance estimation are achieved at low cost, and effective deployment is achieved on embedded devices, ultimately providing an efficient and reliable solution for autonomous driving and driver assistance systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0027] Figure 1 This is a flow chart of a vehicle detection and collision warning method according to an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of the relationship between the camera coordinate system, image coordinate system and world coordinate system;

[0029] Figure 3 Schematic diagram of the relationship between the pixel coordinate system and the image coordinate system;

[0030] Figure 4 This is the relationship diagram between the camera coordinate system and the world coordinate system;

[0031] Figure 5 It is the camera imaging model. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Embodiment 1:

[0035] like Figure 1 As shown, an embodiment of the present invention provides a vehicle detection and collision warning method, comprising:

[0036] Step S1: Get the YOLOv11 vehicle detection model based on the KITTI dataset

[0037] S101: Process the KITTI dataset, remove the DontCare area in the dataset label, use Python programming to convert the dataset format to the dataset annotation format used by the YOLOv11 algorithm, and divide the dataset into training set, test set, and validation set in a ratio of 8:1:1;

[0038] S102: Based on the processed data set, select YOLOv11m.pt as the weight file to train the vehicle detection model;

[0039] S103: Evaluate the trained vehicle detection model. The evaluation indicators include precision, recall, mean Average Precision (mAP), and the number of frames processed per second (FPS).

[0040] S2. Deploy the YOLOv11 vehicle detection model on embedded devices

[0041] S201: The present invention adopts an embedded device as Nvidia Jetson Nano, and installs an Ultralytics software package on the device;

[0042] S202: Install PyTorch and related dependencies to ensure compatibility with YOLOv11 version;

[0043] S203: Convert the YOLOv11 vehicle detection model to the TensorRT format recognizable by PyTorch and optimize the model on Nvidia Jetson Nano to reduce memory usage and improve performance.

[0044] S3. Based on the YOLOv11 vehicle detection model, the actual distance between the front vehicle and the camera is calculated using the monocular vision principle.

[0045] S301: Use the YOLOv11 vehicle detection model deployed on Nvidia Jetson Nano to identify image data sent back by the camera and mark the identified vehicles in the image;

[0046] S302: Calibrate the camera using Zhang Zhengyou calibration method to obtain internal and external parameters of the camera;

[0047] Specifically, a chessboard image taken by a camera is placed in the CameraCalibration toolbox in MATLAB software, and after corner detection, the camera internal and external parameters are calculated.

[0048] S303: Select the midpoint of the bottom edge of the vehicle annotation box as a feature point, and convert the pixel coordinates of the point into world coordinates;

[0049] Specifically, the relationship between the camera coordinate system, image coordinate system and world coordinate system is as follows: Figure 2 As shown, the coordinate transformation relationship is as follows:

[0050] Pixel coordinate system: The origin of the coordinate system is the upper left corner of the image O0, and the uv coordinate system is established in pixels (pixel), using (u, v) T Represents the coordinate value of a point in the pixel coordinate system.

[0051] Image coordinate system: The origin of the coordinate is the image center O1, and the coordinates in the pixel coordinate system are (u0, v0). The XY coordinate system is established in units of mm, and the coordinates of a point in the image are represented by (x, y).

[0052] The relationship between a point in the image in the uv coordinate system and the XY coordinate system is as follows Figure 3 As shown, the conversion relationship is:

[0053]

[0054] In formula (1), dx and dy represent the physical conversion ratio of each pixel in the X and Y directions respectively. The coordinates of a point in the pixel coordinate system in the world coordinate system are (u, v, 1). The conversion relationship between the pixel coordinate system and the image coordinate system is shown in formula (2). The conversion relationship between a point in the image in the uv coordinate system and the XY coordinate system is:

[0055]

[0056] Camera coordinate system: The camera optical center O is the origin of the coordinate system, X C , Y C , Z C The coordinates of a point in the camera are expressed as (X C ,Y C ,Z C ). The conversion relationship between the camera coordinate system and the image coordinate system is:

[0057]

[0058] Where f is the focal length of the camera, in mm. The above relationship is expressed as formula (4) using homogeneous coordinates and matrices.

[0059]

[0060] World coordinate system: An absolute coordinate system that describes the target position. Generally, the world coordinate system is used as a reference system to determine the objective position of the camera and other objects. (X W ,Y W ,Z W ) T Represents the coordinates of a point in space.

[0061] The relationship between the camera coordinate system and the world coordinate system is as follows Figure 4 As shown, the spatial point Q is a point in the world coordinate system, q is the corresponding point of Q in the camera coordinate system, and the homogeneous coordinates of Q in the world coordinate system (X W ,Y W ,Z W ,1) T and the homogeneous coordinates (X C ,Y C ,Z C ,1) T The relationship between them is:

[0062]

[0063] Combining formulas (2), (4), and (5), we can get the relationship between pixel coordinates and world coordinates:

[0064]

[0065] make

[0066]

[0067] Formula (6) can be simplified to

[0068]

[0069] Among them, M1 is called the intrinsic parameter matrix, which is only related to the internal parameters of the camera, and M2 is determined by the position and posture of the camera relative to the vehicle and is called the external parameters of the camera.

[0070] S304: According to the pinhole imaging principle, Figure 5 As shown, the actual distance between the camera and the front vehicle is calculated by combining the image coordinate transformation relationship within the camera.

[0071] Specifically, the focal length of the camera is f, the pixel size is μ, the camera is installed on the vehicle at a height h relative to the ground, and the pitch angle of the camera is (x0, y0) is the origin of the image coordinate system, that is, the intersection of the image plane and the optical axis of the camera. The intersection of the front vehicle and the road plane has coordinates (x, y) in the image plane coordinate system. The horizontal distance d between the intersection of the front vehicle and the road plane and the camera is:

[0072]

[0073] S4. Compare the actual distance with the safe distance and decide whether to issue a warning.

[0074] Specifically, when the distance d to the vehicle ahead is less than the safety distance D S When the distance d to the vehicle ahead is greater than the safe distance D S When the distance is too close, no reminder will be given.

[0075] Among them, the safety distance D s The calculation formula is as follows:

[0076] D s =D1+d0+V1(T w +T r ) (11)

[0077] Among them, D1 is the distance of the vehicle ahead, the safe parking distance d0, V1 is the speed of the vehicle, T w is the hydraulic brake response time, T r The driver's reaction time.

[0078] In the auxiliary system of an autonomous vehicle, the embodiment of the present invention is used for real-time detection and distance measurement of the vehicle in front. By deploying the lightweight YOLOv11 model on NVIDIA Jetson Nano, it is possible to accurately detect the vehicle in front in a variety of complex scenarios and calculate the distance between the vehicles. When it is detected that the distance of the vehicle in front is too close, the system reminds the driver through a voice alarm to prevent a collision accident.

[0079] The embodiments of the present invention have the following technical effects:

[0080] 1. Efficient detection of YOLOv11: Compared with the previous generation YOLO model, YOLOv11 significantly improves the accuracy of small and long-distance target detection, especially in low-light conditions.

[0081] 2. High-precision distance estimation based on monocular vision: By utilizing the geometric principles of monocular vision, through precise internal parameter calibration and error correction, the system can achieve high-precision distance estimation without the need for multiple sensors.

[0082] 3. Embedded real-time warning: This system runs on an embedded development board, has the characteristics of lightweight and low power consumption, can achieve real-time warning, and is suitable for scenarios such as autonomous driving, driving assistance, and smart parking.

[0083] Embodiment 2:

[0084] The embodiment of the present invention further provides a vehicle detection and collision warning device, comprising:

[0085] The training module is used to obtain the YOLOv11 vehicle detection model based on the KITTI dataset;

[0086] The detection module is used to calculate the actual distance between the front vehicle and the camera through the monocular vision principle based on the YOLOv11 vehicle detection model;

[0087] The early warning module is used to compare the actual distance with the safe distance and make a decision on whether to issue an early warning.

[0088] As an implementation method of an embodiment of the present invention, the detection module uses the image data returned by the camera according to the YOLOv11 vehicle detection model to mark the identified vehicle in the image, calibrates the camera using the Zhang Zhengyou calibration method, and then obtains the internal and external parameters of the camera; selects the midpoint of the bottom edge of the vehicle annotation box as the feature point, converts the pixel coordinates of the point into world coordinates, and calculates the distance between the camera and the vehicle in front based on the camera imaging principle.

[0089] As an implementation method of the present invention, the safety distance D s The calculation formula is as follows:

[0090] D s =D1+d0+V1(T w +T r )

[0091] Among them, D1 is the distance of the vehicle ahead, the safe parking distance d0, V1 is the speed of the vehicle, T w is the hydraulic brake response time, T r The driver's reaction time.

[0092] Embodiment 3:

[0093] An embodiment of the present invention further provides a vehicle detection and collision warning system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a vehicle detection and collision warning method when executed by the processor.

[0094] Embodiment 4:

[0095] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the vehicle detection and collision warning method when running.

[0096] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A vehicle detection and collision warning method, characterized in that: include: According to the KITTI dataset, the YOLOv11 vehicle detection model is obtained; According to the YOLOv11 vehicle detection model, the actual distance between the front vehicle and the camera is calculated through the monocular vision principle; Compare the actual distance with the safe distance and decide whether to issue an early warning.

2. The vehicle detection and collision warning method according to claim 1, characterized in that: The image data sent back by the camera is recognized using the YOLOv11 vehicle detection model, the recognized vehicle is marked in the image, and the camera is calibrated using the Zhang Zhengyou calibration method to obtain the camera's internal and external parameters. The midpoint of the bottom edge of the vehicle annotation box is selected as the feature point, the pixel coordinates of the point are converted into world coordinates, and the distance between the camera and the front vehicle is calculated based on the camera imaging principle.

3. The vehicle detection and collision warning method according to claim 2, characterized in that: Safety distance D s The calculation formula is as follows: D s =D1+d0+V1(T w +T r ) Among them, D1 is the distance of the vehicle ahead, the safe parking distance d0, V1 is the speed of the vehicle, T w is the hydraulic brake response time, T r The driver's reaction time.

4. A vehicle detection and collision warning device, characterized in that: include: The training module is used to obtain the YOLOv11 vehicle detection model based on the KITTI dataset; The detection module is used to calculate the actual distance between the front vehicle and the camera through the monocular vision principle based on the YOLOv11 vehicle detection model; The early warning module is used to compare the actual distance with the safe distance and make a decision on whether to issue an early warning.

5. The vehicle detection and collision warning device according to claim 4, characterized in that: The detection module uses the image data sent back by the camera according to the YOLOv11 vehicle detection model to mark the identified vehicle in the image, calibrates the camera using the Zhang Zhengyou calibration method, and then obtains the internal and external parameters of the camera; The midpoint of the bottom edge of the vehicle annotation box is selected as the feature point, the pixel coordinates of the point are converted into world coordinates, and the distance between the camera and the front vehicle is calculated based on the camera imaging principle.

6. The vehicle detection and collision warning device according to claim 5, characterized in that: Safety distance D s The calculation formula is as follows: D s =D1+d0+V1(T w +T r ) Among them, D1 is the distance of the vehicle ahead, the safe parking distance d0, V1 is the speed of the vehicle, T w is the hydraulic brake response time, T r The driver's reaction time.

7. A vehicle detection and collision warning system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the vehicle detection and collision warning method according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that: The storage medium stores a computer program, which executes the vehicle detection and collision warning method according to any one of claims 1 to 3 when running.

Images