Vehicle speed measurement system based on radar and machine vision

Abstract

The invention relates to the field of vehicle speed measurement, in particular to a vehicle speed measurement system based on radar and machine vision, which comprises a radar module, a machine vision module, a data fusion module, a speed measurement result output and display module and a system calibration and self-diagnosis module. According to the invention, high-precision speed measurement can be realized under various complex weather conditions, road conditions and vehicle driving states through advantage complementation of radar and machine vision in combination with an advanced hybrid filtering algorithm; the system has strong adaptability to different environment conditions and road scenes; the optimal design of data acquisition, processing, fusion and result output ensures that the system has high real-time performance and can quickly and accurately reflect the current running speed of the vehicle; the speed measurement result can be locally checked in real time, and a remote transmission function is also provided, so that a remote monitoring center can carry out macroscopic monitoring on regional traffic flow, vehicle speed distribution and the like conveniently.

Description

Technical Field

[0001] The present invention relates to the field of vehicle speed measurement, and in particular to a vehicle speed measurement system based on radar and machine vision. Background Art

[0002] Accurately measuring vehicle speed is crucial in many fields such as transportation management, intelligent driving assistance, and road safety monitoring. Traditional vehicle speed measurement methods have many limitations and cannot meet the growing demand for high-precision and high-reliability speed measurement.

[0003] The speed measurement method based on ground sensing coils requires burying coils under the road surface. The construction is complex and costly, and the coils are easily affected by factors such as road damage and water accumulation. Once damaged, they are difficult to repair. At the same time, the speed measurement accuracy will decrease with the use time, and it has poor adaptability to multi-lane and complex road conditions.

[0004] Although laser speed meters have high speed measurement accuracy, they are significantly affected by weather conditions. In bad weather such as rain, fog, and dust, the propagation of the laser beam is blocked, resulting in increased measurement errors or even failure to measure speed normally. Moreover, laser speed meters can usually only perform point measurements on a single vehicle, making it difficult to achieve real-time monitoring of large-area traffic on multiple lanes.

[0005] The video speed measurement method relies on the camera to collect vehicle images and calculates the speed by analyzing the displacement of the vehicle in the image sequence. However, this method is extremely sensitive to lighting conditions. Too strong or too dark light, driving against the light, etc. will seriously affect the image quality, thereby reducing the speed measurement accuracy. In addition, when the vehicle is blocked or deformed, the video speed measurement algorithm is prone to misjudgment and poor stability.

[0006] With the rapid development of sensor technology and image processing technology, multi-sensor fusion speed measurement solutions combining radar and machine vision have gradually become a research hotspot. However, most of the existing fusion speed measurement systems have problems such as simple data fusion algorithms, poor sensor collaboration, and insufficient system real-time performance, which cannot give full play to the respective advantages of radar and machine vision.

[0007] Therefore, those skilled in the art provide a vehicle speed measurement system based on radar and machine vision to solve the problems raised in the above background technology. Summary of the invention

[0008] In order to solve the above technical problems, the present invention provides a vehicle speed measurement system based on radar and machine vision.

[0009] Including radar module: high-performance millimeter-wave radar is used to transmit millimeter-wave signals and receive echo signals. It uses the Doppler effect and distance measurement principle to preliminarily calculate the radial speed and relative distance of the vehicle. It has a built-in adaptive signal processing algorithm and is equipped with intelligent target recognition and tracking functions;

[0010] Machine vision module: uses a high-definition camera to collect vehicle image sequences, calculates the vehicle's two-dimensional plane speed through preprocessing and deep learning-based target detection algorithms, and introduces motion compensation algorithms;

[0011] Data fusion module: A hybrid filtering algorithm based on extended Kalman filtering and particle filtering is used. The radar measurement data is used as the main observation value and the machine vision measurement data is used as the auxiliary observation value. The vehicle state model is established and the covariance matrix is ​​used to weigh the two sensor data.

[0012] Speed ​​measurement result output and display module: receives the fused speed measurement results, displays them on the local monitoring terminal after visualization, and transmits them remotely to the monitoring center;

[0013] System calibration and self-diagnosis module: calibrate radar and machine vision sensors, monitor system operation status in real time, issue alarms when abnormalities occur, and take fault-tolerant measures.

[0014] Preferably, the adaptive signal processing algorithm of the radar module automatically adjusts the radar's transmission power, signal gain and filtering parameters according to the environmental noise and interference signal strength monitored in real time.

[0015] Preferably, the image processing unit of the machine vision module uses an adaptive histogram equalization algorithm to automatically adjust the pixel grayscale value according to the brightness distribution of different areas of the image, enhance the overall contrast of the image, and use Gaussian filtering to remove noise in the image.

[0016] Preferably, the deep learning model of the machine vision module is pre-trained on a large number of image data sets containing various vehicle types, postures and complex road conditions.

[0017] Preferably, during the fusion process, the data fusion module uses a covariance matrix to accurately model the measurement noise according to the measurement noise characteristics of the radar and machine vision sensors, and adjusts the parameters of the covariance matrix in real time to ensure that the fusion algorithm can reasonably weigh the data of the two sensors, so that the final speed measurement result is more accurate and reliable.

[0018] Preferably, the speed measurement result output and display interface of the display module adopts an intuitive and clear chart format, while marking the vehicle's license plate number, lane information, and timestamp key data.

[0019] Preferably, during the system initialization phase and regular operation, the system calibration and self-diagnosis module performs calibration operations on the radar and machine vision sensors, uses standard targets to calibrate the speed measurement accuracy and distance measurement accuracy of the radar and the image acquisition parameters and target detection accuracy of the machine vision, and adjusts the operating parameters of the sensors to achieve optimal performance.

[0020] Preferably, the system calibration and self-diagnosis module monitors the operating status of each component of the system in real time, including sensor signal strength, data transmission rate, processor load, etc. Once an abnormal situation is found, such as sensor failure, data transmission interruption, algorithm operation timeout, etc., an alarm signal is immediately issued, and corresponding fault-tolerant measures are taken, such as switching backup sensors, restarting related modules, etc., to ensure uninterrupted operation of the system.

[0021] Technical effects and advantages of the present invention:

[0022] High-precision speed measurement: Through the complementary advantages of radar and machine vision, combined with advanced hybrid filtering algorithms, high-precision speed measurement can be achieved in various complex weather conditions, road conditions and vehicle driving conditions. Whether it is sunny or rainy, foggy, or dusty, whether it is driving in a straight line or changing speed on a curve, the vehicle speed can be accurately measured, providing a reliable basis for traffic management, intelligent driving, etc.

[0023] Strong adaptability: The system is highly adaptable to different environmental conditions and road scenes. The high penetration of radar enables it to work stably in bad weather, and the image analysis capability of machine vision enables it to cope with complex traffic scenes, such as multiple vehicles in parallel and vehicle occlusion. The two work together to greatly broaden the application scope of the system.

[0024] Real-time and reliability: The optimized design of data acquisition, processing, fusion and result output ensures that the system has high real-time performance and can quickly and accurately reflect the current speed of the vehicle. At the same time, the setting of system calibration and self-diagnosis module ensures the long-term stable operation of the system, reduces the speed measurement error caused by sensor failure, algorithm abnormality, etc., and improves the reliability of speed measurement results.

[0025] Macro-monitoring and decision support: Speed ​​measurement results can not only be viewed locally in real time, but also have remote transmission functions, which facilitates the remote monitoring center to conduct macro-monitoring of regional traffic flow, vehicle speed distribution, etc., provide strong data support for traffic planning, congestion relief and other decisions, and promote the intelligent development of traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the system structure of a vehicle speed measurement system based on radar and machine vision provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0028] Example 1

[0029] See also Figure 1 In this embodiment, a vehicle speed measurement system based on radar and machine vision is provided, including a radar module

[0030] The high-performance millimeter-wave radar is selected, which has the characteristics of high resolution and strong penetration, and can work stably under various weather conditions. The millimeter-wave radar transmits millimeter-wave signals at a certain frequency in the direction of the vehicle's travel, and receives the echo signal reflected by the vehicle. By analyzing the frequency change, time delay and other parameters of the echo, using the Doppler effect and distance measurement principle, the radial speed and relative distance of the vehicle are preliminarily calculated.

[0031] In order to improve the accuracy and reliability of radar speed measurement, the radar module has a built-in adaptive signal processing algorithm. The algorithm automatically adjusts the radar's transmit power, signal gain, and filtering parameters based on the real-time monitored environmental noise and interference signal strength. For example, in a noisy urban road environment, the transmit power is appropriately increased to enhance the echo signal strength, while the filtering process is strengthened to remove clutter interference and ensure accurate acquisition of vehicle reflection signals.

[0032] In addition, the radar module is also equipped with intelligent target recognition and tracking functions. Using advanced clustering algorithms and target feature extraction technology, the vehicle target can be accurately identified from a large number of echo signals and continuously tracked. Even in complex traffic scenarios such as multiple vehicles running in parallel and vehicles changing lanes frequently, the target vehicle can be stably locked, providing a guarantee for subsequent accurate speed measurement.

[0033] Machine Vision Module

[0034] A high-definition camera is used as a visual sensor, and the camera is installed in a suitable position to ensure that the vehicle driving picture can be captured clearly. The camera collects the vehicle image sequence at a high frame rate and transmits it to the image processing unit.

[0035] The image processing unit first performs preprocessing operations on the collected images. To address the problem of uneven illumination, the adaptive histogram equalization algorithm is used to automatically adjust the pixel grayscale value according to the brightness distribution of different areas of the image, enhance the overall image contrast, and make the vehicle outline clearer and more recognizable. At the same time, methods such as Gaussian filtering are used to remove noise in the image and improve image quality.

[0036] Next, the preprocessed images are analyzed using a deep learning-based object detection algorithm. The deep learning model is pre-trained on a large number of image datasets containing various vehicle types, postures, and complex road conditions. It can quickly and accurately identify vehicles in images and accurately locate the position and outline of vehicles. By analyzing the changes in the vehicle position in consecutive image frames and combining the mapping relationship between the image pixel coordinates and the actual physical space, the two-dimensional plane speed of the vehicle is calculated.

[0037] In order to further improve the accuracy of machine vision speed measurement, the machine vision module introduces a motion compensation algorithm. When the camera itself is shaking or is installed on a mobile platform (such as a vehicle-mounted camera), the camera's motion information is obtained through the optical flow method or inertial measurement unit (IMU), and the vehicle image is compensated accordingly to eliminate the speed measurement error caused by camera movement.

[0038] Data fusion module

[0039] As the core hub of the system, the data fusion module is responsible for deeply fusing the data collected by the radar module and the machine vision module. A hybrid filtering algorithm based on the combination of extended Kalman filter (EKF) and particle filter (PF) is adopted to give full play to the advantages of both.

[0040] First, the vehicle state model is established with the radial velocity and relative distance measured by radar as the main observation values ​​and the two-dimensional plane velocity measured by machine vision as the auxiliary observation value. The model contains state variables such as the vehicle's position, velocity, acceleration, and environmental parameters such as road curvature and slope.

[0041] The EKF algorithm is used to make a preliminary estimate of the vehicle state, and its linear processing capability is used to quickly converge to a better solution. On this basis, the PF algorithm is introduced to make a fine estimate of state variables with strong nonlinearity, such as the speed change of the vehicle when driving on a curve, the acceleration mutation during acceleration, etc. The PF algorithm can better adapt to complex vehicle motion models and improve speed measurement accuracy through random sampling and weight update of a large number of particles.

[0042] In the fusion process, the covariance matrix is ​​used to accurately model the measurement noise according to the measurement noise characteristics of radar and machine vision sensors. Through statistical analysis of real-time collected data, the parameters of the covariance matrix are dynamically adjusted to ensure that the fusion algorithm can reasonably weigh the data of the two sensors, making the final speed measurement result more accurate and reliable.

[0043] Speed ​​measurement result output and display module

[0044] This module receives the final speed measurement results output by the data fusion module and visualizes them. On the one hand, the speed measurement results can be displayed in real time on the local monitoring terminal for direct viewing by traffic management personnel and drivers (such as vehicle-mounted assisted driving systems). The display interface uses an intuitive and clear chart format, while marking key data such as the vehicle's license plate number (if recognizable), lane information, and timestamp, so that users can quickly understand the vehicle's driving status.

[0045] On the other hand, the speed measurement results also have the function of remote transmission, and can send data to the remote monitoring center through wired or wireless communication. The remote monitoring center can summarize and analyze the data of multiple speed measurement points, realize macro-monitoring of regional traffic flow, vehicle speed distribution, etc., and provide a strong basis for decision-making such as traffic planning and congestion relief.

[0046] System calibration and self-diagnosis module

[0047] In order to ensure the long-term stable operation of the system, the present invention also designs a system calibration and self-diagnosis module. During the system initialization phase and regular operation, the module performs calibration operations on the radar and machine vision sensors. Using standard targets (such as simulated vehicle models with known speed and distance), the radar's speed measurement accuracy, distance measurement accuracy, and the machine vision image acquisition parameters and target detection accuracy are calibrated respectively, and the operating parameters of the sensors are adjusted to achieve the best performance state.

[0048] At the same time, the self-diagnosis module monitors the operating status of each component of the system in real time, including sensor signal strength, data transmission rate, processor load, etc. Once an abnormal situation is found, such as sensor failure, data transmission interruption, algorithm operation timeout, etc., an alarm signal is immediately issued, and corresponding fault-tolerant measures are taken, such as switching backup sensors, restarting related modules, etc., to ensure the uninterrupted operation of the system. When using the present invention, the staff first installs the high-definition camera on the crossbar above the road or a suitable fixed position during the system initialization phase to ensure that its field of view covers the target lane. The millimeter-wave radar is installed near the camera, and the angles of the two are adjusted to point in the same direction. Start the system calibration and self-diagnosis module, place the standard simulation vehicle model within the detection range of the radar and the camera, obtain the radar echo data and the camera image data respectively according to the calibration procedure, and calibrate the sensor parameters through a specific algorithm to ensure that the measurement accuracy of the two meets the requirements; at the same time, check the connection and communication status of each component of the system to ensure normal data transmission.

[0049] When the radar module is working, the millimeter-wave radar transmits millimeter-wave signals in the direction of the vehicle at a frequency of 20 times per second (adjustable according to actual needs); in a normal measurement process, if it rains, the rain will cause certain scattering and attenuation of the millimeter-wave signals. At this time, the adaptive signal processing algorithm appropriately increases the radar's transmission power and strengthens the filtering process according to the monitored environmental noise and interference signal strength, accurately extracts the vehicle's reflected signal from the complex echo signal; through the Doppler effect and distance measurement principle, the radial speed and relative distance of the vehicle are preliminarily calculated, and these data are transmitted to the data fusion module.

[0050] The machine vision module runs synchronously, and the high-definition camera collects vehicle image sequences at a high frame rate of 30 frames per second (adjustable according to actual needs). When the collected images have uneven lighting, such as in the evening when some lanes are in shadow areas, the adaptive histogram equalization algorithm automatically adjusts the pixel grayscale value according to the brightness distribution of different areas of the image to enhance the overall contrast of the image. After the Gaussian filter removes the noise in the image, the target detection algorithm based on deep learning analyzes the image to quickly identify the vehicle and accurately locate its position and outline. Combined with the changes in the vehicle position in continuous image frames and the mapping relationship between the pixel coordinates of the image and the actual physical space, the two-dimensional plane speed of the vehicle is calculated and the data is also transmitted to the data fusion module.

[0051] After receiving the data from the radar and machine vision modules, the data fusion module uses the radial velocity and relative distance measured by the radar as the main observation values ​​and the two-dimensional plane velocity measured by the machine vision as the auxiliary observation value to establish the vehicle state model; firstly, the EKF algorithm is used to make a preliminary estimate of the vehicle state and quickly converge to a better solution. Since the speed change of the vehicle when driving on a curve is highly nonlinear, the PF algorithm is introduced on this basis to make a fine estimate of the nonlinear state variables; in the fusion process, according to the statistical analysis of the measurement noise of the two sensors in the early stage, the initial parameters of the covariance matrix are set, and as the vehicle drives, new data is continuously collected, and the covariance matrix is ​​adjusted in real time to optimize the fusion effect.

[0052] The speed measurement result output and display module receives the final speed measurement result output by the data fusion module. On the one hand, it displays it in real time on the local monitoring terminal. The display interface presents key data such as the vehicle's license plate number (if recognizable), lane information, timestamp, and speed measurement result in the form of intuitive charts; on the other hand, the speed measurement result is sent to the remote monitoring center through communication methods such as 4G or wired network. The remote monitoring center summarizes and analyzes the data of multiple speed measurement points to achieve macro-monitoring of regional traffic flow, vehicle speed distribution, and other conditions.

[0053] During the entire system operation, the system calibration and self-diagnosis module runs continuously. If a camera failure is detected, such as a blocked lens or a damaged image acquisition card, an alarm signal will be immediately issued and the backup camera will be automatically switched to ensure uninterrupted operation of the system. At the same time, the sensor is calibrated regularly to ensure long-term stable performance of the system.

[0054] The electrical components appearing in this article are all electrically connected to an external main controller and 220V AC power, and the main controller can be a conventional known device that controls a computer, etc. The specific implementation method of the present disclosure omits the detailed description of known functions and known components. To ensure the compatibility of the equipment, the operating methods used are consistent with the parameters of marketed equipment.

[0055] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.

Claims

1. The vehicle speed measurement system based on radar and machine vision is characterized by: include Radar module: It uses high-performance millimeter-wave radar to transmit millimeter-wave signals and receive echo signals. It uses the Doppler effect and distance measurement principle to preliminarily calculate the radial speed and relative distance of the vehicle. It has a built-in adaptive signal processing algorithm and is equipped with intelligent target recognition and tracking functions. Machine vision module: uses a high-definition camera to collect vehicle image sequences, calculates the vehicle's two-dimensional plane speed through preprocessing and deep learning-based target detection algorithms, and introduces motion compensation algorithms; Data fusion module: A hybrid filtering algorithm based on extended Kalman filtering and particle filtering is used. The radar measurement data is used as the main observation value and the machine vision measurement data is used as the auxiliary observation value. The vehicle state model is established and the covariance matrix is ​​used to weigh the two sensor data. Speed ​​measurement result output and display module: receives the fused speed measurement results, displays them on the local monitoring terminal after visualization, and transmits them remotely to the monitoring center; System calibration and self-diagnosis module: calibrate radar and machine vision sensors, monitor system operation status in real time, issue alarms when abnormalities occur, and take fault-tolerant measures.

2. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: The radar module's adaptive signal processing algorithm automatically adjusts the radar's transmit power, signal gain, and filtering parameters based on real-time monitored environmental noise and interference signal strength.

3. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: The image processing unit of the machine vision module uses an adaptive histogram equalization algorithm to automatically adjust the pixel grayscale value according to the brightness distribution of different areas of the image, enhance the overall contrast of the image, and use Gaussian filtering to remove noise in the image.

4. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: The deep learning model of the machine vision module is pre-trained on a large number of image data sets containing various vehicle types, postures, and complex road conditions.

5. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: During the fusion process, the data fusion module uses the covariance matrix to accurately model the measurement noise according to the measurement noise characteristics of the radar and machine vision sensors, and adjusts the parameters of the covariance matrix in real time to ensure that the fusion algorithm can reasonably weigh the data of the two sensors.

6. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: The display interface of the speed measurement result output and display module adopts an intuitive and clear chart form, and the vehicle's license plate number, lane information, and timestamp key data are marked at the same time.

7. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: During the system initialization phase and regular operation, the system calibration and self-diagnosis module performs calibration operations on the radar and machine vision sensors, uses standard targets to calibrate the speed measurement accuracy and distance measurement accuracy of the radar, and the image acquisition parameters and target detection accuracy of the machine vision, and adjusts the operating parameters of the sensors.

8. The vehicle speed measurement system based on radar and machine vision according to claim 1, characterized in that: The system calibration and self-diagnosis module monitors the operating status of each system component in real time, including sensor signal strength, data transmission rate, processor load, etc. Once an abnormal situation is found, an alarm signal is immediately issued and corresponding fault-tolerant measures are taken.

Images