RSU antenna integrated with artificial intelligence technology

Abstract

The invention discloses an RSU antenna integrated with an artificial intelligence technology, and belongs to the field of RSU antennas. The RSU antenna integrated with the artificial intelligence technology comprises a DSRC module, an information collection module, an artificial intelligence processing module, a control module and a power supply and communication interface module. The problems that in the prior art, the resolution ratio is relatively low, the resolution ratio of a microwave radar is relatively low, the anti-interference performance is relatively weak, the wave beam is wide, and the resolution capability and the angle measurement precision are insufficient are solved, the camera and the radar are fused, the radar resolution capability and the angle measurement precision are improved, and the angle measurement precision is improved. Moreover, the method is beneficial to resisting electronic interference, clutter interference, multipath reflection interference and the like, improves the measurement precision and the resolution capability of distance and speed, can analyze target features, can effectively supplement regions which cannot be involved by a radar, achieves the rapid and accurate vehicle feature recognition, tracking and positioning functions, supports the rapid transaction between an RSU antenna and a vehicle-mounted OBU, and improves the reliability of the vehicle-mounted OBU. And the lane passing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of RSU antennas, and in particular to an RSU antenna integrating artificial intelligence technology. Background Art

[0002] DSRC technology is used to implement non-stop fast lanes in highway and parking lot management.

[0003] The Chinese patent with the publication number CN207020727U discloses a roadside unit based on DSRC transaction and radar technology. Its structure includes DSRC transaction module, radar module, control module, power supply and communication interface module. The DSRC transaction module sends and receives signals through the array antenna, and then completes the functions of modulation and demodulation, encoding and decoding to realize the interaction with OBU; the radar module is used to determine the vehicle information data; the control module receives the vehicle information data of the radar module, and sends and receives the interactive information data with OBU through the DSRC transaction module; the power supply and communication interface module supplies power to the control module, radar module and DSRC module, and ensures accurate communication between the modules. The radar has the ability to distinguish multiple targets. The roadside unit measures the distance of the same target through multiple radar hosts, and can calculate the three-dimensional spatial coordinates of the detected target. Through the combination of the DSRC transaction module and the radar module, it can accurately trade with the OBU (On Board Unit), effectively solve the problems of adjacent channel interference and trading, and improve the success rate of transactions.

[0004] In actual use, the above patents rely heavily on the radar's positioning function for vehicles. The radar's positioning capability is related to the resolution. The resolution refers to the ability to distinguish adjacent targets in front, behind, left and right at a certain distance. Obviously, the higher the resolution, the stronger the radar's recognition capability. The radar's resolution is closely related to the frequency it uses. The higher the frequency, the higher the resolution and the stronger the positioning capability. The radars of the above technologies are usually microwave radars, which can generally only detect tall buildings and large targets such as airplanes and ships, and have relatively low resolution. In addition to the requirements for resolution, anti-interference is also an important problem that radars need to solve. Otherwise, no matter how high the resolution is, it will not work. For example, when using microwave radar to detect ground or low-altitude targets, the echo signal is often submerged by the reflected wave from the ground, resulting in blind spots that cannot be detected. Microwave radars have low resolution and weak anti-interference, and their beam width, resolution capability and angular measurement accuracy are insufficient. Therefore, they do not meet existing needs. For this reason, we have proposed an RSU antenna that integrates artificial intelligence technology. Summary of the invention

[0005] The purpose of the present invention is to provide an RSU antenna integrated with artificial intelligence technology, which integrates the camera and radar, is beneficial to improving the radar resolution and angle measurement accuracy, and is beneficial to resisting electronic interference, clutter interference and multipath reflection interference, etc., improving the measurement accuracy and resolution of distance and speed and analyzing target characteristics, and can also effectively supplement the areas that the radar cannot cover, realize fast and accurate vehicle feature recognition, tracking, and positioning functions, accurately detect, track and video monitor vehicles within the target range around the clock, detect and process various abnormal events, sort out lane transaction queues, support RSU antennas and vehicle-mounted OBU quick transactions, realize lane vehicle queue transactions, improve lane transaction accuracy, and solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: an RSU antenna integrating artificial intelligence technology, comprising:

[0007] DSRC module, used to send and receive signals through array antenna, and then complete the modulation, demodulation and encoding and decoding functions to achieve interaction with OBU;

[0008] The information collection module is used to take photos of the vehicle, improve the vehicle information collected by the millimeter wave radar module, and pre-process the collected information data;

[0009] The artificial intelligence processing module is used to analyze and evaluate the lane status based on the data information collected by the information collection module, using the artificial intelligence technology deep learning algorithm and the TensorFlow target detection and target tracking technology to obtain the vehicle license plate number and location information;

[0010] The control module is used to receive the information data from the information collection module and the artificial intelligence module, fuse the received information data, and control the DSRC module to send and receive interactive information data with the OBU;

[0011] The power supply and communication interface module is used to power the artificial intelligence processing module, control module, millimeter wave radar module, camera module and DSRC module, and ensure accurate communication between the modules.

[0012] Preferably, the information collection module includes:

[0013] Millimeter-wave radar module, used to collect vehicle distance information, speed information, angle information, object type, obstacle and environmental perception information;

[0014] The camera module is used to collect characteristic information data of vehicles, drivers, pedestrians and landmark objects on the traffic road;

[0015] The preprocessing module is used to preprocess the information data collected by the millimeter wave radar module and the camera module.

[0016] Preferably, the information collection module further includes:

[0017] A parameter real-time monitoring module, used for real-time monitoring of the operating parameters of the millimeter-wave radar module, wherein the operating parameters of the millimeter-wave radar module include ranging accuracy, speed resolution (speed resolution refers to the accuracy with which the radar can distinguish two targets with similar speeds), angle resolution (angle resolution refers to the accuracy with which the radar can distinguish two targets with similar angles) and radar data refresh rate;

[0018] A first data comparison module, used for comparing the distance measurement accuracy with a preset distance measurement accuracy threshold;

[0019] A first operation evaluation coefficient acquisition module, configured to retrieve the angle resolution and the speed resolution to acquire the first operation evaluation coefficient when the ranging accuracy is lower than a preset ranging accuracy threshold;

[0020] The first operation evaluation coefficient is obtained by the following formula:

[0021]

[0022] Among them, R 01 represents the first operation evaluation coefficient; n represents the number of unit times experienced by the millimeter wave radar module in operation, and the unit time is 1s; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; V max represents the maximum value of velocity resolution corresponding to n unit time; w max Indicates the maximum angular resolution corresponding to n unit time; Z y Indicates the preset ranging accuracy threshold; F 01 and F 02 denote the first coefficient and the second coefficient respectively, and the first coefficient and the second coefficient are obtained by the following formula:

[0023]

[0024] Among them, F 01 and F 02 Represent the first coefficient and the second coefficient respectively; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; V max Indicates the maximum value of velocity resolution corresponding to n unit time; Vmin Indicates the minimum velocity resolution corresponding to n unit time; w max represents the maximum angular resolution corresponding to n unit time; w min Indicates the minimum angular resolution corresponding to n unit time; Z indicates the ranging accuracy corresponding to the range accuracy threshold value lower than the preset range accuracy threshold;

[0025] A second data comparison module, used for comparing the first operation evaluation coefficient with a preset first evaluation coefficient threshold;

[0026] A first abnormal warning module, configured to determine that the millimeter wave radar module is operating abnormally and issue an abnormal warning when the first operation evaluation coefficient is lower than a preset first evaluation coefficient threshold;

[0027] The secondary abnormality determination module is used to retrieve the radar data refresh rate for secondary abnormality determination when the first operation evaluation coefficient is not lower than a preset first evaluation coefficient threshold.

[0028] Preferably, the secondary abnormality determination module comprises:

[0029] A radar data refresh rate calling module, configured to call the radar data refresh rate when the first operation evaluation coefficient is not lower than a preset first evaluation coefficient threshold;

[0030] A second operation evaluation coefficient acquisition module, configured to acquire a second operation evaluation coefficient by using the first operation evaluation coefficient in combination with the radar data refresh rate;

[0031] The second operation evaluation coefficient is obtained by the following formula:

[0032]

[0033] Among them, R 02 Represents the second operation evaluation coefficient; R 01 Represents the first operation evaluation coefficient; R y01 represents the first evaluation coefficient threshold; n represents the number of unit times experienced by the millimeter wave radar module, and the unit time is 1s; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; S i Indicates the radar data refresh rate corresponding to the i-th unit time;

[0034] A third data comparison module, used for comparing the second operation evaluation coefficient with a preset second evaluation coefficient threshold;

[0035] The second abnormal warning module is used to determine that the millimeter wave radar module is operating abnormally and issue an abnormal warning when the second operation evaluation coefficient is lower than a preset second evaluation coefficient threshold.

[0036] Preferably, the preprocessing module comprises:

[0037] Perform data cleaning, standardization and normalization on the information data collected by the millimeter wave radar module and camera module;

[0038] Remove communication interference, white noise interference and random pulse interference from the information data collected by the millimeter wave radar module and the camera module;

[0039] Data enhancement is performed on the data after interference removal, so that the characteristic parameters in the data after interference removal are enhanced.

[0040] Preferably, the artificial intelligence processing module includes:

[0041] A coordinate conversion module is used to determine the position information of the vehicle according to the information data collected by the millimeter wave radar module and the camera module, perform vehicle coordinate position conversion according to the vehicle position information, and determine coordinate conversion information data;

[0042] A feature analysis module is used to perform feature analysis on the information data collected by the millimeter wave radar module and the camera module based on the coordinate conversion information data, and determine the image features of the scene where the vehicle is located;

[0043] An information acquisition module is used to preprocess and identify the image features of the scene where the vehicle is located, and obtain a lane feature identification information data set;

[0044] Faster R-CNN network model, used to build, train and evaluate the Faster R-CNN network model;

[0045] The intelligent analysis module uses the Faster R-CNN network model to extract the position and direction information data of the lane line, evaluates the motion state of the target vehicle and analyzes the lane state, obtains the lane state based on the analysis results, and combines the vehicle license plate number and position information obtained by the camera module to analyze the traffic flow and density, and visualize the detection and tracking results.

[0046] Preferably, the information acquisition module includes:

[0047] Setting labeling rule information data, extracting a feature from the scene image where the vehicle is located in turn, traversing the scene image where the vehicle is located, and obtaining multiple feature sets;

[0048] Construct a target scatter plot set based on multiple feature sets, and form a gridding solution set based on the target scatter plot set;

[0049] The target scatter plot set is partitioned based on the gridding scheme set, and the mutual information data values ​​are calculated for the multiple partition results in turn to obtain multiple maximum mutual information data values;

[0050] Determine multiple target maximum information data coefficients based on multiple maximum mutual information data values, arrange the multiple target maximum information data coefficients in descending order, and perform reverse matching to obtain a factor feature sequence;

[0051] Extract the factor features with a preset ranking threshold in the factor feature sequence to form a target factor feature set;

[0052] The target factor feature set is identified by using the labeling conditions in the labeling rule information data to obtain a lane feature identification information data set including lane line images and labeling information data.

[0053] Preferably, the Faster R-CNN network model includes:

[0054] The model building module is used to build the Faster R-CNN network model, use the lane feature identification information dataset containing lane line images and annotation information data to complete the location and category recognition of the target vehicle, and divide the labeled image data into a training set and a validation set;

[0055] The model training module is used to train the Faster R-CNN network model using the training set and adjust the hyperparameters of the Faster R-CNN network model through the back propagation algorithm;

[0056] The model evaluation module is used to evaluate the performance of the Faster R-CNN network model using the validation set, adjust the Faster R-CNN network model parameters according to the evaluation results, and optimize the performance of the Faster R-CNN network model.

[0057] Preferably, the intelligent analysis module includes:

[0058] The lane detection module is used to detect the lane image where the target vehicle is located using the trained Faster R-CNN network model and extract the position and direction information data of the lane.

[0059] The target tracking module is used to evaluate the motion state of the target vehicle by calculating the pixel displacement between consecutive frames;

[0060] A state analysis module is used to analyze the lane state according to the position and direction information data of the lane line where the target vehicle is located;

[0061] The result display module is used to visualize the detection and tracking results and display the lane position information and status analysis results of the target vehicle.

[0062] Preferably, the target tracking module includes:

[0063] Use the object detection algorithm to detect the vehicle in each frame captured by the camera module, and use the SORT algorithm to track the movement of the vehicle between different frames;

[0064] Use feature extraction algorithms to extract key points on the vehicle, and use matching algorithms to find corresponding feature points between consecutive frames;

[0065] For each pair of matched feature points, calculate the coordinate difference between the current frame and the previous frame, i.e., pixel displacement, and convert the pixel displacement into actual physical displacement;

[0066] According to the calculated actual displacement, the motion state of the target vehicle is evaluated, and the calculated motion state data is smoothed and filtered to reduce noise and errors to obtain the final motion state of the target vehicle.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] The present invention integrates the camera and the radar. Under the condition of the same antenna caliber, the millimeter-wave radar has a narrower beam, which is beneficial to improving the radar resolution and angle measurement accuracy, and is beneficial to resisting electronic interference, clutter interference and multipath reflection interference, etc., and is beneficial to improving the measurement accuracy and resolution of distance and speed and analyzing target characteristics. It can also effectively supplement the areas that the radar cannot cover, and performs artificial intelligence analysis and processing through the artificial intelligence processing module, analyzes the decision-making plan and the processing plan according to the set algorithm, and can automatically control the entire system for linkage processing. It combines the characteristics of radar precise event detection and video visualization as well as artificial intelligence self-learning evolution, realizes fast and accurate vehicle feature recognition, tracking, and positioning functions, performs accurate detection, tracking and video monitoring of vehicles within the target range around the clock, detects and processes various abnormal events, sorts out lane transaction queues, supports quick transactions between RSU antennas and on-board OBUs, realizes lane vehicle queue transactions, improves lane transaction accuracy, and improves lane traffic efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of an RSU antenna module integrating artificial intelligence technology of the present invention;

[0070] Figure 2 Schematic diagram of the RSU antenna method integrating artificial intelligence technology of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. According to the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In order to solve the problems of relatively low resolution of existing patents, low resolution and weak anti-interference of microwave radar, insufficient beam width, resolution and angle measurement accuracy, please refer to Figure 1-Figure 2 , this embodiment provides the following technical solutions:

[0073] RSU antenna with integrated AI technology, including:

[0074] DSRC module, used to send and receive signals through array antenna, and then complete the modulation, demodulation and encoding and decoding functions to achieve interaction with OBU;

[0075] The information collection module is used to take photos of the vehicle, improve the vehicle information collected by the millimeter wave radar module, and pre-process the collected information data;

[0076] The artificial intelligence processing module is used to analyze and evaluate the lane status based on the data information collected by the information collection module, using the artificial intelligence technology deep learning algorithm and the TensorFlow target detection and target tracking technology to obtain the vehicle license plate number and location information;

[0077] The control module is used to receive the information data from the information collection module and the artificial intelligence module, fuse the received information data, and control the DSRC module to send and receive interactive information data with the OBU;

[0078] The power supply and communication interface module is used to power the artificial intelligence processing module, control module, millimeter wave radar module, camera module and DSRC module, and ensure accurate communication between the modules.

[0079] Information collection module, including:

[0080] Millimeter-wave radar module, used to collect vehicle distance information, speed information, angle information, object type, obstacle and environmental perception information;

[0081] The camera module is used to collect characteristic information data of vehicles, drivers, pedestrians and landmark target objects on the traffic road, such as license plate, color, face shape, outline, shape, operating status, etc.

[0082] Millimeter-wave radar can achieve higher-precision target speed detection through the principle of Doppler shift, and is less affected by light and weather factors. The camera solution is low-cost and can identify different objects. It has advantages in object height and width measurement accuracy, lane line recognition, pedestrian recognition accuracy, etc. It is an indispensable sensor for realizing lane departure warning, traffic sign recognition and other functions. Therefore, the camera and radar are integrated. Under the condition of the same antenna caliber, the millimeter-wave radar has a narrower beam, which is conducive to improving the radar resolution and angle measurement accuracy, and is conducive to resisting electronic interference, clutter interference and multipath reflection interference. Due to the high operating frequency, a large signal bandwidth and Doppler frequency shift may be obtained, which is conducive to improving the measurement accuracy and resolution of distance and speed and analyzing target characteristics. The antenna caliber and components are small in size and light in weight, which is conducive to loading. Combining the camera module can reduce the dependence of the roadside unit on the radar module, and can also effectively supplement the areas that the radar cannot cover. Through the artificial intelligence processing module, artificial intelligence analysis and processing are performed, and the decision-making plan and processing plan are analyzed according to the set algorithm, and the entire system can be automatically controlled for linkage processing.

[0083] The preprocessing module is used to preprocess the information data collected by the millimeter wave radar module and the camera module.

[0084] Millimeter-wave radar sends and receives electromagnetic waves, calculates the time from the electromagnetic waves being sent to being reflected, and thus determines the distance of the target object. It can accurately measure the distance between the vehicle and other objects. Millimeter-wave radar uses the Doppler effect to measure the relative speed of the target object. Through beamforming technology, such as array antennas, the azimuth of the target object can be determined to obtain the angle information of the target object. Millimeter-wave radar can not only obtain distance, speed and angle information, but also identify different types of objects such as vehicles, pedestrians, bicycles, etc. through signal processing and pattern recognition technology. Millimeter-wave radar can detect obstacles ahead, such as other vehicles or pedestrians, and issue warnings immediately. It can provide better perception capabilities in severe weather conditions, such as being able to accurately identify and track target objects in rain, snow, fog and haze.

[0085] Preferably, the information collection module includes:

[0086] Millimeter-wave radar module, used to collect vehicle distance information, speed information, angle information, object type, obstacle and environmental perception information;

[0087] The camera module is used to collect characteristic information data of vehicles, drivers, pedestrians and landmark objects on the traffic road;

[0088] The preprocessing module is used to preprocess the information data collected by the millimeter wave radar module and the camera module.

[0089] Specifically, the information collection module further includes:

[0090] A parameter real-time monitoring module, used for real-time monitoring of the operating parameters of the millimeter-wave radar module, wherein the operating parameters of the millimeter-wave radar module include ranging accuracy, speed resolution (speed resolution refers to the accuracy with which the radar can distinguish two targets with similar speeds), angle resolution (angle resolution refers to the accuracy with which the radar can distinguish two targets with similar angles) and radar data refresh rate;

[0091] A first data comparison module, used for comparing the distance measurement accuracy with a preset distance measurement accuracy threshold;

[0092] A first operation evaluation coefficient acquisition module, configured to retrieve the angle resolution and the speed resolution to acquire the first operation evaluation coefficient when the ranging accuracy is lower than a preset ranging accuracy threshold;

[0093] The first operation evaluation coefficient is obtained by the following formula:

[0094]

[0095] Among them, R 01 represents the first operation evaluation coefficient; n represents the number of unit times experienced by the millimeter wave radar module in operation, and the unit time is 1s; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; V max represents the maximum value of velocity resolution corresponding to n unit time; w max Indicates the maximum angular resolution corresponding to n unit time; Z y Indicates the preset ranging accuracy threshold; F 01 and F 02 denote the first coefficient and the second coefficient respectively, and the first coefficient and the second coefficient are obtained by the following formula:

[0096]

[0097] Among them, F 01 and F 02 Represent the first coefficient and the second coefficient respectively; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; V max Indicates the maximum value of velocity resolution corresponding to n unit time; V min Indicates the minimum velocity resolution corresponding to n unit time; w maxrepresents the maximum angular resolution corresponding to n unit time; w min Indicates the minimum angular resolution corresponding to n unit time; Z indicates the ranging accuracy corresponding to the range accuracy threshold value lower than the preset range accuracy threshold;

[0098] A second data comparison module, used for comparing the first operation evaluation coefficient with a preset first evaluation coefficient threshold;

[0099] A first abnormal warning module, configured to determine that the millimeter wave radar module is operating abnormally and issue an abnormal warning when the first operation evaluation coefficient is lower than a preset first evaluation coefficient threshold;

[0100] The secondary abnormality determination module is used to retrieve the radar data refresh rate for secondary abnormality determination when the first operation evaluation coefficient is not lower than a preset first evaluation coefficient threshold.

[0101] The technical effect of the above technical solution is: through the parameter real-time monitoring module, the key operating parameters of the millimeter wave radar module can be obtained in real time, including ranging accuracy, speed resolution, angle resolution and radar data refresh rate. These parameters are crucial for evaluating the performance and operating status of the radar. Using the first data comparison module and the first operation evaluation coefficient acquisition module, when the ranging accuracy is lower than the preset threshold, it can automatically trigger further evaluation of the speed resolution and angle resolution, and calculate the first operation evaluation coefficient. This mechanism helps to issue an early warning at the beginning of the radar performance degradation, thereby avoiding possible failures or accidents. The calculation formula of the first operation evaluation coefficient takes into account the fluctuation of the speed resolution and angle resolution, as well as their distribution relative to the maximum and minimum values. This comprehensive evaluation method can more comprehensively reflect the operating status of the radar and improve the accuracy of the evaluation. Through the second data comparison module and the first abnormal warning module, when the first operation evaluation coefficient is lower than the preset threshold, the radar operation will be immediately judged to be abnormal and an early warning will be issued. This multi-level early warning mechanism can ensure that necessary measures are taken before the radar performance is seriously degraded. When the first operation evaluation coefficient is not lower than the preset threshold, the secondary abnormal judgment module will further retrieve the radar data refresh rate for judgment. This step helps to detect problems in advance when the radar performance is close to or reaches its limit, providing a time window for subsequent maintenance and optimization. This technical solution can significantly improve the reliability and safety of millimeter-wave radar modules through real-time monitoring, comprehensive evaluation and multi-level early warning mechanisms. This is especially important for systems that rely on radar for precise perception and decision-making (such as self-driving cars, intelligent transportation systems, etc.).

[0102] In summary, this technical solution provides an efficient and accurate method for operation monitoring and abnormal warning of millimeter-wave radar modules, which helps to improve the performance and safety of the entire system.

[0103] Specifically, the secondary abnormality determination module includes:

[0104] A radar data refresh rate calling module, configured to call the radar data refresh rate when the first operation evaluation coefficient is not lower than a preset first evaluation coefficient threshold;

[0105] A second operation evaluation coefficient acquisition module, configured to acquire a second operation evaluation coefficient by using the first operation evaluation coefficient in combination with the radar data refresh rate;

[0106] The second operation evaluation coefficient is obtained by the following formula:

[0107]

[0108] Among them, R 02 Represents the second operation evaluation coefficient; R 01 Represents the first operation evaluation coefficient; R y01 represents the first evaluation coefficient threshold; n represents the number of unit times experienced by the millimeter wave radar module, and the unit time is 1s; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; S i Indicates the radar data refresh rate corresponding to the i-th unit time;

[0109] A third data comparison module, used for comparing the second operation evaluation coefficient with a preset second evaluation coefficient threshold;

[0110] The second abnormal warning module is used to determine that the millimeter wave radar module is operating abnormally and issue an abnormal warning when the second operation evaluation coefficient is lower than a preset second evaluation coefficient threshold.

[0111] The technical effect of the above technical solution is: by introducing the radar data refresh rate as a new evaluation indicator and combining it with the first operation evaluation coefficient, the technical solution can achieve a more refined evaluation of the operation status of the millimeter wave radar module. The radar data refresh rate reflects the speed at which the radar system updates data, which is crucial for real-time monitoring and rapid response. The calculation formula of the second operation evaluation coefficient comprehensively considers the first operation evaluation coefficient (i.e., the previous evaluation results of the velocity resolution and angle resolution), the radar data refresh rate, and the preset first evaluation coefficient threshold. This comprehensive consideration method can more comprehensively reflect the overall performance of the radar system and improve the accuracy of the evaluation. On the basis of the original early warning mechanism, the technical solution adds a secondary abnormality determination module to form a multi-level early warning mechanism. When the first operation evaluation coefficient is not lower than the preset threshold, it is not directly determined that the radar is operating normally, but is further combined with the radar data refresh rate for a secondary evaluation. This mechanism can more effectively identify potential operation anomalies and improve the accuracy and timeliness of early warnings. Through real-time monitoring and refined evaluation, the technical solution can promptly detect the operation anomalies of the millimeter wave radar module and remind it through the abnormal warning module. This helps relevant personnel to take timely measures to repair and optimize, thereby improving the reliability and stability of the entire system. The evaluation indicators and early warning mechanism in this technical solution can be adjusted and optimized according to actual conditions. For example, different evaluation coefficient thresholds and calculation formulas can be set according to different application scenarios and needs. This flexibility enables the technical solution to adapt to application needs in different environments and conditions.

[0112] In summary, this technical solution, by introducing the radar data refresh rate as a new evaluation indicator and combining it with the first operation evaluation coefficient for refined evaluation, achieves more comprehensive and accurate monitoring and early warning of the millimeter wave radar module's operating status. This helps to improve the reliability and stability of the system and provide strong guarantees for related applications.

[0113] Preprocessing modules include:

[0114] Perform data cleaning, standardization and normalization on the information data collected by the millimeter wave radar module and the camera module;

[0115] Remove communication interference, white noise interference and random pulse interference from the information data collected by the millimeter wave radar module and the camera module;

[0116] Data enhancement is performed on the data after interference removal, so that the characteristic parameters in the data after interference removal are enhanced.

[0117] By preprocessing the information data collected by the millimeter-wave radar module and the camera module, the data quality can be improved. By cleaning missing values ​​and outliers, the integrity and accuracy of the data can be ensured. Through standardization and normalization, the impact of dimensional inconsistency between features can be reduced, and the training efficiency and prediction accuracy of the model can be improved. Through data enhancement and data balancing, the generalization ability and robustness of the model can be improved.

[0118] Artificial intelligence processing module, including:

[0119] A coordinate conversion module is used to determine the position information of the vehicle according to the information data collected by the millimeter wave radar module and the camera module, perform vehicle coordinate position conversion according to the vehicle position information, and determine coordinate conversion information data;

[0120] A feature analysis module is used to perform feature analysis on the information data collected by the millimeter wave radar module and the camera module based on the coordinate conversion information data, and determine the image features of the scene where the vehicle is located;

[0121] An information acquisition module is used to preprocess and identify the image features of the scene where the vehicle is located, and obtain a lane feature identification information data set;

[0122] Faster R-CNN network model, used to build, train and evaluate the Faster R-CNN network model;

[0123] The intelligent analysis module uses the Faster R-CNN network model to extract the position and direction information data of the lane line, evaluates the motion state of the target vehicle and analyzes the lane state, obtains the lane state based on the analysis results, and combines the vehicle license plate number and position information obtained by the camera module to analyze the traffic flow and density, and visualize the detection and tracking results.

[0124] Information acquisition module, including:

[0125] Setting labeling rule information data, extracting a feature from the scene image where the vehicle is located in turn, traversing the scene image where the vehicle is located, and obtaining multiple feature sets;

[0126] Construct a target scatter plot set based on multiple feature sets, and form a gridding solution set based on the target scatter plot set;

[0127] The target scatter plot set is partitioned based on the gridding scheme set, and the mutual information data values ​​are calculated for the multiple partition results in turn to obtain multiple maximum mutual information data values;

[0128] Determine multiple target maximum information data coefficients based on multiple maximum mutual information data values, arrange the multiple target maximum information data coefficients in descending order, and perform reverse matching to obtain a factor feature sequence;

[0129] Extract the factor features with a preset ranking threshold in the factor feature sequence to form a target factor feature set;

[0130] The target factor feature set is identified by using the labeling conditions in the labeling rule information data to obtain a lane feature identification information data set including lane line images and labeling information data.

[0131] Faster R-CNN network model, including:

[0132] The model building module is used to build the Faster R-CNN network model, use the lane feature identification information dataset containing lane line images and annotation information data to complete the location and category recognition of the target vehicle, and divide the labeled image data into training set and validation set

[0133] The model training module is used to train the Faster R-CNN network model using the training set and adjust the hyperparameters of the Faster R-CNN network model, such as the learning rate and batch size, through the back-propagation algorithm, so that the model can accurately identify the features of the target object;

[0134] The model evaluation module is used to evaluate the performance of the Faster R-CNN network model using the validation set, adjust the Faster R-CNN network model parameters according to the evaluation results, and optimize the performance of the Faster R-CNN network model.

[0135] Intelligent analysis module, including:

[0136] The lane detection module is used to detect the lane image where the target vehicle is located using the trained Faster R-CNN network model and extract the position and direction information data of the lane;

[0137] The target tracking module is used to evaluate the motion state of the target vehicle by calculating the pixel displacement between consecutive frames and to track lane lines in dynamic scenes. It can use methods such as optical flow and Kalman filtering to track targets.

[0138] The state analysis module is used to analyze the lane status, such as lane departure, lane congestion, etc., based on the position and direction information data of the lane line where the target vehicle is located. A threshold can be set to determine whether the lane status is normal;

[0139] The result display module is used to visualize the detection and tracking results, and to display the lane line position information and status analysis results of the target vehicle. The lane lines and status analysis results can be drawn using the drawing function of OpenCV.

[0140] Target tracking module, including:

[0141] Use the object detection algorithm to detect the vehicle in each frame captured by the camera module, and use the SORT algorithm to track the movement of the vehicle between different frames;

[0142] Use feature extraction algorithms to extract key points on the vehicle, and use matching algorithms to find corresponding feature points between consecutive frames;

[0143] For each pair of matched feature points, calculate the coordinate difference between the current frame and the previous frame, i.e., pixel displacement, and convert the pixel displacement into actual physical displacement;

[0144] According to the calculated actual displacement, the motion state of the target vehicle is evaluated, and the calculated motion state data is smoothed and filtered to reduce noise and errors to obtain the final motion state of the target vehicle.

[0145] Millimeter-wave radar can achieve higher-precision target speed detection through the principle of Doppler shift and is less affected by lighting and weather factors. The camera solution is low-cost, can identify different objects, and has advantages in object height and width measurement accuracy, lane line recognition, and pedestrian recognition accuracy. It is an indispensable sensor for realizing lane departure warning, traffic sign recognition and other functions. The camera and radar are integrated and then combined with the artificial intelligence processing module to form this roadside unit. The motion state of the target vehicle is evaluated by analyzing the pixel changes between consecutive frames. It does not require additional training process, has a fast calculation speed, and can capture subtle movements, such as motion blur or fast target movement, so the estimation of target motion is more accurate.

[0146] When using the RSU antenna integrated with artificial intelligence technology of the present invention, combined with Figure 1 and Figure 2 , including the following steps:

[0147] Step 1: First, send and receive signals through the array antenna and interact with the OBU;

[0148] Step 2: Use the millimeter-wave radar module and camera module to collect the vehicle's distance information, speed information, angle information, object type, obstacle and environmental perception information, take photos of the vehicle, and pre-process the collected information data. The RSU with integrated artificial intelligence technology combines the millimeter-wave radar and camera to collect information. The millimeter-wave radar measures the distance and speed and tracks the trajectory of the object; the camera detects the shape, complementing each other's advantages to achieve information fusion of target data;

[0149] Step 3: Based on the data collected by the information collection module, the lane status is analyzed and evaluated using the deep learning algorithm of artificial intelligence technology and the target detection and target tracking technology of TensorFlow;

[0150] Step 4: Get the lane status based on the analysis results, and analyze the traffic flow and density by combining the vehicle license plate number and location information obtained by the camera module. Use artificial intelligence to perform data analysis and density estimation on the traffic flow to achieve intelligent traffic management and regulation, and ultimately achieve the goal of alleviating traffic congestion. This is a balanced solution for cost, accuracy, and speed.

[0151] Step 5: Receive information data from the information collection module and the artificial intelligence module, fuse the received information data, and upload the fused lane status information.

[0152] In summary, the RSU antenna integrated with artificial intelligence technology of the present invention uses a camera and a millimeter-wave radar to collect data for the observed target respectively, and obtains different perception information data, which can complement each other. Then the artificial intelligence processing module performs feature extraction and pattern recognition processing on the output data of each sensor, and accurately associates the target by category. Finally, the artificial intelligence fusion algorithm is used to integrate all sensor data of the same target, so as to draw a consistent conclusion about the target. Therefore, the camera and the radar are fused. When the antenna caliber is the same, the millimeter-wave radar has a narrower beam, which is beneficial to improving the radar resolution and angle measurement accuracy, and is beneficial to resisting electronic interference, clutter interference and multipath reflection interference, etc., which is beneficial to improving the measurement accuracy and resolution of distance and speed and can analyze target characteristics. The millimeter-wave radar is small in size and light in weight, which is conducive to carrying. Combined with the camera module, it can reduce the dependence of the roadside unit on the radar module, and can also effectively supplement the areas that the radar cannot cover. It performs artificial intelligence analysis and processing through the artificial intelligence processing module, analyzes the decision-making plan and processing plan according to the set algorithm, and can automatically control the entire system for linkage processing. It combines the characteristics of radar's precise event detection and video visualization as well as artificial intelligence's self-learning and evolution to achieve fast and accurate vehicle feature recognition, tracking, and positioning functions, and performs accurate detection, tracking, and video monitoring of vehicles within the target range around the clock. It detects and processes a variety of abnormal events, sorts out lane transaction queues, supports quick transactions between RSU antennas and on-board OBUs, realizes lane vehicle queue transactions, improves lane transaction accuracy, and improves lane traffic efficiency.

[0153] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0154] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. RSU antenna integrated with artificial intelligence technology, characterized in that: include: DSRC module, used to send and receive signals through array antenna, complete modulation and demodulation and encoding and decoding functions, and interact with OBU; The information collection module is used to take photos of the vehicle, improve the vehicle information collected by the millimeter wave radar module, and pre-process the collected information data; The artificial intelligence processing module is used to analyze and evaluate the lane status based on the data information collected by the information collection module, using the artificial intelligence technology deep learning algorithm and the target detection and target tracking technology of TensorFlow to obtain the vehicle license plate number and location information; The control module is used to receive the information data from the information collection module and the artificial intelligence module, fuse the received information data, and control the DSRC module to send and receive interactive information data with the OBU; The power supply and communication interface module is used to power the artificial intelligence processing module, control module, millimeter wave radar module, camera module and DSRC module, and ensure accurate communication between the modules.

2. The RSU antenna integrated with artificial intelligence technology according to claim 1, characterized in that: The information collection module includes: Millimeter-wave radar module, used to collect vehicle distance information, speed information, angle information, object type, obstacle and environmental perception information; The camera module is used to collect characteristic information data of vehicles, drivers, pedestrians and landmark objects on the traffic road; The preprocessing module is used to preprocess the information data collected by the millimeter wave radar module and the camera module.

3. The RSU antenna integrated with artificial intelligence technology according to claim 2, characterized in that: The information collection module further includes: A parameter real-time monitoring module, used for real-time monitoring of the operating parameters of the millimeter-wave radar module, wherein the operating parameters of the millimeter-wave radar module include ranging accuracy, velocity resolution, angle resolution and radar data refresh rate; A first data comparison module, used for comparing the distance measurement accuracy with a preset distance measurement accuracy threshold; A first operation evaluation coefficient acquisition module, configured to retrieve the angle resolution and the speed resolution to acquire the first operation evaluation coefficient when the ranging accuracy is lower than a preset ranging accuracy threshold; The first operation evaluation coefficient is obtained by the following formula: Among them, R 01 represents the first operation evaluation coefficient; n represents the number of unit times experienced by the millimeter wave radar module in operation, and the unit time is 1s; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; V max represents the maximum value of velocity resolution corresponding to n unit time; w max Indicates the maximum angular resolution corresponding to n unit time; Z y Indicates the preset ranging accuracy threshold; F 01 and F 02 denote the first coefficient and the second coefficient respectively, and the first coefficient and the second coefficient are obtained by the following formula: Among them, F 01 and F 02 Represent the first coefficient and the second coefficient respectively; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; V max Indicates the maximum value of velocity resolution corresponding to n unit time; V min Indicates the minimum velocity resolution corresponding to n unit time; w max represents the maximum angular resolution corresponding to n unit time; w min Indicates the minimum angular resolution corresponding to n unit time; Z indicates the ranging accuracy corresponding to the range accuracy threshold value lower than the preset range accuracy threshold; A second data comparison module, used for comparing the first operation evaluation coefficient with a preset first evaluation coefficient threshold; A first abnormal warning module, configured to determine that the millimeter wave radar module is operating abnormally and issue an abnormal warning when the first operation evaluation coefficient is lower than a preset first evaluation coefficient threshold; The secondary abnormality determination module is used to retrieve the radar data refresh rate for secondary abnormality determination when the first operation evaluation coefficient is not lower than a preset first evaluation coefficient threshold.

4. The RSU antenna integrated with artificial intelligence technology according to claim 3, characterized in that: The secondary abnormality determination module comprises: A radar data refresh rate calling module, configured to call the radar data refresh rate when the first operation evaluation coefficient is not lower than a preset first evaluation coefficient threshold; A second operation evaluation coefficient acquisition module, configured to acquire a second operation evaluation coefficient by using the first operation evaluation coefficient in combination with the radar data refresh rate; The second operation evaluation coefficient is obtained by the following formula: Among them, R 02 Represents the second operation evaluation coefficient; R 01 Represents the first operation evaluation coefficient; R y01 represents the first evaluation coefficient threshold; n represents the number of unit times experienced by the millimeter wave radar module, and the unit time is 1s; V fi represents the velocity resolution corresponding to the i-th unit time; w fi represents the angular resolution corresponding to the i-th unit time; S i Indicates the radar data refresh rate corresponding to the i-th unit time; A third data comparison module, used for comparing the second operation evaluation coefficient with a preset second evaluation coefficient threshold; The second abnormal warning module is used to determine that the millimeter wave radar module is operating abnormally and issue an abnormal warning when the second operation evaluation coefficient is lower than a preset second evaluation coefficient threshold.

5. The RSU antenna integrated with artificial intelligence technology according to claim 2, characterized in that: The preprocessing module comprises: Perform data cleaning, standardization and normalization on the information data collected by the millimeter wave radar module and camera module; Remove communication interference, white noise interference and random pulse interference from the information data collected by the millimeter wave radar module and the camera module; Data enhancement is performed on the data after interference removal, so that the characteristic parameters in the data after interference removal are enhanced.

6. The RSU antenna integrated with artificial intelligence technology according to claim 1, characterized in that: The artificial intelligence processing module comprises: A coordinate conversion module is used to determine the position information of the vehicle according to the information data collected by the millimeter wave radar module and the camera module, perform vehicle coordinate position conversion according to the vehicle position information, and determine coordinate conversion information data; A feature analysis module is used to perform feature analysis on the information data collected by the millimeter wave radar module and the camera module based on the coordinate conversion information data, and determine the image features of the scene where the vehicle is located; An information acquisition module is used to preprocess and identify the image features of the scene where the vehicle is located, and obtain a lane feature identification information data set; Faster R-CNN network model, used to build, train and evaluate the Faster R-CNN network model; The intelligent analysis module uses the Faster R-CNN network model to extract the position and direction information data of the lane line, evaluates the motion state of the target vehicle and analyzes the lane state, obtains the lane state based on the analysis results, and combines the vehicle license plate number and position information obtained by the camera module to analyze the traffic flow and density, and visualize the detection and tracking results.

7. The RSU antenna integrated with artificial intelligence technology according to claim 6, characterized in that: The information acquisition module comprises: Setting labeling rule information data, extracting a feature from the scene image where the vehicle is located in turn, traversing the scene image where the vehicle is located, and obtaining multiple feature sets; Construct a target scatter plot set based on multiple feature sets, and form a gridding solution set based on the target scatter plot set; The target scatter plot set is partitioned based on the gridding scheme set, and the mutual information data values ​​are calculated for the multiple partition results in turn to obtain multiple maximum mutual information data values; Determine multiple target maximum information data coefficients based on multiple maximum mutual information data values, arrange the multiple target maximum information data coefficients in descending order, and perform reverse matching to obtain a factor feature sequence; Extract the factor features with a preset ranking threshold in the factor feature sequence to form a target factor feature set; The target factor feature set is identified by using the labeling conditions in the labeling rule information data to obtain a lane feature identification information data set including lane line images and labeling information data.

8. The RSU antenna integrated with artificial intelligence technology according to claim 6, characterized in that: The Faster R-CNN network model includes: The model building module is used to build the Faster R-CNN network model, use the lane feature identification information dataset containing lane line images and annotation information data to complete the location and category recognition of the target vehicle, and divide the labeled image data into a training set and a validation set; The model training module is used to train the Faster R-CNN network model using the training set and adjust the hyperparameters of the Faster R-CNN network model through the back propagation algorithm; The model evaluation module is used to evaluate the performance of the Faster R-CNN network model using the validation set, adjust the Faster R-CNN network model parameters according to the evaluation results, and optimize the performance of the Faster R-CNN network model.

9. The RSU antenna integrated with artificial intelligence technology according to claim 1, characterized in that: The intelligent analysis module comprises: The lane line detection module is used to detect the lane line image where the target vehicle is located using the trained Faster R-CNN network model and extract the position and direction information data of the lane line. The target tracking module is used to evaluate the motion state of the target vehicle by calculating the pixel displacement between consecutive frames; A state analysis module is used to analyze the lane state according to the position and direction information data of the lane line where the target vehicle is located; The result display module is used to visualize the detection and tracking results and display the lane position information and status analysis results of the target vehicle.

10. The RSU antenna integrated with artificial intelligence technology according to claim 9, characterized in that: The target tracking module comprises: Use the object detection algorithm to detect the vehicle in each frame captured by the camera module, and use the SORT algorithm to track the movement of the vehicle between different frames; Use feature extraction algorithms to extract key points on the vehicle, and use matching algorithms to find corresponding feature points between consecutive frames; For each pair of matched feature points, the coordinate difference between the current frame and the previous frame is calculated. The coordinate difference is the pixel displacement, and the pixel displacement is converted into the actual physical displacement; According to the calculated actual displacement, the motion state of the target vehicle is evaluated, and the calculated motion state data is smoothed and filtered to obtain the final motion state of the target vehicle.

Images