Lightning-vision fusion type traffic safety facility bus patrolling method and system oriented to low-signal scene

Abstract

The invention discloses a low-signal scene-oriented thunder-vision fusion type traffic safety facility bus patrolling method and system, and the method comprises the steps: firstly configuring a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU on an urban public service vehicle according to a patrolling task, and collecting corresponding data, thereby obtaining collected data; fusing the collected data by adopting an extended Kalman filter (EKF) algorithm to obtain fused data; according to the invention, a function of carrying out high-precision routing inspection on traffic safety facilities by adopting an urban public service vehicle routing inspection mode in a low-signal environment is realized, and by integrating a laser radar, a camera, a GNSS and an IMU sensor, the positioning precision of the traffic safety facilities can be improved by utilizing a multi-source data fusion technology under the condition that an RTK signal is weak or no signal exists; the reliability of the inspection result is ensured, the inspection efficiency is improved, and the method is suitable for being widely popularized and used.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety facility inspection, and in particular to a radar-visual fusion traffic safety facility public transportation inspection method and system for low-signal scenarios. Background Art

[0002] With the acceleration of urbanization, traffic safety facilities play an increasingly important role in ensuring road safety and smooth operation. These traffic safety facilities include traffic lights, signs, guardrails, pedestrian crossing facilities, etc.

[0003] At present, with the increase of traffic flow, it is particularly important to inspect and maintain traffic safety facilities in a timely and accurate manner. However, there are many challenges in the actual inspection process, especially in the inspection tasks under low signal environments. Most of the existing inspection methods for traffic safety facilities rely on manual inspections or single sensors for data collection, which leads to the fact that the real-time dynamic positioning RTK signal is often blocked in some areas with complex terrain or dense urban high-rise buildings, making the position data unstable and inaccurate. In this case, the existing GNSS positioning system generally cannot provide reliable data support, thus affecting the accuracy and efficiency of the inspection. The existing inspection system often relies on only a single type of sensor for data collection, resulting in a lack of multi-sensor data fusion capability, which limits the system's adaptability under different environmental conditions and makes it difficult to give full play to the advantages of various sensors. The existing manual inspection method is not only time-consuming but also easily affected by human factors, resulting in insufficient reliability and consistency of the inspection results. At the same time, manual inspection is difficult to achieve real-time monitoring and dynamic adjustment, and cannot meet the needs of modern urban traffic management. Therefore, it is necessary to design a public transportation inspection method and system for traffic safety facilities with radar-visual fusion in low signal scenarios. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and to better and more effectively solve the problem of accurate inspection and data collection of traffic safety facilities in areas with weak signals or no coverage. A method and system for public transportation inspection of traffic safety facilities in low-signal scenarios is provided. The method and system realize the function of high-precision inspection of traffic safety facilities by urban public service vehicles in low-signal environments. By integrating lidar, camera, GNSS and IMU sensors, the positioning accuracy of traffic safety facilities can be improved by using multi-source data fusion technology when the RTK signal is weak or there is no signal, which not only ensures the reliability of the inspection results, but also improves the inspection efficiency.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: A method for public transportation inspection of traffic safety facilities using radar and vision fusion in low signal scenarios, comprising the following steps: Step A: According to the inspection task, a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU are configured on the urban public service vehicle and corresponding data are collected respectively, so as to obtain collected data; Step B, using the extended Kalman filter algorithm EKF to fuse the collected data and obtain fused data; Step C, using high-gain real-time dynamic carrier phase differential antenna RTK and real-time compensation algorithm to correct the positioning error of the fused data and obtain the corrected data; Step D, based on the corrected data, the YOLO algorithm is used to perform target detection and feature extraction on the traffic safety facilities to obtain the category and location coordinates of the traffic safety facilities; In step E, the obtained traffic safety position coordinates are precisely located in combination with the position compensation signals in the fused data and the corrected data, and then the obtained traffic safety facility categories are feature analyzed and shape features are extracted for 3D modeling, so as to track changes in traffic safety facilities and complete the inspection of traffic safety facilities in low-signal scenarios.

[0006] The above-mentioned method for public transportation inspection of traffic safety facilities with radar and vision fusion in low-signal scenarios, step A, configures a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU on the urban public service vehicle according to the inspection task and collects corresponding data respectively, thereby obtaining collected data, wherein specifically, the laser radar is used to collect point cloud data , Camera acquires visual data And inertial measurement sensor IMU records attitude data , and the signal status of the global navigation satellite module GNSS is , so the signal status of the global navigation satellite module GNSS under low signal conditions , and the input of the collected data is , assuming the actual location of the traffic safety facility is , the state propagation model of the inertial measurement sensor IMU is shown in formula (1), (1) in, For speed, is the acceleration data, is the time step.

[0007] In the above-mentioned method for public transportation inspection of traffic safety facilities with radar-visual fusion in low-signal scenarios, step B uses the extended Kalman filter algorithm EKF to fuse the collected data and obtain the fused data. Specifically, the point cloud data , visual data and posture data Integrate into a unified state estimate and calculate the precise location of traffic safety facilities. The specific steps are as follows: Step B1, the process of using the extended Kalman filter algorithm EKF to perform prediction update, as shown in formula (2), , (2) in, is the fusion state estimate, is the state transfer function, To accurately locate the traffic safety facilities after integration, is the state transfer matrix, is the process noise covariance matrix; Step B2: Point cloud data and visual data Get the observed value and update the estimated state, as shown in formula (3), , (3) in, is the updated state estimate, Observations provided for LiDAR and vision data, is the Kalman gain, is the observation matrix, is the observation function, To update the precise location of traffic safety facilities.

[0008] In the above-mentioned method for patrolling public transportation with radar-visual fusion traffic safety facilities in low-signal scenarios, step C uses a high-gain real-time dynamic carrier phase differential antenna RTK and a real-time compensation algorithm to correct the positioning error of the fused data and obtain the corrected data. Specifically, the error is set , and then use the compensation algorithm to add the error into the prediction model for correction, as shown in formula (3), (4) in, For the corrected position information, After fusion, the position information is the error correction factor.

[0009] In the above-mentioned method for public transportation inspection of traffic safety facilities based on radar and vision fusion in low-signal scenarios, in step D, based on the corrected data, the YOLO algorithm is used to perform target detection and feature extraction on the traffic safety facilities to obtain the category and location coordinates of the traffic safety facilities. Specifically, the visual data is used to obtain the location coordinates of the traffic safety facilities. The visual features extracted from And from point cloud data The extracted point cloud features are set as , and then the visual features and point cloud features Combine to get comprehensive features , then the comprehensive features Input into the classifier to identify and classify traffic safety facilities, and different types of traffic safety facilities , and the classifier output is the traffic safety facility category and the corresponding location coordinates ,The specific steps of using YOLO algorithm to detect targets and extract features of traffic safety facilities are as follows: Step D1, data preparation, image enhancement, image scaling and image normalization are performed on the corrected data to convert it into a format suitable for YOLO algorithm model input; Step D2, model architecture, the YOLO algorithm model is composed of a feature extraction layer and a fully connected layer, the feature extraction layer is used to extract the shape features and color features of traffic safety facilities in the input image by using convolution and pooling operations, and the fully connected layer is used to map the extracted features to the target category prediction and bounding box coordinate prediction and output the category probability of the traffic safety facility and the bounding box coordinates of the traffic safety facility; Step D3, target monitoring, the YOLO algorithm model divides the entire image into grids, and each grid is responsible for predicting the traffic safety facilities in its area, as shown in formula (5): , (5) in, is the category probability of traffic safety facilities, are the bounding box coordinates of traffic safety facilities, and are the relative coordinates of the center of the bounding box, and is the width and height of the bounding box; Step D4, post-processing, filters out duplicate bounding boxes through the non-maximum suppression (NMS) algorithm and selects the box with the highest confidence as the final detection result. The non-maximum suppression (NMS) algorithm is shown in formula (6): (6) in, is the intersection-and-union ratio, Used to indicate the degree of overlap between two bounding boxes; Step D5, target monitoring, classification and marking of traffic safety facility locations.

[0010] The aforementioned method for patrolling public transportation of traffic safety facilities with radar-visual fusion in low-signal scenarios, step E, accurately locates the obtained traffic safety position coordinates combined with the position compensation signal in the fused data and the corrected data, and then performs feature analysis on the obtained traffic safety facility category and extracts shape features for 3D modeling, thereby tracking the changes of traffic safety facilities, and then completing the patrol operation of traffic safety facilities in low-signal scenarios, which is specifically based on the compensated position information Correction of position errors of traffic safety facilities , assuming the actual location of the traffic safety facility is , as shown in formula (7), (7) in, is the traffic safety facility offset estimated by vision and lidar.

[0011] A public transportation inspection system for traffic safety facilities with radar and vision fusion in low-signal scenarios includes a data acquisition module, a data fusion module, a data correction module, a target detection module and a positioning modeling module. The data acquisition module is used to configure a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU on a city public service vehicle according to the inspection task and collect corresponding data respectively, so as to obtain the collected data; the data fusion module is used to fuse the collected data using an extended Kalman filter algorithm EKF and obtain the fused data; the data correction module is used to use a high-gain real-time dynamic carrier phase to correct the fused data The differential antenna RTK and real-time compensation algorithm are used to correct the positioning error and obtain the corrected data; the target detection module is used to use the YOLO algorithm to perform target detection and feature extraction on the traffic safety facilities based on the corrected data and obtain the category and position coordinates of the traffic safety facilities; the positioning modeling module is used to accurately locate the obtained traffic safety position coordinates in combination with the position compensation signal in the fused data and the corrected data, and then perform feature analysis on the obtained traffic safety facility category and extract shape features for 3D modeling, so as to track the changes of traffic safety facilities, and then complete the inspection of traffic safety facilities in low-signal scenarios.

[0012] The beneficial effects of the present invention are as follows: a method and system for public transportation inspection of traffic safety facilities in low-signal scenarios with radar-visual fusion type of the present invention, firstly, according to the inspection task, a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU are configured on an urban public service vehicle and corresponding data are collected respectively to obtain collected data, then the collected data is fused by using an extended Kalman filter algorithm EKF to obtain fused data, then the fused data is corrected for positioning errors by using a high-gain real-time dynamic carrier phase differential antenna RTK and a real-time compensation algorithm to obtain corrected data, then based on the corrected data, a YOLO algorithm is used to perform target detection and feature extraction on traffic safety facilities and obtain the category and position coordinates of traffic safety facilities, then the obtained traffic safety position coordinates are combined with the position compensation signal in the fused data and the corrected data to accurately locate, then the obtained traffic safety facility category is feature analyzed and shape features are extracted for 3D modeling, thereby tracking changes in traffic safety facilities, and then completing the inspection of traffic safety facilities in low-signal scenarios; the present invention realizes a method for using urban public service vehicles for inspection in a low-signal environment. The invention has the function of high-precision inspection of traffic safety facilities, and by integrating laser radar, camera, GNSS and IMU sensors, it can use multi-source data fusion technology to improve the positioning accuracy of traffic safety facilities when the RTK signal is weak or no signal, which not only ensures the reliability of the inspection results, but also improves the inspection efficiency. The invention can also dynamically adjust the sensor parameters through real-time compensation algorithm and adaptive inspection strategy to optimize the data acquisition quality. This intelligent inspection process not only reduces the need for manual intervention, but also significantly improves the speed and efficiency of inspection. At the same time, the use of deep learning model for target detection can accurately identify traffic safety facilities, and the precise positioning of traffic safety facilities can be achieved through comprehensive analysis of the repaired signal data, which provides an important basis for subsequent maintenance and repair work. The invention uses urban public service vehicles as carriers to overcome the limitations of traditional inspection methods, improve the intelligence level and operation quality of traffic safety facility inspection, and provide more efficient and reliable technical support for urban traffic management. The use of data fusion algorithm can integrate visual and radar information to overcome their respective limitations, which not only provides a more comprehensive road condition assessment, but also improves the detection capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is an overall flow chart of a public transportation inspection method for radar-visual fusion traffic safety facilities in low-signal scenarios of the present invention. DETAILED DESCRIPTION

[0014] The present invention will be further described below in conjunction with the accompanying drawings.

[0015] like Figure 1As shown, a method for public transportation inspection of traffic safety facilities in a low-signal scenario using a radar-visual fusion type is provided in the present invention, comprising the following steps: Step A: According to the inspection task, a laser radar, camera, global navigation satellite module GNSS and inertial measurement sensor IMU are configured on the urban public service vehicle and the corresponding data are collected respectively to obtain the collected data, in which the laser radar is used to collect point cloud data. , Camera acquires visual data And inertial measurement sensor IMU records attitude data , and the signal status of the global navigation satellite module GNSS is , so the signal status of the global navigation satellite module GNSS under low signal conditions , and the input of the collected data is , assuming the actual location of the traffic safety facility is , the state propagation model of the inertial measurement sensor IMU is shown in formula (1), (1) in, For speed, is the acceleration data, is the time step.

[0016] Step B: Use the extended Kalman filter algorithm EKF to fuse the collected data and obtain the fused data. Specifically, the point cloud data , visual data and posture data Integrate into a unified state estimate and calculate the precise location of traffic safety facilities. The specific steps are as follows: Step B1, the process of using the extended Kalman filter algorithm EKF to perform prediction update, as shown in formula (2), , (2) in, is the fusion state estimate, is the state transfer function, To accurately locate the traffic safety facilities after integration, is the state transfer matrix, is the process noise covariance matrix; Step B2: Point cloud data and visual data Get the observed value and update the estimated state, as shown in formula (3), , (3) in, is the updated state estimate, Observations provided for LiDAR and vision data, is the Kalman gain, is the observation matrix, is the observation function, To update the precise location of traffic safety facilities.

[0017] Step C: Use high-gain real-time dynamic carrier phase differential antenna RTK and real-time compensation algorithm to correct the positioning error of the fused data and obtain the corrected data. Specifically, the error is set , and then use the compensation algorithm to add the error into the prediction model for correction, as shown in formula (3), (4) in, For the corrected position information, After fusion, the position information is the error correction factor.

[0018] Step D: Based on the corrected data, the YOLO algorithm is used to detect and extract features of traffic safety facilities to obtain the category and location coordinates of traffic safety facilities. Specifically, the visual data is used to extract the target and feature of traffic safety facilities. The visual features extracted from And from point cloud data The extracted point cloud features are set as , and then the visual features and point cloud features Combine to get comprehensive features , then the comprehensive features Input into the classifier to identify and classify traffic safety facilities, and different types of traffic safety facilities , and the classifier output is the traffic safety facility category and the corresponding location coordinates ,The specific steps of using YOLO algorithm to detect targets and extract features of traffic safety facilities are as follows: Step D1, data preparation, image enhancement, image scaling and image normalization are performed on the corrected data to convert it into a format suitable for YOLO algorithm model input; Step D2, model architecture, the YOLO algorithm model is composed of a feature extraction layer and a fully connected layer, the feature extraction layer is used to extract the shape features and color features of traffic safety facilities in the input image by using convolution and pooling operations, and the fully connected layer is used to map the extracted features to the target category prediction and bounding box coordinate prediction and output the category probability of the traffic safety facility and the bounding box coordinates of the traffic safety facility; Step D3, target monitoring, the YOLO algorithm model divides the entire image into grids, and each grid is responsible for predicting the traffic safety facilities in its area, as shown in formula (5): , (5) in, is the category probability of traffic safety facilities, are the bounding box coordinates of traffic safety facilities, and are the relative coordinates of the center of the bounding box, and is the width and height of the bounding box; Step D4, post-processing, filters out duplicate bounding boxes through the non-maximum suppression (NMS) algorithm and selects the box with the highest confidence as the final detection result. The non-maximum suppression (NMS) algorithm is shown in formula (6): (6) in, is the intersection-and-union ratio, Used to indicate the degree of overlap between two bounding boxes; Step D5, target monitoring, classification and marking of traffic safety facility locations.

[0019] Step E, accurately locate the obtained traffic safety location coordinates combined with the position compensation signal in the fused data and the corrected data, then perform feature analysis on the obtained traffic safety facility category and extract shape features for 3D modeling, so as to track the changes of traffic safety facilities, and then complete the inspection of traffic safety facilities in low-signal scenarios, which is specifically based on the compensated position information Correction of position errors of traffic safety facilities , assuming the actual location of the traffic safety facility is , as shown in formula (7), (7) in, is the traffic safety facility offset estimated by vision and lidar.

[0020] A radar-visual fusion type public transportation inspection system for traffic safety facilities in low-signal scenarios comprises a data acquisition module, a data fusion module, a data correction module, a target detection module and a positioning modeling module. The data acquisition module is used to configure a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU on a city public service vehicle according to the inspection task and respectively collect corresponding data, thereby obtaining the collected data; the data fusion module is used to fuse the collected data using an extended Kalman filter algorithm EKF and obtain fused data; the data correction module is used to correct the positioning error of the fused data using a high-gain real-time dynamic carrier phase differential antenna RTK and a real-time compensation algorithm and obtain corrected data; the target detection module is used to perform target detection and feature extraction on traffic safety facilities based on the corrected data using a YOLO algorithm and obtain the category and position coordinates of the traffic safety facilities; the positioning modeling module is used to accurately locate the obtained traffic safety position coordinates in combination with the position compensation signal in the fused data and the corrected data, and then perform feature analysis on the obtained traffic safety facility category and extract shape features for 3D modeling, thereby tracking the changes of the traffic safety facilities, and then completing the inspection of traffic safety facilities in low-signal scenarios.

[0021] In order to better illustrate the use effect of the present invention, a specific implementation example of the invention is introduced below as shown in Table 2. This embodiment compares three types of algorithms: Kalman filter algorithm (KF), particle filter algorithm (PF) and extended Kalman filter algorithm (EKF algorithm). As shown in Table 1, the experimental data set of this embodiment includes test data in three scenarios: road inspection data in a low-signal environment, inspection data in a normal signal environment, and data under the influence of dynamic obstacles. There are 900 data sets in this embodiment, including 400 low-signal data, 300 normal signal data and 200 obstacle interference data. Among them, 600 are used for training and 300 are used for testing. During the training process, the tanh activation function is used, and the mini-batch gradient descent method (Mini-batch Gradient Descent) is used, the batch size is 64, and 1000 epochs are trained.

[0022] Table 1 Algorithm parameter settings parameter Numeric Learning rate (Eta) 0.001 Activation Function tanh Batch size 64 Epochs 1000 Number of data samples (training set) 600 Number of data samples (test set) 300 Training Algorithms Mini-batch Gradient Descent (Mini-batch GD) Kalman filter parameters (Q, R) Q=0.1, R=0.3 Particle filter particle number 500 Table 2 Algorithm experimental results Scene Type algorithm Positioning accuracy (meters) Processing time (seconds) System stability (standard deviation) Low signal environment Kalman Filter (KF) 1.95 0.45 0.12 Particle Filter (PF) 2.10 0.38 0.15 Extended Kalman Filter (EKF) 1.42 0.32 0.08 Normal signal environment Kalman Filter (KF) 1.08 0.36 0.10 Particle Filter (PF) 1.12 0.33 0.12 Extended Kalman Filter (EKF) 0.95 0.28 0.05 Dynamic obstacles interfere with the environment Kalman Filter (KF) 2.30 0.47 0.14 Particle Filter (PF) 2.50 0.42 0.17 Extended Kalman Filter (EKF) 1.85 0.36 0.09 As can be seen from Table 2, in a low-signal environment, the extended Kalman filter algorithm EKF used in the present invention can significantly outperform the Kalman filter algorithm and the particle filter algorithm in terms of positioning accuracy and system stability for the detection and positioning requirements of traffic safety facilities in low-signal scenarios. Specifically, the EKF algorithm improves the positioning accuracy by 12.3%, and reduces the processing time by about 10% compared to other algorithms. At the same time, in terms of system stability, the EKF algorithm exhibits a lower standard deviation, indicating that it is more reliable in a dynamic signal environment. Therefore, this algorithm can accurately identify and correct positioning errors caused by signal loss or interference, so that in a complex and dynamic low-signal environment, it can still maintain high-precision positioning and status assessment of traffic facilities. When the environmental signal changes or interferes, the system can compensate and adjust in real time to ensure the accuracy and reliability of the data.

[0023] The present invention can effectively convert the collected raw data from the low-signal space to the precise positioning space through real-time signal compensation and dynamic Kalman filtering technology, thereby making the inspection of traffic safety facilities in low-signal environments more efficient and accurate. With the continuous accumulation of traffic facility inspection data, the system can further improve its detection performance by continuously optimizing the Kalman filter parameters to adapt to more types of complex road conditions and environments with different signal strengths. In addition, with the integration of more historical data, the self-learning ability of the algorithm is enhanced, and it can gradually optimize and adapt to more dynamic and complex practical application scenarios.

[0024] In summary, the present invention provides a method and system for public transportation inspection of traffic safety facilities in low-signal scenarios with radar-visual fusion. First, according to the inspection task, a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU are configured on an urban public service vehicle and corresponding data are collected respectively to obtain collected data. The collected data is then fused using an extended Kalman filter algorithm EKF to obtain fused data. Then, the fused data is corrected for positioning errors using a high-gain real-time dynamic carrier phase differential antenna RTK and a real-time compensation algorithm to obtain corrected data. Then, based on the corrected data, the YOLO algorithm is used to perform target detection and feature extraction on the traffic safety facilities and obtain the category and position coordinates of the traffic safety facilities. Subsequently, the obtained traffic safety position coordinates are precisely located in combination with the position compensation signals in the fused data and the corrected data. Then, the obtained traffic safety facility category is feature analyzed and shape features are extracted for 3D modeling, thereby tracking changes in traffic safety facilities, and then completing the inspection of traffic safety facilities in low-signal scenarios. The present invention realizes the use of urban public service vehicles for inspection in a low-signal environment. The invention has the function of high-precision inspection of traffic safety facilities, and by integrating laser radar, camera, GNSS and IMU sensors, it can use multi-source data fusion technology to improve the positioning accuracy of traffic safety facilities when the RTK signal is weak or no signal, which not only ensures the reliability of the inspection results, but also improves the inspection efficiency. The invention can also dynamically adjust the sensor parameters through real-time compensation algorithm and adaptive inspection strategy to optimize the data acquisition quality. This intelligent inspection process not only reduces the need for manual intervention, but also significantly improves the speed and efficiency of inspection. At the same time, the use of deep learning model for target detection can accurately identify traffic safety facilities, and through comprehensive analysis of the repaired signal data, it can achieve accurate positioning of traffic safety facilities, which provides an important basis for subsequent maintenance and repair work. The invention uses urban public service vehicles as carriers to overcome the limitations of traditional inspection methods, improve the intelligence level and operation quality of traffic safety facility inspections, and provide more efficient and reliable technical support for urban traffic management. The use of data fusion algorithm can integrate visual and radar information to overcome their respective limitations, which not only provides a more comprehensive road condition assessment, but also improves the detection capability.

[0025] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for public transportation inspection of traffic safety facilities using radar and visual fusion in low signal scenarios, characterized by: The following steps are included: Step A: According to the inspection task, a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU are configured on the urban public service vehicle and corresponding data are collected respectively, so as to obtain collected data; Step B, using the extended Kalman filter algorithm EKF to fuse the collected data and obtain fused data; Step C, using high-gain real-time dynamic carrier phase differential antenna RTK and real-time compensation algorithm to correct the positioning error of the fused data and obtain the corrected data; Step D, based on the corrected data, the YOLO algorithm is used to perform target detection and feature extraction on the traffic safety facilities to obtain the category and location coordinates of the traffic safety facilities; In step E, the obtained traffic safety position coordinates are precisely located in combination with the position compensation signals in the fused data and the corrected data, and then the obtained traffic safety facility categories are feature analyzed and shape features are extracted for 3D modeling, so as to track changes in traffic safety facilities and complete the inspection of traffic safety facilities in low-signal scenarios.

2. According to claim 1, a method for public transportation inspection of traffic safety facilities using radar and visual fusion in low-signal scenarios, characterized in that: Step A: According to the inspection task, a laser radar, camera, global navigation satellite module GNSS and inertial measurement sensor IMU are configured on the urban public service vehicle and the corresponding data are collected respectively to obtain the collected data, in which the laser radar is used to collect point cloud data. , Camera acquires visual data And inertial measurement sensor IMU records attitude data , and the signal status of the global navigation satellite module GNSS is , so the signal status of the global navigation satellite module GNSS under low signal conditions , and the input of the collected data is , assuming the actual location of the traffic safety facility is , the state propagation model of the inertial measurement sensor IMU is shown in formula (1), （1） in, For speed, is the acceleration data, is the time step.

3. According to claim 2, a method for public transportation inspection of traffic safety facilities using radar and visual fusion in low-signal scenarios is characterized by: Step B: Use the extended Kalman filter algorithm EKF to fuse the collected data and obtain the fused data. Specifically, the point cloud data , visual data and posture data Integrate into a unified state estimate and calculate the precise location of traffic safety facilities. The specific steps are as follows: Step B1, the process of using the extended Kalman filter algorithm EKF to perform prediction update, as shown in formula (2), ， （2） in, is the fusion state estimate, is the state transfer function, To accurately locate the traffic safety facilities after integration, is the state transfer matrix, is the process noise covariance matrix; Step B2, through point cloud data and visual data Get the observed value and update the estimated state, as shown in formula (3), ， （3） in, is the updated state estimate, Observations provided for LiDAR and vision data, is the Kalman gain, is the observation matrix, is the observation function, To update the precise location of traffic safety facilities.

4. According to claim 3, a method for public transportation inspection of traffic safety facilities using radar and visual fusion in low-signal scenarios is characterized by: Step C: Use high-gain real-time dynamic carrier phase differential antenna RTK and real-time compensation algorithm to correct the positioning error of the fused data and obtain the corrected data. Specifically, the error is set , and then use the compensation algorithm to add the error to the prediction model for correction, as shown in formula (3), （4） in, For the corrected position information, After fusion, the position information is the error correction factor.

5. According to claim 4, a method for public transportation inspection of traffic safety facilities using radar and visual fusion in low-signal scenarios is characterized by: Step D: Based on the corrected data, the YOLO algorithm is used to detect and extract features of traffic safety facilities to obtain the category and location coordinates of traffic safety facilities. Specifically, the visual data is used to extract the target and feature of traffic safety facilities. The visual features extracted from And from point cloud data The extracted point cloud features are set as , and then the visual features and point cloud features Combine to get comprehensive features , then the comprehensive features Input into the classifier to identify and classify traffic safety facilities, and different types of traffic safety facilities , and the classifier output is the traffic safety facility category and the corresponding location coordinates ,The specific steps of using YOLO algorithm to detect targets and extract features of traffic safety facilities are as follows: Step D1, data preparation, image enhancement, image scaling and image normalization are performed on the corrected data to convert it into a format suitable for YOLO algorithm model input; Step D2, model architecture, the YOLO algorithm model is composed of a feature extraction layer and a fully connected layer, the feature extraction layer is used to extract the shape features and color features of traffic safety facilities in the input image by using convolution and pooling operations, and the fully connected layer is used to map the extracted features to the target category prediction and bounding box coordinate prediction and output the category probability of the traffic safety facility and the bounding box coordinates of the traffic safety facility; Step D3, target monitoring, the YOLO algorithm model divides the entire image into grids, and each grid is responsible for predicting the traffic safety facilities in its area, as shown in formula (5): ， （5） in, is the category probability of traffic safety facilities, are the bounding box coordinates of traffic safety facilities, and are the relative coordinates of the center of the bounding box, and is the width and height of the bounding box; Step D4, post-processing, filters out duplicate bounding boxes through the non-maximum suppression (NMS) algorithm and selects the box with the highest confidence as the final detection result. The non-maximum suppression (NMS) algorithm is shown in formula (6): （6） in, is the intersection-and-union ratio, Used to indicate the degree of overlap between two bounding boxes; Step D5, target monitoring, classification and marking of traffic safety facility locations.

6. According to claim 5, a method for public transportation inspection of traffic safety facilities using radar and visual fusion in low-signal scenarios is characterized by: Step E, accurately locate the obtained traffic safety location coordinates combined with the position compensation signal in the fused data and the corrected data, then perform feature analysis on the obtained traffic safety facility category and extract shape features for 3D modeling, so as to track the changes of traffic safety facilities, and then complete the inspection of traffic safety facilities in low-signal scenarios, which is specifically based on the compensated position information Correction of position errors of traffic safety facilities , assuming the actual location of the traffic safety facility is , as shown in formula (7), （7） in, is the traffic safety facility offset estimated by vision and lidar.

7. A radar-visual fusion traffic safety facility public transportation inspection system for low signal scenarios, wherein the specific inspection process of the traffic safety facility public transportation inspection system is based on the traffic safety facility public transportation inspection method according to any one of claims 1 to 6, characterized in that: It includes a data acquisition module, a data fusion module, a data correction module, a target detection module and a positioning modeling module. The data acquisition module is used to configure a laser radar, a camera, a global navigation satellite module GNSS and an inertial measurement sensor IMU on an urban public service vehicle according to the inspection task and collect corresponding data respectively, so as to obtain collected data; The data fusion module is used to fuse the collected data using the extended Kalman filter algorithm EKF and obtain fused data; The data correction module is used to correct the positioning error of the fused data using a high-gain real-time dynamic carrier phase differential antenna RTK and a real-time compensation algorithm to obtain the corrected data; The target detection module is used to perform target detection and feature extraction on traffic safety facilities using the YOLO algorithm based on the corrected data and obtain the category and location coordinates of the traffic safety facilities; The positioning modeling module is used to accurately locate the obtained traffic safety position coordinates in combination with the position compensation signal in the fused data and the corrected data, and then perform feature analysis on the obtained traffic safety facility categories and extract shape features for 3D modeling, so as to track the changes in traffic safety facilities and complete the inspection of traffic safety facilities in low-signal scenarios.

Images