Trolley inspection method based on thunder-vision fusion
By using the fusion method of radar and vision sensors and the data processing of Kalman filters in the small robot inspection system, the problem of inaccurate and failure of sensors in harsh environments is solved, and an efficient and reliable inspection task is achieved.
Patent Information
- Application Number
- CN202411970153.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-03-14
AI Technical Summary
The existing small robot patrol system is inaccurate in the measurement of dust, rain, snow or insufficient light, and cannot automatically switch when the sensor fails or the accuracy decreases, resulting in interruption or failure of the inspection task, affecting teaching efficiency and reliability.
The fusion method based on radar and vision sensor is adopted, and the data of lidar and vision sensors are fused through a Kalman filter, and automatically switch to another normally working sensor for data acquisition when either sensor fails or the accuracy decreases.
It significantly improves the robustness and accuracy of inspections, ensures adaptability in different environments, avoids interruptions in inspection tasks caused by sensor failure, and improves the reliability and practicality of the system.
Smart Images
Figure CN119620098A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of automatic navigation technology, and in particular to a vehicle inspection method based on radar and vision fusion. Background Art
[0002] In the field of modern industrial automation and intelligent robots, inspection systems are widely used in multiple actual production scenarios such as intelligent manufacturing, unmanned warehousing, energy equipment inspection, etc. Although using a single sensor, such as Lidar or Vision Sensor, they have high accuracy and search range, not to mention using multiple sensors, which can fully adapt to complex scenarios intelligently.
[0003] However, various small robot devices for education and teaching are now available on the market. Due to the influence of non-real industry applications such as teaching objectives, cost budget, technology maturity, and ease of use, a large number of traditional single sensors with high errors, instability, and low precision are used. In the teaching based on simulated sandbox maps, various training and experiments such as inspection, obstacle avoidance, reversing into the warehouse, and sign recognition often face many challenges. For example, laser radar may cause inaccurate measurements in dust, rain, snow, or insufficient light, and visual cameras may also make recognition errors when the light changes or is blocked.
[0004] When the sensor fails or its accuracy decreases, it cannot switch automatically, resulting in interruption or failure of the inspection task, affecting teaching efficiency and reliability, and seriously causing damage to the equipment. Summary of the invention
[0005] In view of the above problems, the present invention proposes a vehicle inspection method based on radar and vision fusion, which is mainly used in robot experiments for college students. By realizing the fusion and switching of radar and vision sensors on the vehicle, the fusion switching capability of different functional sensors under low-precision sensors is ensured, the smooth progress of robot inspection tasks is achieved, and the adaptive capability in different environments is enhanced by dynamically adjusting the sensor strategy.
[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is: a vehicle inspection method based on radar and visual fusion, comprising:
[0007] The car is equipped with a laser radar and a visual sensor, and the data from the laser radar and the visual sensor are fused and processed through a Kalman filter;
[0008] When any sensor fails or its accuracy decreases, it automatically switches to another normally working sensor for data collection.
[0009] As a possible implementation, further, the Kalman filter combines historical data and current measurement values to output a state estimation prediction value, which specifically includes the following steps:
[0010] 1): First, perform state prediction, and predict the value of the current state based on the state estimation and state transfer matrix of the previous moment. The expression is as follows:
[0011] x ^ _k=F×x ^ _(k-1)+B×u
[0012] In the above formula, x ^ _k represents the current state estimate, F represents the state transfer matrix, x ^ _(k-1) represents the state estimation at the previous moment, B represents the control input matrix, and u represents the control input;
[0013] 2): Update the prediction error covariance matrix, the expression is as follows:
[0014] P_k=F×P_(k-1)×F^T+Q
[0015] In the above formula, P_k represents the prediction error covariance matrix at the current moment, P_(k-1) represents the prediction error covariance matrix at the previous moment, T represents the transpose of the matrix, and Q represents the process noise covariance matrix;
[0016] 3) Based on the prediction error covariance and the measurement noise covariance, the Kalman gain is calculated, and the expression is as follows:
[0017] K_k=P_k×H^T×(H×P_k×H^T+R)^-1
[0018] In the above formula, K_k represents the Kalman gain, H represents the observation matrix, and R represents the measurement noise covariance matrix;
[0019] 4) Use the data from the lidar and visual sensors to update the state estimate, as shown below:
[0020] x ^ _k=x ^ _k+K_k×(z_k-H×x ^ _k)
[0021] In the above formula, z_k represents the current observation value; (x on the right side of the equal sign ^ _k is the x obtained in step 1) ^ _k)
[0022] 5) Update the error covariance matrix
[0023] P_k=(I-K_k×H)×P_k
[0024] In the above formula, I represents the unit matrix; (the P_k on the right side of the equal sign is the P_k obtained in step 2))
[0025] 6) Output state estimation prediction value, which is given by H×x ^ _k is calculated.
[0026] As a possible implementation, further, when any sensor fails or the accuracy decreases, it automatically switches to another normally working sensor for data collection, which specifically includes the following steps:
[0027] Monitor the output data of the lidar and vision sensor. When the error between the measured value of any sensor and the predicted value of the Kalman filter exceeds the set threshold, it automatically switches to another normally working sensor for data collection.
[0028] As a possible implementation, further, after the sensor is switched, the status of the switched sensor is continuously evaluated, and when it returns to normal, it is reintegrated into the data fusion process.
[0029] As a possible implementation, further, if the error between the measured value of the lidar and the predicted value of the Kalman filter exceeds 5 cm or ±5 degrees, stop using the lidar data and immediately switch to the visual sensor for data collection and processing.
[0030] As a possible implementation, further, if the image clarity of the visual sensor is lower than the contrast 0.3 or the brightness 50, or the number of feature matches is lower than 30, stop using the data of the visual sensor and immediately switch to the lidar for data collection and processing.
[0031] As a possible implementation, further, the laser radar application algorithm includes: a point cloud processing algorithm, a SLAM algorithm and an obstacle detection algorithm;
[0032] The visual sensor application algorithm includes: feature extraction and matching algorithm, target recognition algorithm and image processing algorithm.
[0033] As a possible implementation, further, the point cloud processing algorithm uses a Voxel Grid filter to downsample the point cloud data by dividing the point cloud into grids and representing the grid with the average value of the points in each grid; at the same time, a RANSAC algorithm is used to identify and extract specific geometric features;
[0034] The SLAM algorithm specifically adopts the Gmapping algorithm, which combines the laser radar data with the motion information of the car and uses the particle filtering method to evaluate and update the position information of the car;
[0035] The obstacle detection algorithm uses the point cloud data of the laser radar to identify obstacles in the environment based on geometric features; the algorithm extracts potential obstacle features by analyzing the distribution of points in the point cloud.
[0036] As a possible implementation, further, the feature extraction and matching algorithm uses the ORB algorithm to extract key points in the image, and calculates a descriptor for each feature point. The extracted key points are matched through the descriptors, and the Hamming distance is used as a similarity metric to find similar areas in the image;
[0037] The target recognition algorithm performs target detection and recognition through the YOLO model;
[0038] The image processing algorithm adopts adaptive threshold processing and edge detection algorithm; the adaptive threshold processing dynamically adjusts the threshold by calculating the local mean of the neighborhood around each pixel; the edge detection algorithm uses the Canny edge detection method to identify important contours and edge information in the image.
[0039] The present invention also provides a computer-readable storage medium, in which at least one instruction, at least one program, code set or instruction set is stored. The at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned vehicle inspection method based on radar and vision fusion.
[0040] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:
[0041] 1) The present invention can efficiently integrate the fusion capabilities of LiDAR and visual sensors, significantly improving the robustness and accuracy of inspections. By using a Kalman filter for data fusion, it is possible to perform state estimation and prediction on the measurements of LiDAR and visual sensors, thereby generating a high-precision environmental model, and has the ability to update in real time to dynamically adjust parameters and optimize the fusion effect.
[0042] 2) The present invention also has an intelligent switching mechanism, which can quickly switch to another sensor when a sensor fails to ensure the continuity of the inspection task. This mechanism improves the reliability and practicality of the trolley system by setting an error threshold and monitoring the working status of the sensor in real time. At the same time, the status of the switched sensor can be continuously evaluated through dynamic adjustment, and it can be reintegrated into the data fusion process when it returns to normal, ensuring the adaptability of the trolley system in a changing environment.
[0043] 3) The present invention not only improves the technical performance of the trolley inspection system, but also provides stronger adaptability and reliability for the application of teaching experiments, and has significant market application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 It is a schematic diagram of the working process of the Kalman filter;
[0046] Figure 2 It is a schematic diagram of the physical experimental vehicle;
[0047] Figure 3 It is a schematic diagram of the traffic sand table map;
[0048] Figure 4 It is a schematic diagram of the lidar related algorithm;
[0049] Figure 5 It is a schematic diagram of the algorithm related to the visual sensor;
[0050] Figure 6 It is a brief schematic diagram of the process of the present invention. DETAILED DESCRIPTION
[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.
[0052] This embodiment provides a vehicle inspection method based on radar and visual fusion, including:
[0053] 1. The car is equipped with a laser radar and a visual sensor, and the data of the laser radar and the visual sensor are fused and processed through the Kalman filter. In this embodiment, the car is equipped with NVIDIA Jetson Nano, which is responsible for coordinating the data processing and task execution of each sensor. At the same time, EAI X2L laser radar and visual sensor (USB camera) are integrated to achieve diversified data collection and support accurate navigation and positioning, such as Figure 2 shown.
[0054] After the lidar and vision sensors acquire new data, the Kalman filter will combine historical data and current measurements to output state estimation predictions. This process involves dynamic updates of state variables and adjustments to the error covariance matrix, thereby improving the accuracy and reliability of data fusion and generating an accurate environmental model. Figure 1 As shown, the specific steps include:
[0055] Step 1): When processing the data from the lidar and visual sensor, the Kalman filter first performs state prediction, predicting the value of the current state based on the state estimate and state transfer matrix at the previous moment; this process takes into account the control input of the system and makes a preliminary estimate of the current state; the expression is as follows:
[0056] x ^ _k=F×x ^ _(k-1)+B×u
[0057] Step 2): Next, update the prediction error covariance matrix; this step is based on the error covariance and process noise covariance of the previous moment to ensure a more accurate reflection of system uncertainty; the expression is as follows:
[0058] P_k=F×P_(k-1)×F^T+Q
[0059] Step 3): Then, the Kalman gain is calculated based on the prediction error covariance and the measurement noise covariance; this gain determines how to weight the measurement results of each sensor when updating the state to better fuse the information; the expression is as follows:
[0060] K_k=P_k×H^T×(H×P_k×H^T+R)^-1
[0061] Step 4): On this basis, the data from the lidar and visual sensors are used to update the state estimate; this step ensures that the new measurement information is effectively integrated, thereby improving the accuracy of the state estimate, as shown below:
[0062] x ^ _k=x ^ _k+K_k×(z_k-H×x ^ _k)
[0063] Among them, x on the right side of the equal sign ^ _k is the x obtained in step 1) ^ _k.
[0064] Step 5): Update the error covariance matrix to reflect the uncertainty of the current state estimate. This update ensures that the system remains efficient and stable in a dynamic environment; the expression is as follows:
[0065] P_k=(I-K_k×H)×P_k
[0066] Among them, P_k on the right side of the equal sign is P_k obtained in step 2).
[0067] Step 6): Output the state estimation prediction value, which is given by H×x ^ _k is calculated.
[0068] In the above formulas, the definition of each parameter is: ^ _k represents the current state estimate, x ^ (k-1) represents the state estimate at the previous moment; F represents the state transfer matrix, B represents the control input matrix, and u represents the control input; P_k represents the prediction error covariance matrix at the current moment, P_(k-1) represents the prediction error covariance matrix at the previous moment, Q represents the process noise covariance matrix, and T represents the transpose of the matrix; K_k represents the Kalman gain, H represents the observation matrix, R represents the measurement noise covariance matrix, and z_k represents the current observation value; I represents the unit matrix.
[0069] The above-mentioned radar-visual fusion method not only improves the measurement accuracy of the sensor, but also effectively reduces the noise and error that may be generated by a single sensor. At the same time, the Kalman filter has the ability to update in real time. Every time the sensor obtains new data, the filter can dynamically adjust its internal parameters according to the difference between the current measurement value and the predicted value, making the fusion result more accurate. This adaptive characteristic enables the system to maintain an efficient working state in a complex or changing environment, enhancing the adaptability and robustness of the system.
[0070] 2. When any sensor fails or its accuracy decreases, it will automatically switch to another normally working sensor for data collection, which includes the following steps:
[0071] By setting the error threshold and monitoring the output data of the LiDAR and vision sensors, when the error between the measured value of any sensor and the predicted value of the Kalman filter exceeds the set threshold, the intelligent switching mechanism is quickly activated to automatically switch to another normally working sensor for data collection to ensure the continuity of the inspection task. At the same time, after the sensor is switched, the status of the switched sensor is continuously evaluated and it is reintegrated into the data fusion process when it returns to normal.
[0072] Specifically, the failure of LiDAR usually occurs in complex environments. When the LiDAR is blocked or interfered by reflections, it may cause abnormal measurements. For example, if the LiDAR cannot normally identify surrounding obstacles, or if the data within the measurement range fluctuates greatly. Therefore, in this embodiment, if the error between the LiDAR measurement value and the Kalman filter prediction value exceeds 5 cm or ±5 degrees, the car system will stop using the LiDAR data and rely on the visual sensor for data collection and processing.
[0073] On the other hand, visual sensor failure may occur in situations of insufficient light, rapid movement, or complex backgrounds. For example, when the ambient light changes dramatically (such as strong light exposure or sudden entry into a dark area), the image quality may degrade or data may be lost. Therefore, in this embodiment, if the image clarity of the visual sensor is lower than the contrast ratio of 0.3 or the brightness of 50, or the number of feature matches is lower than 30, the car system will stop using the data of the visual sensor and immediately switch to the lidar for data collection and processing.
[0074] In addition, the adjustment mechanism in this embodiment not only continuously monitors the switched sensor after the sensor is switched, but also reintegrates it into the data fusion process in real time based on its recovery status. When the original sensor returns to normal, its performance will be re-evaluated and incorporated into the data fusion process. This mechanism ensures that the trolley system always uses the best available sensor for inspection.
[0075] In this embodiment, refer to the attached Figure 4 As shown, the LiDAR application algorithms include:
[0076] a. Point cloud processing algorithm: mainly used for filtering and feature extraction of three-dimensional point cloud data collected by LiDAR. This algorithm uses Voxel Grid filter to downsample point cloud data to reduce data volume and computational burden while maintaining the structural characteristics of the environment. This process reduces data complexity by dividing the point cloud into grids (Voxel) and representing the grid with the average value of the points in each grid. At the same time, the RANSAC algorithm (random sampling consensus algorithm) is used to identify and extract specific geometric features, such as planes, cylinders, etc. These features are crucial for subsequent environmental understanding and obstacle recognition.
[0077] b. SLAM algorithm: The core of the SLAM (Simultaneous Localization and Mapping) algorithm is to build an environmental map in real time and locate the position of the car at the same time. This implementation specifically adopts the Gmapping algorithm, which is a SLAM implementation based on particle filtering. The Gmapping algorithm combines the lidar data with the motion information of the car, and uses the particle filtering method to evaluate and update the position information of the car. This method not only improves the accuracy of map construction, but also can adapt quickly in dynamic environments. By continuously updating the position of particles, the car system can maintain efficient navigation and positioning capabilities in complex environments.
[0078] c. Obstacle detection algorithm: Using the point cloud data of the LiDAR, obstacles in the environment are identified based on geometric features. The algorithm extracts potential obstacle features, such as height, width, and shape, by analyzing the distribution of points in the point cloud. These geometric features help the system determine which areas are safe and which areas have obstacles, thereby ensuring the safety of the inspection path.
[0079] In this embodiment, refer to the attached Figure 5 As shown, the vision sensor application algorithm includes:
[0080] a. Feature extraction and matching algorithm: This algorithm is a key step in the visual sensor to process image data. In this embodiment, the ORB (Oriented FAST and Rotated BRIEF) algorithm is used to extract key points in the image. The ORB algorithm has rotation invariance and scale invariance, can efficiently identify feature points in the image, and calculate descriptors for each feature point. The extracted key points are then matched by descriptors, using the Hamming distance as a similarity metric to find similar areas in the image.
[0081] b. Target recognition algorithm: In this embodiment, the YOLO model is used for target detection. This model can simultaneously complete bounding box regression and classification through a single forward pass, with high speed and accuracy. Through the training data set, YOLO can learn the characteristics of different objects and adapt to a variety of complex scenes, thereby providing necessary visual feedback during the inspection process.
[0082] c. Image processing algorithm: In this embodiment, adaptive threshold processing and edge detection algorithms are used. Adaptive threshold processing dynamically adjusts the threshold by calculating the local mean of the neighborhood around each pixel, thereby effectively coping with lighting changes and complex backgrounds. Canny edge detection is used to identify important contours and edge information in the image, improving the effectiveness of feature extraction.
[0083] In summary, refer to the attached Figure 6As shown, the trolley inspection method based on radar and vision fusion provided by the present invention is based on a trolley platform and is equipped with a laser radar and a visual sensor. Through the close cooperation of various components, it can move autonomously in a preset complex environment to achieve efficient inspection tasks.
[0084] Among them, the laser radar emits laser beams by continuously rotating, measures the distance to surrounding objects, and generates high-density point cloud data. These point cloud data are filtered and feature extracted by point cloud processing algorithms (including Voxel Grid filter and RANSAC algorithm) to eliminate noise and redundant information, ensuring that the generated environmental map is clear and reliable. At the same time, the SLAM algorithm (using the gmapping algorithm) combines these point cloud data with the movement status of the car to achieve real-time map construction and positioning, allowing the car to accurately navigate in complex environments. In addition, the obstacle detection algorithm analyzes the data obtained by the laser radar in real time, identifies potential obstacles, and avoids collisions between the car and them, thereby ensuring the safety of the inspection process.
[0085] Visual sensors also play an important role in the inspection process. They capture environmental images and use feature extraction and matching algorithms (using the ORB algorithm) to extract key points and match them to identify specific objects or environmental features. This information can supplement the data from the lidar and improve the understanding of the environment. When the visual sensor identifies a specific target, the target recognition algorithm YOLO model will further analyze it to obtain richer environmental information. In addition, image processing algorithms (including adaptive threshold processing and edge detection) process the collected images to improve image quality and ensure the accuracy of subsequent feature extraction.
[0086] In the data fusion stage, the data from the lidar and visual sensor are received by the Kalman filter module for effective fusion processing. Through the weighted average method, the Kalman filter can update the environmental status in real time, improving the accuracy and robustness of navigation.
[0087] When the error between the measured value of any sensor and the predicted value of the Kalman filter exceeds the set threshold, the system will quickly activate the intelligent switching mechanism and automatically switch to another normally working sensor for data collection. At the same time, after the sensor is switched, the status of the switched sensor will be continuously evaluated, and it will be reintegrated into the data fusion process when it returns to normal. The present invention can also dynamically adjust the parameters of the Kalman filter based on historical data to optimize the fusion effect. Through this series of orderly operations, the present invention can realize an efficient and reliable inspection process in a complex environment.
[0088] In addition, each functional unit in the embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0090] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A vehicle inspection method based on radar and visual fusion, characterized in that: include: The car is equipped with a laser radar and a visual sensor, and the data from the laser radar and the visual sensor are fused and processed through a Kalman filter; When any sensor fails or its accuracy decreases, it automatically switches to another normally working sensor for data collection.
2. The vehicle inspection method based on radar and visual fusion according to claim 1 is characterized in that: The Kalman filter combines historical data and current measurement values to output a state estimation prediction value, which specifically includes the following steps: 1): First, perform state prediction, and predict the value of the current state based on the state estimation and state transfer matrix of the previous moment. The expression is as follows: x ^ _k=F×x ^ _(k-1)+B×u In the above formula, x ^ _k represents the current state estimate, F represents the state transfer matrix, x ^ _(k-1) represents the state estimation at the previous moment, B represents the control input matrix, and u represents the control input; 2): Update the prediction error covariance matrix, the expression is as follows: P_k=F×P_(k-1)×F^T+Q In the above formula, P_k represents the prediction error covariance matrix at the current moment, P_(k-1) represents the prediction error covariance matrix at the previous moment, T represents the transpose of the matrix, and Q represents the process noise covariance matrix; 3) Based on the prediction error covariance and the measurement noise covariance, the Kalman gain is calculated, and the expression is as follows: K_k=P_k×H^T×(H×P_k×H^T+R)^-1 In the above formula, K_k represents the Kalman gain, H represents the observation matrix, and R represents the measurement noise covariance matrix; 4) Use the data from the lidar and visual sensors to update the state estimate, as shown below: x ^ _k=x ^ _k+K_k×(z_k-H×x ^ _k) In the above formula, z_k represents the current observation value; 5) Update the error covariance matrix P_k=(I-K_k×H)×P_k In the above formula, I represents the unit matrix; 6) Output state estimation prediction value, which is given by H×x ^ _k is calculated.
3. The vehicle inspection method based on radar and visual fusion according to claim 1 is characterized in that: When any sensor fails or its accuracy decreases, it automatically switches to another normally working sensor for data collection, which specifically includes the following steps: Monitor the output data of the lidar and vision sensor. When the error between the measured value of any sensor and the predicted value of the Kalman filter exceeds the set threshold, it automatically switches to another normally working sensor for data collection.
4. The vehicle inspection method based on radar and visual fusion according to claim 3 is characterized in that: The status of the switched sensor is continuously evaluated after the sensor is switched and reincorporated into the data fusion process when it returns to normal.
5. The vehicle inspection method based on radar and visual fusion according to claim 4 is characterized in that: If the error between the lidar measurement and the Kalman filter prediction exceeds 5 cm or ±5 degrees, stop using the lidar data and immediately switch to the visual sensor for data collection and processing.
6. The vehicle inspection method based on radar and visual fusion according to claim 4 is characterized in that: If the image clarity of the visual sensor is lower than the contrast ratio of 0.3 or the brightness of 50, or the number of feature matches is lower than 30, stop using the data of the visual sensor and immediately switch to the lidar for data collection and processing.
7. The vehicle inspection method based on radar and visual fusion according to claim 1 is characterized in that: The laser radar application algorithm includes: point cloud processing algorithm, SLAM algorithm and obstacle detection algorithm; The visual sensor application algorithm includes: feature extraction and matching algorithm, target recognition algorithm and image processing algorithm.
8. The vehicle inspection method based on radar and visual fusion according to claim 7 is characterized in that: The point cloud processing algorithm uses a Voxel Grid filter to downsample the point cloud data by dividing the point cloud into grids and representing the grid with the average value of the points in each grid; at the same time, the RANSAC algorithm is used to identify and extract specific geometric features; The SLAM algorithm specifically adopts the Gmapping algorithm, which combines the laser radar data with the motion information of the car and uses the particle filtering method to evaluate and update the position information of the car; The obstacle detection algorithm uses the point cloud data of the laser radar to identify obstacles in the environment based on geometric features; the algorithm extracts potential obstacle features by analyzing the distribution of points in the point cloud.
9. The vehicle inspection method based on radar and visual fusion according to claim 7 is characterized in that: The feature extraction and matching algorithm uses the ORB algorithm to extract key points in the image and calculates descriptors for each feature point. The extracted key points are matched through the descriptors and the Hamming distance is used as a similarity metric to find similar areas in the image. The target recognition algorithm performs target detection and recognition through the YOLO model; The image processing algorithm adopts adaptive threshold processing and edge detection algorithm; the adaptive threshold processing dynamically adjusts the threshold by calculating the local mean of the neighborhood around each pixel; the edge detection algorithm uses the Canny edge detection method to identify important contours and edge information in the image.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vehicle inspection method based on radar and vision fusion as described in any one of claims 1 to 9 is implemented.