An intelligent parking map data construction and accurate matching method based on multi-sensor fusion

Abstract

This invention discloses a method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion, relating to the field of intelligent parking technology. The method includes the following steps: S1, multi-sensor data acquisition; S2, heterogeneous data fusion and hierarchical processing; S3, dynamic environment modeling and map construction; S4, accurate matching and localization of multi-source data; and S5, path planning and decision control. This method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion effectively integrates data from heterogeneous sensors through a hybrid hierarchical fusion structure. It not only balances computational efficiency and accuracy but also significantly improves the comprehensiveness of environmental perception through a data complementarity mechanism. It can accurately identify parking lines, surrounding vehicles, and dynamic obstacles. The dynamic parking map constructed based on this data can reflect the latest position and status of static and dynamic elements in the parking lot in real time, providing a solid foundation for subsequent accurate matching and path planning.

Description

Technical Field

[0001] This invention relates to the field of intelligent parking technology, specifically to a method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion. Background Technology

[0002] With the acceleration of urbanization and the continuous growth of car ownership, intelligent parking technology has become a key breakthrough in alleviating urban parking problems. Traditional automatic parking systems mainly rely on a single sensor for environmental perception. For example, ultrasonic sensors detect nearby obstacles at centimeter-level distances, while cameras extract target information such as parking lines and surrounding vehicles through image recognition technology. However, single sensors have significant limitations: ultrasonic sensors are susceptible to interference from reflections from metal surfaces, cameras experience a sharp decline in performance under adverse weather conditions such as rain, snow, and fog, and neither can provide three-dimensional spatial information, resulting in insufficient parking accuracy and reliability in complex scenarios.

[0003] The introduction of multi-sensor fusion technology has brought a qualitative leap to intelligent parking systems. This technology integrates data from heterogeneous sensors such as ultrasonic sensors, cameras, lidar, and inertial measurement units (IMUs) to form a complementary sensing network. For example, lidar can construct a high-precision 3D point cloud model, the IMU monitors the vehicle's motion status in real time through accelerometers and gyroscopes, and the GPS module provides global positioning reference in open areas. This multi-source data fusion not only expands the environmental perception range but also significantly improves the system's fault tolerance through data redundancy design. Research shows that multi-sensor fusion improves parking space recognition accuracy by more than 40% compared to a single-camera solution and increases stability by 65% ​​in strong direct sunlight or backlighting scenarios.

[0004] However, existing multi-sensor fusion technologies still face two major challenges: First, the dynamic adaptability of map data construction is insufficient. Traditional parking maps are mostly based on static environment modeling, making it difficult to cope with real-time changes in dynamic obstacles (such as pedestrians and moving vehicles) within parking lots. Furthermore, differences in parking space layouts and signage styles among different parking lots limit the universality of pre-built maps. Second, the accuracy and robustness of multi-source data matching are insufficient. Due to differences in sampling frequency, coordinate system, and noise characteristics, direct fusion of sensor data can easily lead to information conflicts. Existing fusion solutions often employ loosely coupled architectures, where each sensor processes data independently before simple overlay, and relies heavily on manually designed feature matching rules, making it difficult to adapt to dynamic changes in complex scenarios and directly impacting the accuracy of path planning. Summary of the Invention

[0005] The purpose of this invention is to provide a method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion, comprising the following steps: S1. Multi-sensor data acquisition: Collect raw data from different sensors and perform preliminary preprocessing to improve data quality and the efficiency of subsequent processing. S2. Heterogeneous data fusion and hierarchical processing: A hybrid hierarchical fusion structure is adopted to process data of different precision in stages to balance computational efficiency and accuracy, and to integrate data from different sensors to form a complementary sensing network. S3. Dynamic Environment Modeling and Map Building: Through dynamic environment modeling, the environmental information in the parking lot is updated in real time, and a digital map that can accurately reflect the static and dynamic environment is built. This digital map can accurately reflect the actual situation in the parking lot, including the latest location and status of dynamic and static elements. S4. Multi-source data precise matching and positioning: The vehicle's real-time environmental information is compared and matched with pre-built intelligent parking map data to determine the vehicle's accurate position and direction on the map. S5. Path planning and decision control: Based on the accurate map and vehicle positioning information obtained in the previous steps, the optimal parking path is planned, and the vehicle is controlled to park according to the planned path.

[0007] Furthermore, in step S1, the sensors include, but are not limited to, ultrasonic sensors, cameras, lidar, inertial measurement units (IMUs), and GPS modules. Specifically, data acquisition is performed using the software development kits (SDKs) or application programming interfaces (APIs) of each sensor. These interfaces provide a standard way to interact with the sensors, making the data acquisition process more standardized and efficient. The acquired data includes distance information from ultrasonic sensors, image data from cameras, point cloud data from lidar, motion data from IMUs, and location information from GPS modules. Ultrasonic sensors: used for short-range obstacle detection, measuring distance by emitting and receiving ultrasonic waves; Cameras: Used to capture visual information within the parking lot and extract target information such as parking space lines and surrounding vehicles through image recognition technology; LiDAR: It can build high-precision 3D point cloud models and provide rich environmental spatial information; Inertial Measurement Unit (IMU): Contains accelerometers and gyroscopes, used to monitor the vehicle's motion status in real time, such as acceleration and angular velocity, and provide dynamic information to the system; GPS module: Provides global positioning reference in open areas, especially suitable for large-scale positioning.

[0008] Furthermore, in step S1, the preprocessing specifically includes: Data cleaning: Using digital signal processing techniques, such as filtering algorithms (e.g., mean filtering, median filtering, or Gaussian filtering), to process the raw data in order to remove noise and outliers introduced during sensor measurement and improve data quality; Data synchronization: This is achieved through a timestamp synchronization mechanism, which uses the timestamp information in the sensor data packets to align the data from different sensors in time, ensuring that the data from different sensors are aligned in time. Data calibration: Data calibration is performed using a calibration board or a calibration method specific to a particular scenario to eliminate systematic errors between sensors, as follows: The calibration plate method involves calibrating a sensor by measuring known feature points on a calibration plate of a specific shape and size. For example, for cameras, a checkerboard calibration plate is used for image distortion correction and camera intrinsic parameter calibration; for LiDAR, a calibration object of a specific shape is used to calibrate the accuracy of its point cloud data.

[0009] The calibration method for specific scenarios involves using known features within the scenario for calibration. For example, in a parking lot, known features such as parking lines or walls can be used to calibrate the sensor's external parameters.

[0010] Furthermore, in step S2, the processing logic of the hybrid layered fusion structure is as follows: first, the low-precision data is preliminarily processed, and then it is deeply fused with the high-precision data. Preliminary processing of low-precision data: Preliminary processing of data from low-precision sensors, such as preliminary feature extraction, is performed. The low-precision sensor data includes distance information from ultrasonic sensors and preliminary motion data from IMUs. High-precision data deep fusion: The low-precision data after preliminary processing is deeply fused with data from high-precision sensors, such as feature matching, data association, and state estimation, to achieve complementarity and enhancement between different sensor data. The high-precision sensor data includes image data from cameras, point cloud data from lidar, etc.

[0011] Furthermore, step S3 includes the following sub-steps: S31. Dynamic obstacle detection: Using sensor data, identify dynamic obstacles in the parking lot, such as pedestrians and moving vehicles; S32. Environmental Information Update: Based on the dynamic obstacle detection results and combined with static environmental information (such as parking lines, walls, etc.), the environmental information in the parking lot is updated in real time. S33. Map Optimization: Optimize the constructed digital map, including removing redundant information, smoothing noisy data, and enhancing feature points.

[0012] Furthermore, the specific operation of step S31 is as follows: a target detection algorithm, such as a convolutional neural network (CNN) model based on deep learning, is used to process the sensor data, identify dynamic obstacles, and obtain their motion trajectory and speed information by continuously tracking the dynamic obstacles, so as to provide a basis for subsequent environmental information updates and map optimization.

[0013] Furthermore, in step S32, an incremental update strategy is adopted, that is, only the part of the map data that has changed is updated to improve update efficiency. Specifically, for static obstacles (such as parking lines, walls, etc.), their position information remains unchanged; for dynamic obstacles, the corresponding area in the digital map is updated according to their real-time position, and the detected dynamic obstacle information is integrated into the map.

[0014] Furthermore, step S4 includes the following sub-steps: S41. Feature matching: Compare the environmental features perceived by the vehicle in real time with the features in the pre-built intelligent parking map; S42. Pose estimation: Based on the feature matching results, estimate the vehicle's position and pose in the map, including position coordinates and orientation angle; S43. Error Correction: Algorithms such as Kalman filtering or particle filtering are used to correct the pose estimation results to improve the accuracy of positioning. Kalman filtering reduces errors by recursively updating the state estimate by combining prediction and observation information. Particle filtering, on the other hand, approximates the true state distribution by maintaining a set of state particles and weighting and resampling the particles based on observation information.

[0015] Furthermore, in step S41, a feature extraction algorithm is used to extract key feature points or feature descriptors from sensor data, such as using algorithms like SIFT, SURF, or ORB to extract features from image data, or extracting geometric features from LiDAR point cloud data. At the same time, corresponding feature points or feature sets are extracted from the intelligent parking map. The matching process involves comparing the similarity between real-time features and map features to find the best matching pair, thereby determining the correspondence between the environment perceived by the vehicle and the map.

[0016] Furthermore, in step S42, the pose estimation is specifically performed using either the ICP (Iterative Closest Point) algorithm or the NDT (Normal Distribution Transform) algorithm. The ICP algorithm estimates the vehicle's pose by iteratively finding the closest point pairs and optimizing the transformation matrix to minimize the difference between the two sets of point cloud data. The NDT algorithm divides the space into multiple voxels and calculates the normal distribution of points within each voxel, estimating the pose by matching the normal distributions between different voxels.

[0017] Furthermore, step S5 includes the following sub-steps: S51. Path search: Use the A* algorithm or Dijkstra's algorithm to search for feasible paths from the current location to the target parking space on the map. S52. Path optimization: Optimize the searched path using optimization algorithms (such as genetic algorithm and simulated annealing algorithm) to improve parking efficiency and safety; S53. Decision Control: Controlling the movement of the vehicle based on the path planning results, specifically through the vehicle control system (such as CAN bus).

[0018] This invention provides a method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion, which has the following beneficial effects: 1. This invention effectively integrates data from heterogeneous sensors such as ultrasonic sensors, cameras, lidar, IMU, and GPS modules through a hybrid layered fusion structure. This layered processing method not only balances computational efficiency and accuracy but also significantly improves the comprehensiveness of environmental perception through a data complementarity mechanism. It can accurately identify parking lines, surrounding vehicles, and dynamic obstacles. The dynamic parking map built based on this data can reflect the latest position and status of static and dynamic elements in the parking lot in real time, providing a solid foundation for subsequent accurate matching and path planning.

[0019] 2. This invention extracts key features from real-time sensor data using a feature extraction algorithm and performs similarity matching with features in a pre-built map. It then combines this with ICP or NDT algorithms for pose estimation and uses Kalman filtering or particle filtering to correct errors, significantly improving positioning robustness. Even in complex scenes or when sensor data conflicts occur, the hybrid hierarchical fusion structure maintains its fault tolerance through data redundancy design, ensuring accurate vehicle position and orientation recognition on the map. Based on this, the path planning module can quickly search for the optimal path using A* or Dijkstra algorithms and avoid collisions through optimization techniques such as genetic algorithms, ultimately achieving precise decision-making and control through the vehicle control system. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the operational steps of a method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion, as described in this invention. Detailed Implementation

[0021] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0022] Example 1: Application in Indoor Parking Lots Step S1: Multi-sensor data acquisition: Sensors selected: ultrasonic sensor, camera, lidar, IMU, GPS module (for indoor positioning, UWB positioning system replaces GPS for high-precision positioning). In this embodiment, the data collected by each sensor and its parameter settings are as follows: Ultrasonic sensor: Collects distance information of obstacles around the vehicle, with a sampling frequency of 20Hz and an effective detection distance of 0.1-5 meters.

[0023] Camera: Collects image data in the parking lot at a frame rate of 30fps and a resolution of 1920x1080, used to identify parking space lines and surrounding vehicles.

[0024] LiDAR: Acquires 3D point cloud data with a scanning frequency of 10Hz and an angular resolution of 0.5° to build a high-precision environmental model.

[0025] IMU: Collects vehicle acceleration and angular velocity information at a sampling frequency of 100Hz, used to monitor the dynamic state of the vehicle.

[0026] UWB positioning system: provides high-precision location information of vehicles indoors, with a positioning accuracy of 10 centimeters.

[0027] Step S2: Heterogeneous data fusion and hierarchical processing Preliminary processing of low-precision data: The distance information from the ultrasonic sensor is filtered to remove noise and outliers. Initial motion data from the IMU is integrated to obtain preliminary velocity and displacement information.

[0028] High-precision data deep fusion: Processed ultrasonic and IMU data are matched and correlated with camera image data and LiDAR point cloud data. The ICP algorithm is used to register the point cloud data, estimate the vehicle pose, and optimize the results by combining the image feature point matching results.

[0029] Step S3, Dynamic Environment Modeling and Map Building: Dynamic obstacle detection: The YOLOv5 object detection algorithm is used to process camera images and identify dynamic obstacles such as pedestrians and moving vehicles. Combined with LiDAR point cloud data, a clustering algorithm is used to further confirm the location of dynamic obstacles.

[0030] Environmental information updates: An incremental update strategy is adopted, updating only the map data of areas where dynamic obstacles are located. The location information of static obstacles (such as parking lines and walls) remains unchanged.

[0031] Map optimization: Remove redundant point cloud data, smooth noisy points, and enhance key feature points (such as parking space corner points).

[0032] Step S4: Precise matching and positioning of multi-source data: Feature matching: Extract SIFT feature points from real-time sensor data and match them with feature points in a pre-built smart parking map.

[0033] Pose estimation: The NDT algorithm is used for pose estimation, which estimates the vehicle position and orientation by matching the normal distribution between different voxels.

[0034] Error correction: Kalman filtering is used to correct the pose estimation results and reduce the positioning error.

[0035] Step S5, Path Planning and Decision Control: First, the A* algorithm is used to search for feasible paths from the current location to the target parking space on the map. Then, a genetic algorithm is used to optimize the searched paths to avoid collisions and improve parking efficiency. Finally, control commands are sent to the vehicle's execution system via the CAN bus to achieve automatic parking.

[0036] Implementation Results 1. A high-precision digital map of an indoor parking lot was successfully constructed, including real-time location information of both static and dynamic obstacles.

[0037] 2. Vehicle positioning accuracy reaches within 20 centimeters, parking path planning time is less than 1 second, and parking success rate is over 98%.

[0038] Example 2: Application in outdoor parking lots Step S1: Multi-sensor data acquisition Sensor selection: ultrasonic sensor, camera, lidar, IMU, GPS module. In this embodiment, the data acquisition parameters of the ultrasonic sensor, camera, lidar, and IMU are the same as in Embodiment 1. GPS module: provides global positioning information in open areas with a positioning accuracy of 1-2 meters, used to assist in the initial positioning of the vehicle.

[0039] Step S2: Heterogeneous data fusion and hierarchical processing The processing logic is the same as in Example 1, but considering the large changes in outdoor ambient light, the camera images are preprocessed (such as histogram equalization) to improve the accuracy of target detection.

[0040] Step S3: Dynamic Environment Modeling and Map Building Dynamic obstacle detection: Combining camera and LiDAR data, the Faster R-CNN object detection algorithm is used to identify dynamic obstacles. A multi-frame tracking algorithm (such as SORT) is introduced to improve the stability and accuracy of dynamic obstacle tracking. Environmental information update and map optimization: Same as in Example 1.

[0041] Step S4: Precise matching and positioning of multi-source data: Feature matching: Considering the abundance of feature points in the outdoor environment, the ORB feature extraction algorithm is used to improve the matching speed. Pose estimation and error correction: Same as in Example 1.

[0042] Step S5, Path Planning and Decision Control: The processing logic is the same as in Example 1, but considering that there may be ramps in the outdoor parking lot, vehicle dynamics constraints need to be considered when planning the route.

[0043] Implementation Results 1. A high-precision digital map of an outdoor parking lot was successfully constructed, including static elements such as roads, parking spaces, and green belts, as well as dynamic elements such as pedestrians and vehicles.

[0044] 2. The vehicle positioning accuracy is within 1 meter in open areas and better than 2 meters in complex environments (such as when obstructed by trees).

[0045] 3. Parking path planning time is less than 1.5 seconds, parking success rate is over 96%, and it can adapt to parking needs under different weather and lighting conditions.

[0046] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion, characterized in that, Includes the following steps: S1. Multi-sensor data acquisition: Collect raw data from different sensors and perform preliminary preprocessing; S2. Heterogeneous data fusion and hierarchical processing: A hybrid hierarchical fusion structure is adopted to process data of different precision in stages, and to integrate data from different sensors to form a complementary sensing network. S3. Dynamic Environment Modeling and Map Building: Through dynamic environment modeling, the environmental information in the parking lot is updated in real time to build a digital map that can accurately reflect the static and dynamic environment. S4. Accurate matching and positioning of multi-source data: The vehicle compares and matches the environmental information perceived in real time with the pre-built intelligent parking map data to determine the vehicle's accurate position and direction on the map.

2. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 1, characterized in that, In step S1, the sensors include, but are not limited to, ultrasonic sensors, cameras, lidar, inertial measurement units, and GPS modules. Specifically, data is collected using the SDK or API of each sensor. The collected data includes distance information from ultrasonic sensors, image data from cameras, point cloud data from lidar, motion data from IMU, and location information from GPS modules.

3. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 1, characterized in that, In step S2, the processing logic of the hybrid layered fusion structure is as follows: first, the low-precision data is preliminarily processed, and then it is deeply fused with the high-precision data. Preliminary processing of low-precision data: Preliminary processing of data from low-precision sensors, namely preliminary feature extraction, is performed. The low-precision sensor data includes distance information from ultrasonic sensors and preliminary motion data from IMUs. High-precision data deep fusion: The low-precision data after preliminary processing is deeply fused with the data from high-precision sensors, namely feature matching, data association, and state estimation. The high-precision sensor data includes image data from cameras and point cloud data from lidar.

4. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 1, characterized in that, Step S3 includes the following sub-steps: S31. Dynamic obstacle detection: Using sensor data, identify dynamic obstacles in the parking lot; S32. Environmental Information Update: Based on the dynamic obstacle detection results and combined with static environmental information, the environmental information in the parking lot is updated in real time. S33. Map Optimization: Optimize the constructed digital map, including removing redundant information, smoothing noisy data, and enhancing feature points.

5. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 4, characterized in that, The specific operation of step S31 is as follows: a target detection algorithm is used to process the sensor data, identify dynamic obstacles, and obtain their motion trajectory and speed information by continuously tracking the dynamic obstacles.

6. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 4, characterized in that, In step S32, an incremental update strategy is adopted, that is, only the part of the map data that has changed is updated. Specifically, for static obstacles, their position information remains unchanged; for dynamic obstacles, the corresponding area in the digital map is updated according to their real-time position, and the detected dynamic obstacle information is integrated into the map.

7. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 1, characterized in that, Step S4 includes the following sub-steps: S41. Feature matching: Compare the environmental features perceived by the vehicle in real time with the features in the pre-built intelligent parking map; S42. Pose estimation: Based on the feature matching results, estimate the vehicle's position and pose in the map, including position coordinates and orientation angle; S43. Error Correction: The pose estimation results are corrected using algorithms such as Kalman filtering or particle filtering.

8. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 7, characterized in that, In step S41, a feature extraction algorithm is used to extract key feature points or feature descriptors from sensor data, and corresponding feature points or feature sets are extracted from the intelligent parking map. The matching process involves comparing the similarity between real-time features and map features to find the best matching pair, thereby determining the correspondence between the environment perceived by the vehicle and the map.

9. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 7, characterized in that, In step S42, the pose estimation is specifically performed using either the ICP algorithm or the NDT algorithm. The ICP algorithm estimates the vehicle's pose by iteratively finding the nearest point pair and optimizing the transformation matrix to minimize the difference between the two sets of point cloud data. The NDT algorithm divides the space into multiple voxels and calculates the normal distribution of points within each voxel, estimating the pose by matching the normal distributions between different voxels.

10. The method for constructing and accurately matching intelligent parking map data based on multi-sensor fusion according to claim 1, characterized in that, It also includes the following steps: S5. Path planning and decision control: Based on the accurate map and vehicle positioning information obtained in the previous steps, the optimal parking path is planned and the vehicle is controlled to park according to the planned path. S51. Path search: Use the A* algorithm or Dijkstra's algorithm to search for feasible paths from the current location to the target parking space on the map. S52, Path Optimization: Optimize the searched path using optimization algorithms; S53. Decision Control: Controlling the movement of the vehicle based on the path planning results, specifically through the vehicle control system.

Images