Memory parking map optimization method, electronic equipment and vehicle
By using a surround view camera in memory parking, the 2D point cloud information and parking posture are extracted, and the SLAM map is constructed and updated, the problem of dynamic changes in parking lots affecting memory parking accuracy is solved, and the reliability and safety of parking is improved. The vehicle-side data closed loop and cost-effective improvement have been achieved.
Patent Information
- Application Number
- CN202411183574.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art in memory parking has caused the map matching accuracy to decrease due to dynamic changes in the parking lot environment, which affects the reliability and safety of parking.
Use a surround view camera to obtain the original image, extract the library position information and 2D point cloud information of road signs, combine the parking position information, build a basic SLAM map, and build a new SLAM map and basic SLAM map in real time during memory parking positioning. Compare and match, eliminate dynamic objects, supplement static scenes, and optimize map updates.
It improves memory parking positioning accuracy, increases parking reliability and safety, improves customers' trust in memory parking functions, realizes a closed loop of vehicle-side data, reduces perception needs without increasing costs.
Smart Images

Figure CN119085629A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vehicle intelligent driving technology, and in particular to a memory parking map optimization method, electronic equipment, and a vehicle. Background Art
[0002] With the continuous development of the field of intelligent driving, the use of memory parking function is becoming more and more widespread. Due to the large differences and large number of parking lots, the development of high-precision parking map is not mature, and memory parking still relies more on the self-built map from the vehicle.
[0003] The current existing technology method is to build a SLAM map based on SLAM (simultaneous localization and mapping) technology. When using the memory parking function, the position of the vehicle is determined by matching the perception results with the semantic map, and the target parking space is found by cruising to complete parking. However, the mapping process of the existing technology is usually based on the static environment assumption, that is, it is assumed that the objects in the parking lot environment are static and unchanged. But in fact, due to the presence of a large number of dynamic objects in the parking lot, such as moving pedestrians or vehicles, or semi-static objects such as vehicles parked in storage spaces, temporarily placed objects, etc., the scene in the parking lot will continue to change over time, resulting in a large difference between the constructed map and the actual scene, which will reduce the matching accuracy when performing visual matching, and is not conducive to later path planning based on the map. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a memory parking map optimization method, which can effectively improve the positioning accuracy of the vehicle memory parking process and increase the reliability and safety of parking.
[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0006] A memory parking map optimization method mainly includes:
[0007] 1) Obtaining an original image of the vehicle's surroundings, and extracting 2D point cloud information of storage location information and road signs from the original image;
[0008] 2) Calculate the vehicle's position information based on the vehicle's initial positioning information, acceleration information, angular velocity information, and wheel speed meter information;
[0009] 3) Based on the storage location information, the 2D point cloud information of the road signs and the vehicle posture information, a basic SLAM map is generated using a subgraph construction method;
[0010] 4) During the memory parking positioning process, a new SLAM map is constructed in real time, and compared and matched with the basic SLAM map in real time to eliminate dynamic objects in the basic SLAM map and supplement static scenes to achieve optimization and update of the SLAM map.
[0011] Furthermore, the original image of the vehicle's surrounding environment is acquired through a plurality of vehicle-mounted surround-view cameras, and the original image is subjected to distortion correction and coordinate transformation through a vehicle-mounted visual perception module, and the transformed images are spliced to form a surround-view spliced map of the vehicle's surrounding environment, and the storage location information and 2D point cloud information of road signs are extracted from the surround-view spliced map.
[0012] Furthermore, the vehicle initialization positioning information is obtained by receiving RTK positioning information through the on-board odometer module. At the same time, the on-board odometer module calculates the displacement, speed and rotation angle of the vehicle at the current moment according to the vehicle's acceleration information and angular velocity information, and combines it with the measurement data of the vehicle's wheel speed meter to infer the corrected vehicle position and posture information.
[0013] Furthermore, the generation of the basic SLAM map includes: the on-board mapping module maps the 2D point cloud information features of the storage location information and road signs to the global coordinate system based on the initially received self-vehicle posture information, and adds the first detected frame as a key frame to the sub-map, and then matches and corrects the posture of the current frame point cloud in the sub-map according to the subsequently received self-vehicle posture information, and inserts the point cloud with the corrected posture into the sub-map. When the number of point cloud insertions reaches a preset value, the sub-map construction is terminated.
[0014] Furthermore, the basic SLAM map is stored in a preset storage address of the on-board map storage and update module, and the newly created SLAM map subsequently generated by repositioning based on the basic SLAM map is also stored in the preset storage address. When the storage quantity of the newly created SLAM map reaches a set threshold value one, the stored newly created SLAM map is compared and matched with the basic SLAM map.
[0015] Furthermore, the comparison and matching between the newly created SLAM map and the basic SLAM map includes:
[0016] When a map element in a key frame of the basic SLAM map does not appear in the key frame of the corresponding position in the subsequent newly created SLAM map for more than a preset number of times, the element is considered to be a dynamic object and needs to be removed, and the map information in the corresponding key frame of the newly created SLAM map is updated to the basic SLAM map to remove the dynamic object;
[0017] When the same map element appears in the key frames of the same position in multiple newly created SLAM maps but is not in the key frames of the corresponding position in the basic SLAM map, the element is considered to be a static scene and needs to be added. The map information in the corresponding key frames of the newly created SLAM map is updated to the basic SLAM map to add the static scene.
[0018] Furthermore, when the storage quantity of the newly created SLAM maps exceeds a set threshold value of two, the front newly created SLAM maps are deleted in sequence, and the set threshold value of two is greater than the set threshold value of one.
[0019] Furthermore, when the memory parking positioning is turned on, the basic SLAM map is loaded as prior information, and the 2D point cloud information in each sub-map construction key frame is extracted to match with the prior information, and the vehicle posture information at the current frame moment is corrected according to the matching result.
[0020] Compared with the prior art, the present invention has the following main advantages:
[0021] 1. The present invention proposes a memory parking map optimization method, which extracts 2D point cloud information based on the original image provided by the surround-view camera, constructs a basic SLAM map, and continuously constructs a new SLAM map during the memory parking positioning process, and compares and matches it with the basic SLAM map to eliminate dynamic objects and supplement static scenes, gradually forming a pure scene map of the parking lot on the vehicle side, which can effectively solve the impact of the dynamic environment on the positioning accuracy during the memory parking process, thereby improving the positioning accuracy of the vehicle's memory parking process, increasing the reliability and safety of parking, and enhancing customers' trust in the memory parking function.
[0022] 2. The present invention can form a data closed loop on the vehicle side by performing secondary mapping and vehicle-side map matching optimization while positioning on the vehicle side. At the same time, based on the existing visual perception solution, the present invention can eliminate the influence of dynamic objects and supplement the missed static objects, thereby reducing the demand for perception without increasing costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a schematic diagram of a method for optimizing a memory parking map according to an embodiment of the present invention;
[0024] Figure 2 It is an overall flow chart of the memory parking map optimization method in an embodiment of the present invention;
[0025] Figure 3 A schematic diagram of subgraph construction in an embodiment of the present invention;
[0026] Figure 4 A schematic diagram of subgraph maintenance in an embodiment of the present invention;
[0027] Figure 5 A schematic diagram of posture constraint construction in an embodiment of the present invention;
[0028] Figure 6 A schematic diagram of matching a basic SLAM map with a newly created SLAM map in an embodiment of the present invention;
[0029] Figure 7 The figure is a specific principle block diagram of the map matching method in an embodiment of the present invention. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0031] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.
[0032] In the present invention, unless otherwise clearly specified and limited, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically limited.
[0033] Embodiment 1: This embodiment provides a method for optimizing a memory parking map. Figure 1 As shown, it mainly includes:
[0034] 1) Obtaining an original image of the vehicle's surroundings, and extracting 2D point cloud information of storage location information and road signs from the original image;
[0035] 2) Calculate the vehicle's position information based on the vehicle's initial positioning information, acceleration information, angular velocity information, and wheel speed meter information;
[0036] 3) Based on the storage location information, the 2D point cloud information of the road signs and the vehicle posture information, a basic SLAM map is generated using a subgraph construction method;
[0037] 4) During the memory parking positioning process, a new SLAM map is constructed in real time, and compared and matched with the basic SLAM map in real time to eliminate dynamic objects in the basic SLAM map and supplement static scenes to achieve optimization and update of the SLAM map.
[0038] like Figure 2 As shown, the memory parking map optimization method specifically includes the following steps:
[0039] S1: After the vehicle is powered on, the odometer module receives RTK information and completes the positioning initialization of the vehicle according to the RTK positioning information;
[0040] S2: During vehicle driving, the odometer module integrates the acceleration information and angular velocity information provided by the IMU to calculate the displacement, speed and rotation angle of the vehicle at the current moment during driving, and performs ESKF fusion processing with the measurement data of the wheel speed meter to obtain the corrected vehicle position and posture information;
[0041] S3: The vehicle enters the area where the memory parking function is available, and the driver clicks the map-building button. The system state machine converts the information input by the driver into a state machine signal and starts the corresponding map-building thread.
[0042] S4: Take the pose provided by the odometer as the initial pose, and set the starting point of map construction as the origin of the currently constructed SLAM map;
[0043] S5: The vehicle's four surround-view cameras detect parking spaces, lane lines, arrows, speed bumps, no-parking areas, storage location numbers and other information around the vehicle. The visual perception module performs distortion correction and coordinate conversion on the extracted original images, and then stitches the converted images to form a surround-view stitching map around the vehicle.
[0044] S6: The visual perception module uses the deep learning module to perform semantic segmentation on the spliced image, segmenting information such as parking spaces, lane lines, arrows, speed bumps, no-parking areas, and storage location numbers, and extracts 2D point cloud information to the mapping module;
[0045] S7: The mapping module uses the initial pose input by the odometer to transform the semantic segmentation result input by the visual perception module, maps it to the global coordinate system, adds the first frame as a key frame to the sub-image, and then estimates the pose of the point cloud of the current frame in the sub-image according to the pose input by the odometer, matches the semantic feature information in the current frame with the sub-image to correct the pose of the point cloud, and inserts the point cloud into the sub-image according to the corrected pose. After a certain number of point clouds are inserted, the construction of the sub-image is terminated and the next sub-image is started;
[0046] like Figure 3As shown, the sub-graph construction method specifically includes: adding data, pose prediction, local map scanning and matching, sub-graph construction and grid map updating.
[0047] Among them, the added data mainly includes the processing of point cloud data and the transparent transmission of odometer data. The point cloud data transmitted by the visual perception module is transmitted to the algorithm layer through the interface layer. In the algorithm layer, the maintenance of the point cloud is to match the point cloud of each frame with the sub-image being updated first, and then rasterize the point cloud of the frame according to the matching posture and insert it into the sub-image. For each fixed-size grid, if the point falls within the grid, it is occupied, otherwise it is idle. Since there may be thousands of points in each frame, the amount of point-by-point matching data is large when performing point-map matching, so it is necessary to add a strategy to reduce some points to reduce the computational overhead without affecting the calculation precision and accuracy. In this scheme, adaptive voxel filtering is used to calculate sparse point clouds for point cloud matching. The specific method is: divide xyz by the voxel side length (set in the configuration parameters) and round it to get the coordinates (x, y, z) of the voxel unit where xyz is located, and then put the three integers into a 3*32-byte number to represent the index of the unit. According to the index, it is easy to determine whether this unit is the first insertion point. If it is not the first time, no point will be inserted. Adaptive means that if the number of points after voxel filtering is greater than the minimum number of points set in the configuration parameters, the result of this filtering is returned; otherwise, the voxel side length is adjusted by binary search until the number of points is greater than the minimum number of points set. Adaptive filtering can ensure that the appropriate number of points are processed in each frame.
[0048] Furthermore, since the accuracy of the extracted 2D point cloud is not as high as that of the laser point cloud, the cumulative error of the positioning result obtained by using point cloud matching completely will be relatively large. The transmitted odometer data can be used to maintain accurate real-time pose estimation in a short period of time. A queue of odometer data is maintained in the pose inference device. The length of the queue can be set. The pose difference between the first and last elements of the queue can be used to calculate the linear velocity in the x and y directions and the angular velocity of the z axis (heading angular velocity) for pose estimation. It is necessary to select an appropriate queue length. If it is too short, it will cause pose jitter, and if it is too long, it will cause a lag in pose update. The pose estimation uses a uniform velocity model. The estimated pose is divided into two types: translation (x, y, z) and attitude (quaternion). The predicted pose is relative to the last optimized pose, so the result of this prediction should be added to the pose of the last pose. 1. Translation prediction: The linear velocity calculated by the two data at the beginning and end of the odometer data queue is multiplied by the time difference between the prediction time and the previous pose; 2. Pose prediction: The angular velocity calculated by the two data at the beginning and end of the odometer data queue is multiplied by the time difference between the prediction time and the previous pose. The predicted pose is obtained by the pose inference device and passed to the local scan matching as the initial value and constraint item of the optimization.
[0049] Furthermore, the local map scan matching includes subgraphs, nodes and point clouds, and each node contains a structure of a pose to be optimized and corresponding point cloud data. The subgraph is an object that contains the starting position, the probability value after the point cloud is rasterized, and whether the subgraph is finished. When a new frame of point cloud data arrives, the pose prediction module predicts the pose of the vehicle at that moment, and uses the predicted pose as the initial value to match the point cloud with the subgraph being updated to obtain the optimized pose. After the matching is completed, if the displacement or rotation of the vehicle body is greater than the set value of the configuration parameter, the point cloud is inserted into the two latest subgraphs according to the optimized pose and rasterized, and the number of nodes in the two latest subgraphs is increased by one. When the number of nodes is equal to the set threshold, the state of the previous subgraph is set to end, and a new subgraph is created. The matching uses the CorrelativeScan Matching (CSM) algorithm, and the purpose of the scan matching is to find the appropriate position and angle of the current vehicle body in the subgraph grid map.
[0050] Further, such as Figure 4 As shown, the sub-graph construction and grid map update: after scanning and matching, a relatively accurate posture is obtained. If the displacement or rotation of the posture exceeds the set amount compared with the previous moment, the point cloud of the current frame is transferred to the matching posture, and then rasterized and inserted into the sub-graph, that is, the probability of the corresponding grid in the probability grid map is updated. In the process of inserting the point cloud into the sub-graph, two sub-graphs being updated are maintained. When the number of nodes inserted into the newer sub-graph is equal to the number set in the parameter (for example, N, which can be calibrated), a new sub-graph is created. When the number of nodes inserted into the older sub-graph is 2*N, the sub-graph is marked as ended, and a new sub-graph is created at the same time. Figure 4 As shown in the figure, this ensures that half of the two subgraphs overlap, which is used to build constraints between subgraphs during backend optimization.
[0051] Furthermore, the storage location information sensed in each frame will overlap with the storage location information obtained in the previous frame, and the overlapping storage locations need to be merged through the vehicle posture relationship between frames. The merging uses the anchor points of the corner points of the sensed storage locations to calculate the Euclidean distance between the anchor points. When the Euclidean distance of the storage location anchor points between the two frames is greater than a certain value, it is considered that these are two different storage locations, and the storage locations are inserted. The corner point coordinates of the storage locations in the map are calculated by average when the storage locations are merged to reduce the impact of perception misdetection.
[0052] S8: Calculate the constraint relationships between all keyframe nodes, keyframes and subgraphs, and subgraphs and subgraphs, and finally optimize the pose graph to make the poses of all nodes reach the global optimum, and obtain the final global map (basic SLAM map);
[0053] like Figure 5 As shown in , the pose graph optimization specifically includes: treating nodes and subgraphs as nodes in the pose graph, which are variables to be optimized, and the constraints are the relative pose transformations between two nodes. Figure 5 As shown, each subgraph has a fixed number of nodes (which can be calibrated). Each node in each subgraph will establish intra-subgraph constraints with the subgraph in which it is located, using the Correlative Scan Matching (CSM) algorithm. Between different subgraphs, nodes will build inter-subgraph constraints with subgraphs that have ended but do not belong to them. Since the direct matching method will take up a lot of computing resources, this solution uses the branch and bound method to match nodes and subgraphs. According to all the nodes and the constraints between them, all errors are reduced and evenly distributed to each node to achieve the purpose of reducing the overall error.
[0054] S9: The map management module allocates a storage address for the global map (basic SLAM map);
[0055] S10: The driver turns on the memory parking function, selects the constructed map, and enters the memory cruise mode. The map is loaded as prior information, and for each key frame of positioning recognition, the odometer is combined to extract 2D point cloud information; the odometer calculates the vehicle posture at the current frame, and matches the 2D point cloud information extracted from the current frame with the map at the inferred posture, and corrects the current posture according to the matching result;
[0056] S11: When entering the memory cruise mode, the secondary map construction for the scene is started synchronously. The posture input by the odometer when the driver clicks to enter the memory cruise is used as the initial posture. The first detected frame is added to the sub-map as the key frame. The 2D point cloud extracted in the subsequent positioning process is matched with the sub-map. According to the steps in S8, a global map of the scene is constructed again (new SLAM map). Figure 1 ).
[0057] S12: The map management module creates a new SLAM map Figure 1 ) is stored in the memory address where the basic SLAM map is stored in step S8. The number of maps stored at this address can be calibrated. When the number of newly created SLAM maps stored exceeds a threshold, the front newly created SLAM map is deleted;
[0058] S13: When the number of newly created SLAM maps reaches the set threshold (which can be calibrated), all newly created SLAM maps are matched with the basic SLAM map. For each key frame of the basic SLAM map, search for key frames near the corresponding position in the newly created SLAM map, and match and compare the point cloud information in the filtered key frames to confirm whether there is point cloud information that appears in the basic SLAM map but does not appear in the corresponding position of other newly created SLAM maps for multiple times (the number of times can be calibrated). If so, the point cloud information is considered to be a dynamic object, and this part of the point cloud information is deleted from the key frame of the basic SLAM map. Traverse all key frames in the basic SLAM and delete the point cloud information of dynamic objects;
[0059] S14: Confirm whether there is point cloud information that is not in the basic SLAM map but appears multiple times in the corresponding position in other newly created SLAM maps, and confirm whether the point cloud information that appears multiple times in the newly created SLAM map can be matched. If it can be matched, it is considered that the point cloud information is a static object, but it was missed when the basic SLAM map was built. The point cloud information is supplemented in the basic SLAM map, and its position in the basic SLAM map is predicted based on the pose information corresponding to the point cloud. The point cloud information is matched with other elements in the basic SLAM map to correct the pose and insert the point cloud into the basic SLAM map with the corrected pose. Traverse all key frames in the basic SLAM and supplement the point cloud information of static objects;
[0060] like Figure 6 As shown in the figure, the matching between the basic SLAM map and the newly created SLAM map specifically includes:
[0061] When positioning for the first time, the detected key frame is matched with the basic SLAM map. When the detected key frame matches the key frame i in the basic SLAM map, a label keyi is recorded for the detected key frame. If the frame is the first matching frame, the frame is added as the first frame to the newly created SLAM map. Figure 1 If the frame is not the first matching frame, the frame is matched with the sub-map of the newly created SLAM map and the pose is optimized and inserted into the newly created SLAM map. Figure 1 In the sub-graph, until the positioning is completed and the global optimization is completed to build a new SLAM map Figure 1 .
[0062] In each subsequent positioning, the label keyi is recorded for each positioning key frame matched with the key frame i in the basic SLAM map, and inserted into the corresponding newly built SLAM map. When the number of newly built SLAM maps reaches the threshold value (which can be calibrated), the key frame of the corresponding label in the newly built SLAM map is searched for each key frame in the basic SLAM map and filtered out, and all these filtered label key frames and the corresponding key frames in the basic SLAM map are matched to confirm whether there is point cloud information that appears in the basic SLAM map key frame, but does not appear in the filtered label key frame for many times (which can be calibrated). If there is, it is considered that the point cloud information is a dynamic object that needs to be eliminated, and the key frame is deleted in the basic SLAM map. At the same time, the label key frame without the point cloud information is matched with the sub-map where the basic SLAM map key frame is located, and inserted into the sub-map of the basic SLAM map with the optimized posture. If it is confirmed during matching that there are key frames of the basic SLAM map that do not appear, but there is point cloud information that appears multiple times (calibrable) in the filtered label key frames and can be matched, if there is, then the point cloud information is considered to be a static object and needs to be supplemented. The key frame is deleted from the basic SLAM map, and the label key frame with the point cloud information is matched with the sub-map where the key frame of the basic SLAM map is located, and inserted into the sub-map of the basic SLAM map with the optimized posture.
[0063] S15: Update and store the basic SLAM map.
[0064] Embodiment 2: This embodiment provides a memory parking map optimization method, which also includes a system state machine module, a visual perception module, an odometer module, a mapping module, a map storage and updating module, and a positioning module.
[0065] When a vehicle enters the basement, the visual perception module senses the environment around the vehicle, including lane lines, garage locations, landmarks, etc. It dedistorts, crops, stitches, and calibrates the images taken by the four surround-view cameras to obtain a bird's-eye view of the vehicle's surroundings, processes the information in the images, and extracts 2D point cloud data of garage locations and landmarks through a deep learning model.
[0066] The odometer module receives RTK (real-time high-precision positioning unit) information for initialization, and then fuses the IMU (inertial measurement unit, including three single-axis accelerometers and three single-axis gyroscopes) information with the wheel speed meter information to calculate the odometer information of the vehicle's position and posture;
[0067] The mapping module receives the perception results and odometer information from the visual perception module and performs semantic SLAM mapping;
[0068] The map storage and update module will store the trajectory and SLAM map after the map is built. When storing the basic SLAM map and multiple newly created SLAM maps for the same memory parking scene, the map storage and update module will match the multiple maps and optimize the basic SLAM map based on the matching judgment results, remove dynamic objects, and supplement static scenes;
[0069] The positioning module receives the information input by the visual perception module and the sensor information, and performs pose initialization when the positioning process is just started. After obtaining the initial pose, it performs pose estimation, and then performs visual positioning to match the visual perception results with the SLAM map to obtain the pose information of the current frame. Finally, the pose is optimized through pose fusion to obtain a smoother pose.
[0070] like Figure 7 As shown, while positioning, the mapping module will be started synchronously to perform SLAM mapping on the scene again, and the newly created SLAM map will be sent to the map storage and update module for storage and matching update.
[0071] Furthermore, the system state machine module will convert the control instructions input by the driver, such as memory parking mapping, memory parking, map management (map deletion, map matching, map renaming, etc.), memory parking, etc., into system state machine information, and start the corresponding thread according to the state machine information, and update the mapping and positioning status according to the progress of the thread.
[0072] Furthermore, the visual perception module is composed of four surround-view cameras, which collect environmental information around the vehicle and perform distortion correction, coordinate conversion and image stitching to form a surround-view stitching map around the vehicle, and then input the stitching map into the deep learning model to extract 2D point cloud information of parking spaces, lane lines, arrows, speed bumps, no-parking areas, storage location numbers, etc.
[0073] Furthermore, the odometer module receives RTK information, completes positioning initialization, and then integrates the acceleration information and angular velocity information provided by the IMU to calculate the displacement, speed and rotation angle at the current moment, and performs ESKF (error state Kalman filter) fusion processing with the measurement data of the wheel speed meter to obtain the corrected position and attitude information.
[0074] Furthermore, the mapping module uses the posture provided by the odometer as the initial posture, and maps the point cloud features to the global coordinate system according to the initial posture. As the vehicle travels, the first detected frame is added to the sub-map as a key frame, and then the subsequently detected key frame point cloud information is matched with the sub-map, and the matched key frame is added to the sub-map. When the added key frame reaches the set threshold (calibrable), the next sub-map is constructed; and the accumulated error is reduced by matching between frames and sub-maps, sub-maps and sub-maps, and sub-maps and global maps, and the final global map and optimized trajectory are obtained through back-end optimization and loop detection.
[0075] The sub-map construction specifically includes adding data, pose prediction, local map scanning and matching, sub-map construction and grid map update.
[0076] Furthermore, the global map is obtained by calculating the constraint relationships between all keyframe nodes, keyframes and subgraphs, and subgraphs, and finally optimizing the pose graph to make the poses of all nodes reach the global optimum. Each node in the subgraph will establish intra-subgraph constraints with the subgraph in which it is located, using the Correlative Scan Matching (CSM) algorithm. Between different subgraphs, nodes will construct inter-subgraph constraints with subgraphs that have ended but do not belong to them, using the branch and bound method.
[0077] Furthermore, the positioning module receives sensor data and perception results to give the position information of the current frame, matches it with the loaded basic SLAM map, and performs multi-sensor fusion to finally obtain the precise posture information of the vehicle. In addition, when the positioning module performs relocation, the mapping module will be simultaneously turned on. The mapping module will splice the detected single-frame semantic information according to the relocated posture, perform back-end optimization and loop detection, and build a new SLAM map for the original scene; each relocation will create a corresponding SLAM map;
[0078] Furthermore, when the positioning mode is turned on, the secondary mapping for the scene is turned on synchronously to build a new SLAM map for the scene. When the first positioning is performed, the detected key frame is matched with the basic SLAM map. When the detected key frame matches the key frame i in the basic SLAM map, a label keyi is recorded for the detected key frame. If the frame is the first matching frame, the frame is added as the first frame to the new SLAM map. Figure 1 If the frame is not the first matching frame, the frame is matched with the sub-map of the newly created SLAM map and the pose is optimized and inserted into the newly created SLAM map. Figure 1 In the sub-graph, until the positioning is completed and the global optimization is completed to build a new SLAM map Figure 1In each subsequent positioning, the label keyi is recorded for each positioning key frame that matches the key frame i in the basic SLAM map and inserted into the corresponding new SLAM map.
[0079] Further, the map storage update module will allocate a storage address for the first SLAM map, that is, the basic SLAM map, when receiving the state machine mapping instruction. When the subsequent relocation is performed based on the basic SLAM map, the newly built SLAM map will also be stored at this address. When the number of newly built SLAM maps reaches a set threshold (which can be calibrated), it is matched with the basic SLAM map. When a map element in a key frame of the basic SLAM map does not appear in the key frame of the corresponding position in the newly built SLAM map for multiple times (the number of times can be calibrated), it is considered that the element is a dynamic object and needs to be eliminated, and the map information in the corresponding key frame of the newly built SLAM map is updated to the basic SLAM map to eliminate the dynamic object. When the same map element that is not in the key frame of the corresponding position of the basic SLAM map appears in the key frame of the same position in the newly built multiple SLAM maps, it is considered that the element is a static object, but it was missed when the basic SLAM map was built and needs to be added, then the corresponding map information in the newly built SLAM map is added to the basic SLAM map. The map storage update module stores one basic SLAM map for a memory route, as well as multiple newly created SLAM maps (the specific number can be calibrated) subsequently. When a new SLAM map is stored and the capacity in the storage address exceeds the threshold, the map with the earliest date in the stored new SLAM maps is deleted;
[0080] Furthermore, the map management module will allocate a storage address for each basic SLAM map and match and optimize the basic SLAM map with the corresponding newly created SLAM map;
[0081] When performing map matching specifically, when the number of newly created SLAM maps reaches a threshold (calibrable), the key frames with corresponding labels in the newly created SLAM map are searched for each key frame in the basic SLAM map and filtered out, and all these filtered out label key frames are matched with the corresponding key frames in the basic SLAM map to confirm whether there is point cloud information that appears in the key frames of the basic SLAM map but does not appear in the filtered label key frames for multiple times (calibrable). If so, it is considered that the point cloud information is a dynamic object that needs to be eliminated, and the key frame is deleted from the basic SLAM map. At the same time, the label key frame without the point cloud information is matched with the sub-map where the key frame of the basic SLAM map is located, and inserted into the sub-map of the basic SLAM map with the optimized posture. If it is confirmed during matching that there are key frames of the basic SLAM map that do not appear, but there is point cloud information that appears multiple times (calibrable) in the filtered label key frames and can be matched, if there is, then the point cloud information is considered to be a static object and needs to be supplemented. The key frame is deleted from the basic SLAM map, and the label key frame with the point cloud information is matched with the sub-map where the key frame of the basic SLAM map is located, and inserted into the sub-map of the basic SLAM map with the optimized posture.
[0082] Embodiment 3: Based on the same inventive concept, this embodiment further provides a vehicle electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the memory parking map optimization method as described above when executing the program.
[0083] Embodiment 4: Based on the same inventive concept, this embodiment further provides a manual-automatic vehicle, wherein the vehicle is provided with the vehicle electronic device as described above.
[0084] Furthermore, all parts involved in this application that are not described in detail are the same as the prior art or are implemented using the prior art.
[0085] In summary:
[0086] 1. The present invention proposes a memory parking map optimization method, which extracts 2D point cloud information based on the original image provided by the surround-view camera, constructs a basic SLAM map, and continuously constructs a new SLAM map during the memory parking positioning process, and compares and matches it with the basic SLAM map to eliminate dynamic objects and supplement static scenes, gradually forming a pure scene map of the parking lot on the vehicle side, which can effectively solve the impact of the dynamic environment on the positioning accuracy during the memory parking process, thereby improving the positioning accuracy of the vehicle's memory parking process, increasing the reliability and safety of parking, and enhancing customers' trust in the memory parking function.
[0087] 2. The present invention can form a data closed loop on the vehicle side by performing secondary mapping and vehicle-side map matching optimization while positioning on the vehicle side. At the same time, based on the existing visual perception solution, the present invention can eliminate the influence of dynamic objects and supplement the missed static objects, thereby reducing the demand for perception without increasing costs.
[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0092] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A memory parking map optimization method, characterized in that: include: Acquire an original image of the vehicle's surroundings, and extract location information and point cloud information of road signs from the original image; The position information of the vehicle is calculated based on the initial positioning information, acceleration information, angular velocity information and wheel speed meter information of the vehicle; Based on the location information, the point cloud information of the road signs and the vehicle posture information, a basic SLAM map is generated by using a subgraph construction method; During the memory parking positioning process, a new SLAM map is constructed in real time, and compared and matched with the basic SLAM map in real time to eliminate dynamic objects in the basic SLAM map and supplement static scenes to achieve optimization and update of the SLAM map.
2. The method for optimizing a memory parking map according to claim 1, characterized in that: The original image of the vehicle's surrounding environment is acquired through a plurality of vehicle-mounted surround view cameras, and the original image is subjected to distortion correction and coordinate transformation through a vehicle-mounted visual perception module. The transformed images are stitched together to form a surround view stitching map of the vehicle's surrounding environment, and the storage location information and point cloud information of road signs are extracted from the surround view stitching map.
3. The method for optimizing a memory parking map according to claim 1, characterized in that: The vehicle initialization positioning information is obtained by receiving RTK positioning information through the on-board odometer module. At the same time, the on-board odometer module calculates the displacement, speed and rotation angle of the vehicle at the current moment according to the vehicle's acceleration information and angular velocity information, and combines it with the measurement data of the vehicle's wheel speed meter to infer the corrected vehicle position and posture information.
4. The method for optimizing a memory parking map according to claim 1, characterized in that: The generation of the basic SLAM map includes: the on-board mapping module maps the point cloud information features of the storage location information and the road signs to the global coordinate system based on the initially received self-vehicle posture information, and adds the first detected frame as a key frame to the sub-map, and then matches and corrects the posture of the current frame point cloud in the sub-map according to the subsequently received self-vehicle posture information, and inserts the point cloud with the corrected posture into the sub-map, and ends the sub-map construction when the number of point cloud insertions reaches a preset value.
5. The method for optimizing a memory parking map according to claim 1, characterized in that: The basic SLAM map is stored in a preset storage address of the on-board map storage and update module, and the newly created SLAM map subsequently generated by repositioning based on the basic SLAM map is also stored in the preset storage address. When the storage quantity of the newly created SLAM map reaches a set threshold value one, the stored newly created SLAM map is compared and matched with the basic SLAM map.
6. The method for optimizing a memory parking map according to claim 5, characterized in that: The comparison and matching between the newly created SLAM map and the basic SLAM map includes: When a map element in a key frame of the basic SLAM map does not appear in the key frame of the corresponding position in the subsequent newly created SLAM map for more than a preset number of times, the element is considered to be a dynamic object and needs to be removed, and the map information in the corresponding key frame of the newly created SLAM map is updated to the basic SLAM map to remove the dynamic object; When the same map element appears in the key frames of the same position in multiple newly created SLAM maps but is not in the key frames of the corresponding position in the basic SLAM map, the element is considered to be a static scene and needs to be added. The map information in the corresponding key frames of the newly created SLAM map is updated to the basic SLAM map to add the static scene.
7. The method for optimizing a memory parking map according to claim 5, characterized in that: When the storage quantity of the newly created SLAM maps exceeds a set threshold value of two, the newly created SLAM maps at the front are deleted in sequence, and the set threshold value of two is greater than the set threshold value of one.
8. The method for optimizing a memory parking map according to claim 4, characterized in that: When the memory parking positioning is turned on, the basic SLAM map is loaded as a priori information, and the point cloud information in each sub-map construction key frame is extracted to match with the priori information, and the vehicle posture information at the current frame moment is corrected according to the matching result.
9. A vehicle electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the memory parking map optimization method according to any one of claims 1 to 8 is implemented.
10. A manual-automatic vehicle, characterized in that: The vehicle electronic device comprising the vehicle electronic device according to claim 9.
Citation Information
Cited By
Unmanned aerial vehicle autonomous positioning navigation system and method based on visual SLAM
CN120063285A
Valet parking method and device, storage medium and computer program product
CN120863616A
Parking evaluation method and system, electronic equipment and vehicle
CN121415621A
