Lego Loam dynamic point filtering method and system based on image segmentation

Abstract

The invention discloses a Lego Loam dynamic point filtering method and system based on image segmentation, and belongs to the field of unmanned vehicle mapping and positioning. The method comprises the following steps: segmenting an original image by using an improved YOLOv11 network to obtain a series of segmentation masks; projecting a three-dimensional point cloud of the laser radar to a two-dimensional image plane according to a relative position relationship between the radar and a camera, screening out projection points falling in the segmentation mask, back-projecting the projection points to an original point cloud, expanding the projection points, and removing the projection points from the original point cloud to obtain a point cloud after dynamic point filtering; according to the obtained filtered point cloud, a Lego Loam algorithm is adopted for mapping, pose data are recorded, and a point cloud map is stored. Dynamic point filtering is carried out, the mapping and positioning effects are improved, and the method is particularly suitable for mapping and positioning of the unmanned vehicle in a complex dynamic scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned vehicle mapping and positioning, and particularly relates to a LegoLoam dynamic point filtering method and system based on image segmentation. Background Art

[0002] Mapping and localization technology for autonomous vehicles is a core pillar of both autonomous driving and vehicle navigation systems, addressing the critical questions of "where am I" and "what is the environment like?" With the rapid development of autonomous driving technology, this field has evolved from simple dead reckoning and marker-based navigation in its early stages to high-precision solutions that integrate multiple sensors, artificial intelligence, and cloud computing. Modern autonomous vehicle systems typically utilize multiple sensors, including lidar, cameras, inertial measurement units (IMUs), and GNSS (Global Navigation Satellite Systems), for precise localization. In recent years, the introduction of deep learning technology has further fueled innovative developments in semantic mapping and end-to-end localization. Furthermore, the implementation of concepts such as high-precision maps and vehicle-infrastructure collaboration has provided new approaches for reliable localization in complex scenarios. However, facing challenges such as urban canyons, inclement weather, and dynamic environments, as well as the practical demands of cost control and standardization for commercial applications, this field still faces numerous technical challenges that require resolution. In the future, with the deep integration of emerging technologies such as 5G communications and neural rendering with traditional positioning methods, unmanned vehicle mapping and positioning technology will continue to develop in a smarter, more robust and more economical direction, laying a solid foundation for the large-scale commercial application of autonomous driving.

[0003] SLAM (Simultaneous Localization and Mapping) technology, involved in mapping and localization for autonomous vehicles, is a core foundation for robotic perception and interaction with the environment. Its development is closely tied to advances in fields such as autonomous driving, augmented reality, and service robotics. This technology originated in robotic localization research in the 1980s, initially relying primarily on ultrasonic and simple laser sensors for basic navigation in structured environments. In the early 2000s, breakthroughs in computer vision and lidar technologies spurred rapid development of SLAM, giving rise to two major approaches: filter-based SLAM (such as EKF-SLAM) and graph optimization-based SLAM (such as PTAM). After 2010, the widespread adoption of RGB-D cameras and solid-state lidars fueled the rapid growth of visual SLAM (such as the ORB-SLAM series) and laser SLAM (such as the LOAM series). Simultaneously, the emergence of semantic SLAM enabled robots to understand the semantics of their environment. In recent years, the integration of deep learning and SLAM has fostered end-to-end localization and mapping methods. The introduction of technologies like multi-sensor fusion and edge computing has significantly enhanced the robustness of SLAM systems in complex and dynamic environments. SLAM technology is currently developing towards high precision, lightweight, and semantic features, showing broad application prospects in areas such as autonomous driving, drones, AR / VR, and industrial robotics. However, it still faces key challenges such as long-term stability, adaptability to dynamic environments, and computational efficiency.

[0004] Existing mapping and positioning technologies for autonomous vehicles mostly rely on Euclidean clustering or point cloud segmentation to filter out dynamic obstacles. However, these methods present the following problems: First, some dynamic obstacles cannot be filtered out using pure point cloud segmentation or clustering. Second, a single LiDAR sensor cannot convey visual information, hindering subsequent pedestrian perception and planning. Third, existing LiDAR algorithms lack dynamic point filtering, making them prone to inaccurate positioning and map ghosting in certain scenarios. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a Lego Loam dynamic point filtering method and system based on image segmentation. On the basis of accurate image segmentation, dynamic point filtering is performed by projecting a three-dimensional point cloud onto a two-dimensional image plane, thereby improving the filtering accuracy of detected objects in the scene and improving the efficiency and accuracy of Lego Loam mapping and positioning.

[0006] Technical solution: In order to achieve the above invention objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, a Lego Loam dynamic point filtering method based on image segmentation comprises the following steps:

[0008] The original image is segmented using a modified YOLOv11 network to obtain a series of segmentation masks. The modified YOLOv11 network introduces a flattened attention mechanism before the convolutional layers of the feature extraction network and each detection head of the original YOLOv11 network, removes positional encoding, and reduces the scaling of the feature dimensions after linear expansion.

[0009] Based on the relative positional relationship between the radar and the camera, the 3D point cloud of the lidar is projected onto the 2D image plane. The projected points that fall within the segmentation mask are screened out. These projected points are then back-projected onto the original point cloud and expanded before being removed from the original point cloud to obtain the point cloud after dynamic point filtering.

[0010] According to the obtained filtered point cloud, the Lego Loam algorithm is used to build the map, record the pose data, and save the point cloud map.

[0011] Furthermore, the original image is segmented using an improved YOLOv11 network to obtain a series of segmentation masks, wherein the improved YOLOv11 network introduces a flattened attention mechanism before the feature extraction network and the convolutional layer of each detection head of the original YOLOv11 network, removes the position encoding, and reduces the scaling of the feature dimensions after the linear layer expansion;

[0012] Based on the relative positional relationship between the radar and the camera, the 3D point cloud of the lidar is projected onto the 2D image plane. The projected points that fall within the segmentation mask are screened out. These projected points are then back-projected onto the original point cloud and expanded before being removed from the original point cloud to obtain the point cloud after dynamic point filtering.

[0013] According to the obtained filtered point cloud, the Lego Loam algorithm is used to build the map, record the pose data, and save the point cloud map.

[0014] Furthermore, a flat attention mechanism is introduced into the feature extraction network of the original YOLOv11 network, including:

[0015] Add a flat attention mechanism after the key transition layer of the backbone network;

[0016] Apply flat attention mechanism on the feature fusion path of PANet.

[0017] Furthermore, the scaling ratio of the feature dimension after linear expansion is obtained by dividing the hidden layer dimension by the input dimension.

[0018] Furthermore, based on the relative positional relationship between the radar and the camera, the three-dimensional point cloud of the lidar is projected onto a two-dimensional image plane, including:

[0019] Let the 3D point cloud be pointcloud 3d ={(x0,y0,z0),(x1,y1,z1),...,(x i ,y i ,z i ),...,(x n-1 ,y n-1 ,z n-1 )}, where there are n three-dimensional point clouds, which are transformed into two-dimensional projection points cvpoint through the rotation projection of the relative external parameters of the camera and the lidar and the camera internal parameters 2d ={(tx0,ty0),(tx1,ty1),...,(tx i ,ty i ),...,(tx n-1 ,ty n-1 )}, where the rotation matrix in the external parameter is set to R, the translation matrix is ​​set to T, the fx and fy in the internal parameter matrix represent the focal length of the camera in the x-axis and y-axis directions, and cx and cy represent the intersection of the optical axis and the image plane. The calculation process is expressed as:

[0020]

[0021] Furthermore, these projected points are back-projected to the original point cloud and expanded before being removed from the original point cloud, including:

[0022] The mask S i The two-dimensional projection points in the image are back-projected to the original point cloud, and the back-projected original point cloud is expanded by double its volume in the three-axis direction to obtain the expanded three-dimensional point cloud. The expanded three-dimensional point cloud is culled, and the RGB value of the image is assigned to the filtered three-dimensional point cloud, and it is colored to obtain a colored point cloud. The formula is as follows:

[0023] filterpointcloud 3d =pointcloud 3d {cvpoint 2d ∈S i (i=0,1,...,n)}

[0024] Among them, filterpointcloud 3d is the filtered 3D point cloud.

[0025] Furthermore, based on the obtained filtered point cloud, the Lego Loam algorithm is used to build a map, record the pose data, and save the point cloud map, including:

[0026] Based on the color point cloud obtained after filtering, the Lego Loam algorithm with optimized loop closure detection in the scanning context is used to construct the color point cloud map, and the obtained pose data and mapping point cloud data are recorded.

[0027] Further optimizations of scanning context loop detection include:

[0028] The obtained 3D point cloud data is converted into 2D polar coordinate representation (r i ,θ i ), where r i Represents the radial distance from the point to the sensor, θ i Represents the azimuth; discretize the polar coordinates and convert (r i ,θ i ) is divided into radial and circumferential angles, and the radial angle is divided into N r intervals, and the circular division yields N θ intervals, and calculate the representative value of each azimuth-range interval, and handle the rotation change by column circular shift.

[0029] In the second aspect, a Lego Loam dynamic point filtering system based on image segmentation includes:

[0030] An image segmentation module, which segments the original image using a modified YOLOv11 network to generate a series of segmentation masks. The modified YOLOv11 network introduces a flattened attention mechanism before the convolutional layers of the feature extraction network and each detection head of the original YOLOv11 network, removes positional encoding, and reduces the scaling of the features after linear expansion.

[0031] The dynamic point cloud filtering module is used to project the 3D point cloud of the lidar onto the 2D image plane based on the relative position relationship between the radar and the camera, filter out the projected points that fall within the segmentation mask, back-project these projected points onto the original point cloud, expand them, and then remove them from the original point cloud to obtain the point cloud after dynamic point filtering;

[0032] The mapping and positioning module is used to build a map based on the filtered point cloud using the Lego Loam algorithm, record the pose data, and save the point cloud map.

[0033] In a third aspect, an electronic device comprises: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the LegoLoam dynamic point filtering method based on image segmentation as described in the first aspect of the present invention are implemented.

[0034] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the Lego Loam dynamic point filtering method based on image segmentation as described in the first aspect of the present invention.

[0035] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) Compared with the existing methods of dynamic point filtering based on point cloud segmentation or Euclidean clustering, the present invention adopts a method of dynamic point cloud filtering based on image segmentation of camera images, which has a significant improvement in accuracy and can filter almost all detected objects in the scene.

[0037] (2) Compared with Lego Loam's method of using only three-dimensional laser point clouds for mapping, the color map constructed by the present invention using a camera plus a three-dimensional laser point cloud has more information and content than other maps, which is conducive to subsequent perception and planning.

[0038] (3) Compared with the previous laser SLAM algorithm, the present invention filters out dynamic obstacles, can obtain more accurate matching between point cloud frames, and adds improvements on loop closure, making the positioning posture more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Flow chart of the method of the present invention;

[0040] Figure 2 This is a schematic diagram of the improvement of YOLOv11 using Flat Attention in the present invention;

[0041] Figure 3 Schematic diagram of image segmentation using the YOLOv11 segmentation network in the present invention;

[0042] Figure 4 Schematic diagram comparing point clouds before and after filtering in the present invention;

[0043] Figure 5 This is a schematic diagram comparing the point cloud maps of the original Lego Loam algorithm and the optimized Lego Loam algorithm used in the present invention;

[0044] Figure 6 Schematic diagram of the trajectory before and after errors of trajectory evaluation using EVO in the present invention;

[0045] Figure 7 Schematic diagram of trajectory evaluation APE using EVO in the present invention;

[0046] Figure 8Schematic diagram of using EVO to evaluate RPE trajectory in the present invention. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0048] Lego Loam stands for "Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain." It is a SLAM algorithm designed specifically for ground vehicles, aiming to solve the problem of real-time six-degree-of-freedom pose estimation for ground vehicles in complex environments. The algorithm is lightweight and can achieve real-time pose estimation in low-power embedded systems. However, the algorithm also suffers from the three issues described in the background art: dynamic obstacles cannot be removed through pure point cloud segmentation or clustering filtering. The Lego Loam algorithm focuses primarily on real-time lidar odometry and mapping, and does not have a complex point cloud segmentation or clustering filtering mechanism specifically designed for dynamic obstacles. A single lidar sensor cannot convey visual information. The Lego Loam algorithm operates based on a lidar sensor, which primarily acquires distance information from the environment by emitting and receiving laser beams, thereby constructing a point cloud map. The Lego Loam algorithm, which uses only a single lidar sensor, has limitations in pedestrian perception and planning. The lack of dynamic point filtering can easily lead to inaccurate positioning and map ghosting. In scenes with dynamic objects, dynamic points can interfere with the lidar's perception and matching of the environment. When these dynamic points are included in the positioning and mapping calculations, the algorithm can miscalculate the autonomous vehicle's pose, resulting in inaccurate positioning.

[0049] The embodiment of the present invention provides a Lego Loam dynamic point filtering method based on image segmentation, which attempts to solve the above problems. In this method, image segmentation uses an improved YOLOv11 network, and Lego Loam dynamic point filtering is implemented using an optimized Lego Loam algorithm. The improved YOLOv11 network, based on the original YOLOv11 network, introduces a flat attention module mechanism and removes position encoding, etc., to balance the computational overhead of the attention mechanism and the feedforward network, improve the overall performance of the network, and achieve accurate segmentation of a given image. The optimized Lego Loam algorithm performs a two-dimensional projection on the laser three-dimensional point cloud, expands the two-dimensional projection points that fall within the YOLOv11 segmentation detection frame, and back-projects them to the original point cloud. The points are then removed from the original point cloud to obtain a point cloud after dynamic point filtering; based on the obtained filtered point cloud, the Lego Loam algorithm is used to build a map, record the pose data, and save the point cloud map.

[0050] Figure 1 For the overall flow chart of the method of the present invention, refer to Figure 1 , a Lego Loam dynamic point filtering method based on image segmentation, specifically comprising the following steps:

[0051] Step S1: Improve the YOLOv11 segmentation network and perform target detection.

[0052] First, we improve the original YOLOv11 network by introducing a flattened attention mechanism in the feature extraction network and detection head structure. This mechanism is added after key transition layers in the backbone network, such as after the residual block in CSPDarknet. Secondly, we apply the flattened attention mechanism to the feature fusion path of PANet, helping the model better select useful features across scales. Finally, we add the flattened attention mechanism before the convolutional layer of each detection head. To enhance the representation of key features, we use a simpler single-head or reduced-head design for computational efficiency. By reducing the attention dimension, we reduce computational overhead and are more suitable for real-time object detection tasks. While maintaining global information exchange capabilities, we prioritize the enhancement of local features. Removing explicit position encoding reduces model parameters and computational complexity. For adaptability to segmentation tasks, we prioritize relative position over absolute position.

[0053] Figure 2This is a schematic diagram of the improvement of YOLOv11 using Flat Attention in the present invention. The fully connected layer in the channel dimension is removed, and a 1×1 convolution is used to directly generate a spatial weight map (with the resolution remaining H×W), and pixel-level importance weighting is performed on the multi-scale features of P3 / P4 / P5. And cross-scale feature fusion is enhanced. A FlatAttention module is embedded at the up-sampling and down-sampling intersection of PANet, and through three-step processing of feature concatenation (Concat) → channel compression (1×1Conv) → spatial activation (Sigmoid), dynamic fusion of features at different levels is achieved.

[0054] Secondly, in the CNN module (convolution operation module), the scaling ratio of the dimension after the feature is expanded by the linear layer is defined as MLP, and the expression form is as follows:

[0055]

[0056] The input is a list of multi-scale feature maps [F1, F2, F3], the features are concatenated and compressed in channels by 1×1 convolution, and the weighted feature map generated by the spatial weight is equal to the product of the input feature and the spatial weight. By reducing MLP, such as from 4 to 1.2, the model convergence speed can be improved, the training speed can be increased, and thus the performance of the overall network can be enhanced.

[0057] On the public dataset DAIR-V2X, the original network and the improved network are used to train the dataset respectively, and each parameter and performance index are optimized. At the same time, the improved network is used to infer the original pictures to obtain a series of two-dimensional detection masks. It is set that there are n two-dimensional detection masks in the detection results, and the i-th (i < n) mask Si is an irregular graph composed of multiple two-dimensional pixel points corresponding to n detection targets.

[0058] Step S2: Use the relative position relationship between the lidar and the camera to perform projection to obtain two-dimensional projection points and perform dynamic point cloud filtering.

[0059] According to the relative position relationship between the radar and the camera, the three-dimensional point cloud of the lidar is projected onto a two-dimensional picture. Let a series of three-dimensional point clouds be pointcloud 3d ={(x0,y0,z0),(x1,y1,z1),...,(x i ,y i ,z i ),...,(x n-1 ,y n-1 ,z n-1 )} where there are n three-dimensional point clouds, and through the rotation projection of the relative external parameters of the camera (i.e., the camera) and the lidar and the camera internal parameters, they are transformed into a series of two-dimensional projection points cvpoint 2d={(tx0,ty0),(tx1,ty1),...,(tx i ,ty i ),...,(tx n-1 ,ty n-1 )}. The rotation matrix in the external parameter is set to R, the translation matrix is ​​set to T, and the internal parameter matrix fx and fy represent the focal length of the camera in the x-axis and y-axis directions (in pixels). cx and cy represent the intersection of the optical axis and the image plane. The specific calculation process is:

[0060]

[0061] Delete the two-dimensional projection points in the mask. Specifically, back-project the two-dimensional projection points in the mask Si to the original point cloud, and expand the back-projected three-dimensional point cloud by double its volume in the three-axis direction to obtain the expanded three-dimensional point cloud. Remove the expanded three-dimensional point cloud to obtain the filtered three-dimensional point cloud, which is recorded as filterpointcloud 3d ={(x0,y0,z0),(x1,y1,z1),...,(x i ,y i ,z i ),...,(x m-1 ,y m-1 ,z m-1 )}, where the filtering algorithm filters n point clouds into m point clouds, and assigns the RGB value of the image to the filtered three-dimensional point cloud, and colors it to obtain a colored point cloud. The formula is as follows:

[0062] filterpointcloud 3d =pointcloud 3d {cvpoint 2d ∈S i (i=0,1,...,n)} (3)

[0063] CVPoint 2d The RGB value corresponding to the corresponding two-dimensional image pixel is assigned to filterpointcloud 3d , get the three-dimensional color point cloud filterpointcloud 3d .

[0064] Step S3: construct a three-dimensional point cloud map for the filtered point cloud and record the pose.

[0065] The present invention adds scan context loop detection to Lego Loam, which specifically includes the following steps:

[0066] (1) The time-synchronized images and point cloud folders are read separately and published on topics using ROS (Robot Operationg System). The obtained YOLOv11 segmentation model is lightweight processed and converted into ONNX (Open Neural Network Exchange). The topic image is received and reasoned on to obtain a series of segmentation masks, and then the topic is published using std_msgs.

[0067] (2) After receiving the radar point cloud topic, it is also projected to obtain a series of two-dimensional projection points for topic publishing. The above two are received and compared to finally obtain the filtered three-dimensional point cloud, and the topic / color_pointcloud is published as the mapping point cloud input of Lego Loam.

[0068] (3) Convert the obtained 3D point cloud data into 2D polar coordinate representation and calculate the representative value of each azimuth-distance interval (such as maximum height, etc.) to ensure the stability of the loop. Suppose a frame of 3D point cloud data is given where p i =(x i ,y i ,z i ), first convert the obtained point cloud into polar coordinates (r i ,θ i ), where r i Represents the radial distance from the point to the sensor, θ i represents the azimuth, and the calculation formula is as follows:

[0069]

[0070] Secondly, the polar coordinates are discretized and (r i ,θ i ) is divided into radial and circumferential angles, which are divided into N r The radial division formula is as follows:

[0071] r i ∈[0,r max ](5)

[0072] Divide the ring to get N θ intervals, and finally obtain the polar coordinates (i r ,i θ ) is calculated as follows:

[0073]

[0074] For each grid cell (i r ,i θ), calculate its maximum height or other statistics, such as average height:

[0075] φ(i r ,i θ )=max z(p){p∈(i r ,i θ )} (7)

[0076] The above formula is the Scan Context, which is characterized by rotation invariance: Column-wise Circular Shift handles rotation changes. Height information is encoded to record the maximum height stored in each unit, enhancing scene recognition. Because lidar rotation causes column offsets in the Scan Context, rotation alignment is required, namely column-wise shift matching and similarity calculation. The steps for column-wise shift matching are:

[0077] D(I q ,I c )=min||I q -shift(I c ,k)||2{k∈[0,N θ-1 ]} (8)

[0078] Among them, shift(I c ,k) means I c The columns of cyclic shift k times, the steps to calculate the similarity score are:

[0079]

[0080] Among them, D max is the normalization factor.

[0081] The processed point cloud data is then used as the control input of the improved Lego Loam to construct the point cloud map and estimate the positioning pose, and the map and pose values ​​are saved.

[0082] Step S4: perform EVO trajectory evaluation on the original mapping pose and the mapping pose processed by dynamic point cloud.

[0083] The saved original trajectory and the optimized one are converted into TUM format data, and then the evo_ape command is used to evaluate the absolute pose error, evo_rpe is used to evaluate the relative pose error, and evo_traj is used to draw the trajectory, output the trajectory file, and convert the data format, indicating the errors of the trajectory in the xyz and rpy (roll, pitch, yaw) directions before and after optimization.

[0084] In the embodiment of the present invention, the improved YOLOv11 network is used to reason about the image. Figure 3As shown, the original picture ( Figure 3 (a) in the figure and the image result after inference ( Figure 3 (b) in the figure, where the colored part of the image after inference is a series of 2D masks obtained by segmentation. The point cloud is projected two-dimensionally according to the relative position relationship between the point cloud and the image, and the three-dimensional point cloud is filtered at the same time. The result is as follows Figure 4 As shown, Figure 4 (a) in the figure is the original point cloud. Figure 4 (b) is the filtered point cloud. It can be observed that the point cloud of the vehicle in the scene is basically filtered out. The color point cloud is mapped and the following is obtained: Figure 5 The results shown are as follows: the upper figure (a) is the original point cloud, and the lower figure (b) is the point cloud after optimization. At the same time, the trajectory error is obtained, as shown in Figure 6 、 7 , as shown in 8, Figure 6 The upper middle figure (a) shows a comparison of the trajectories generated by the original Lego Loam algorithm, the optimized Lego Loam algorithm, and the groundtruth trajectory. The middle figure (b) shows a comparison of the three trajectories in roll, pitch, and yaw. The lower figure (c) shows the differences in the three trajectories in position x, y, and z. Figure 7 The upper middle figure (a) is a schematic diagram of the absolute pose error of the SLAM algorithm before optimization and the groundtruth trajectory. The lower middle figure (b) is a schematic diagram of the absolute pose error of the SLAM algorithm after optimization and the groundtruth trajectory. The RPE before optimization is ±1.77, and the RPE after optimization is ±0.14. Figure 8 The upper middle figure (a) shows the relative pose error between the SLAM algorithm before optimization and the groundtruth trajectory. The lower middle figure (b) shows the relative pose error between the SLAM algorithm and the groundtruth trajectory after optimization. The APE before optimization is ±0.257, and the APE after optimization is ±0.215. It can be seen that the absolute pose error after optimization is better than before optimization in both position deviation and relative pose error, demonstrating the effectiveness of the proposed method.

[0085] Based on the same technical concept as the method embodiment, another embodiment of the present invention provides a Lego Loam dynamic point filtering system based on image segmentation, including:

[0086] An image segmentation module, which segments the original image using a modified YOLOv11 network to generate a series of segmentation masks. The modified YOLOv11 network introduces a flattened attention mechanism before the convolutional layers of the feature extraction network and each detection head of the original YOLOv11 network, removes positional encoding, and reduces the scaling of the features after linear expansion.

[0087] The dynamic point cloud filtering module is used to project the 3D point cloud of the lidar onto the 2D image plane based on the relative position relationship between the radar and the camera, filter out the projected points that fall within the segmentation mask, back-project these projected points onto the original point cloud, expand them, and then remove them from the original point cloud to obtain the point cloud after dynamic point filtering;

[0088] The mapping and positioning module is used to build a map based on the filtered point cloud using the Lego Loam algorithm, record the pose data, and save the point cloud map.

[0089] It should be understood that the Lego Loam dynamic point filtering system based on image segmentation in the embodiment of the present invention can implement all the technical solutions in the above method embodiment, and the functions of its various functional modules can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can refer to the relevant description in the above embodiment, and will not be repeated here.

[0090] The present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the Lego Loam dynamic point filtering method based on image segmentation as described above are implemented.

[0091] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the Lego Loam dynamic point filtering method based on image segmentation are implemented as described above.

[0092] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices (systems), electronic devices, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] The present invention is described with reference to flowcharts of methods according to embodiments of the present invention. It should be understood that each process in the flowcharts and combinations of processes in the flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 A device that specifies functions in a process or multiple processes.

[0094] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.

Claims

1. A Lego Loam dynamic point filtering method based on image segmentation, characterized in that: The following steps are involved: The original image is segmented using a modified YOLOv11 network to obtain a series of segmentation masks. The modified YOLOv11 network introduces a flattened attention mechanism before the convolutional layers of the feature extraction network and each detection head of the original YOLOv11 network, removes positional encoding, and reduces the scaling of the feature dimensions after linear expansion. Based on the relative positional relationship between the radar and the camera, the 3D point cloud of the lidar is projected onto the 2D image plane. The projected points that fall within the segmentation mask are screened out. These projected points are then back-projected onto the original point cloud and expanded before being removed from the original point cloud to obtain the point cloud after dynamic point filtering. According to the obtained filtered point cloud, the Lego Loam algorithm is used to build the map, record the pose data, and save the point cloud map.

2. The method according to claim 1, characterized in that The flat attention mechanism is introduced into the feature extraction network of the original YOLOv11 network, including: Add a flat attention mechanism after the key transition layer of the backbone network; Apply flat attention mechanism on the feature fusion path of PANet.

3. The method according to claim 1, characterized in that The scaling ratio of the feature dimension after the linear layer expansion is obtained by dividing the hidden layer dimension by the input dimension.

4. The method according to claim 1, wherein Based on the relative position relationship between the radar and the camera, the 3D point cloud of the lidar is projected onto the 2D image plane, including: Let the 3D point cloud be pointcloud 3d ={(x0,y0,z0),(x1,y1,z1),...,(x i ,y i ,z i ),...,(x n-1 ,y n-1 ,z n-1 )}, where there are n three-dimensional point clouds, which are transformed into two-dimensional projection points cvpoint through the rotation projection of the relative external parameters of the camera and the lidar and the camera internal parameters 2d ={(tx0,ty0),(tx1,ty1),...,(tx i ,ty i ),...,(tx n-1 ,ty n-1 )}, where the rotation matrix in the external parameter is set to R, the translation matrix is ​​set to T, the fx and fy in the internal parameter matrix represent the focal length of the camera in the x-axis and y-axis directions, and cx and cy represent the intersection of the optical axis and the image plane. The calculation process is expressed as:

5. The method according to claim 4, characterized in that These projected points are back-projected to the original point cloud and then expanded and removed from the original point cloud, including: The mask S i The two-dimensional projection points in the image are back-projected to the original point cloud, and the back-projected original point cloud is expanded by double its volume in the three-axis direction to obtain the expanded three-dimensional point cloud. The expanded three-dimensional point cloud is culled, and the RGB value of the image is assigned to the filtered three-dimensional point cloud, and it is colored to obtain a colored point cloud. The formula is as follows: filterpointcloud 3d =pointcloud 3d {cvpoint 2d ∈S i (i=0,1,...,n)} Among them, filterpointcloud 3d is the filtered 3D point cloud.

6. The method according to claim 1, wherein Based on the filtered point cloud, the Lego Loam algorithm is used to build a map, record the pose data, and save the point cloud map, including: Based on the color point cloud obtained after filtering, the Lego Loam algorithm with optimized loop closure detection in the scanning context is used to construct the color point cloud map, and the obtained pose data and mapping point cloud data are recorded.

7. The method according to claim 6, characterized in that Scan context loop detection optimization includes: The obtained 3D point cloud data is converted into 2D polar coordinate representation (r i ,θ i ), where r i Represents the radial distance from the point to the sensor, θ i Represents the azimuth; discretize the polar coordinates and convert (r i ,θ i ) is divided into radial and circumferential angles, and the radial angle is divided into N r intervals, and the circular division yields N θ intervals, and calculate the representative value of each azimuth-range interval, and handle the rotation change by column circular shift.

8. A Lego Loam dynamic point filtering system based on image segmentation, characterized in that: include: An image segmentation module, which segments the original image using a modified YOLOv11 network to generate a series of segmentation masks. The modified YOLOv11 network introduces a flattened attention mechanism before the convolutional layers of the feature extraction network and each detection head of the original YOLOv11 network, removes positional encoding, and reduces the scaling of the features after linear expansion. The dynamic point cloud filtering module is used to project the 3D point cloud of the lidar onto the 2D image plane based on the relative position relationship between the radar and the camera, filter out the projected points that fall within the segmentation mask, back-project these projected points onto the original point cloud, expand them, and then remove them from the original point cloud to obtain the point cloud after dynamic point filtering; The mapping and positioning module is used to build a map based on the filtered point cloud using the Lego Loam algorithm, record the pose data, and save the point cloud map.

9. An electronic device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the Lego Loam dynamic point filtering method based on image segmentation are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the Lego Loam dynamic point filtering method based on image segmentation are implemented as described in any one of claims 1 to 7.

Images