Method for solving post-fusion space-time discontinuity of automatic driving sensor

Abstract

The invention discloses a method for solving post-fusion space-time discontinuity of an automatic driving sensor, and belongs to the technical field of automobile automatic driving, and the method comprises the steps: extracting a target object of each image based on Res-Net50, carrying out the modeling of a part with low continuity through an L2, 1 norm, separating the target object from a background, fusing a feature target recognized by a CNN, and constructing a space-time tensor cube; carrying out iterative refinement by utilizing residual learning, retaining credible information in the input part of point clouds, carrying out adjustment and improvement in combination with a PRN network structure, copying the original part of point clouds, carrying out next effective transmission, reconstructing incomplete point clouds, and ensuring the stability and completeness of a space structure during later fusion; based on the accurate depth estimation of the LiDAR and the feature tracking capability of a camera, the SLAM-LIMO method is utilized to collect the joint optimization of multi-frame image data and the depth of a feature point obtained by the laser point cloud, and a method for extracting the depth of the image feature point cloud from the LiDAR is provided, so that the problem of fragmentation in the discontinuous space in the post-fusion time of a sensor is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile automatic driving, and in particular relates to a method for solving the spatiotemporal discontinuity of post-fusion of automatic driving sensors. Background Art

[0002] At present, in order to improve the ability of the on-board computer of the autonomous driving vehicle to understand the environment more clearly, the method of multi-sensor fusion is adopted to solve the problem of environmental perception through the mutual complementation between multiple sensors. Multi-sensor fusion is divided into three types: front, middle and back. The present invention proposes a solution to the shortcomings of one of the sensors, that is, back fusion. Each sensor performs deep learning model processing on the target object separately, so that each outputs a result with the sensor's own attributes. The recognition result of each sensor is input into the fusion module. After the fusion module is successfully associated, the fused data is uploaded to the on-board computer, and then a decision is made according to the actual situation.

[0003] However, post-sensor fusion still suffers from discontinuous perception in time and fragmented perception in space. Discontinuous perception in time: the results collected by the camera are in single-frame units. The commonly used perception method is to connect the detection results of consecutive single frames in series, which is similar to the post-fusion strategy and cannot fully utilize the useful information in time series. Fragmented perception in space: due to the installation angle problem, continuous coverage and unified recognition in the spatial domain cannot be achieved, resulting in only part of the target being captured. It is impossible to make correct detection results based on the incomplete information, making the subsequent fusion effect impossible to guarantee.

[0004] Among the many existing patents, there are many explorations of multi-sensor post-fusion technology, for example: CN116434017A is a multi-sensor post-fusion method and device based on the time series tracking results of a single sensor, but this method is still dominated by a single sensor; there is also CN118332501A, a multi-sensor perception information post-fusion method and device based on clustering and graph optimization. This technology belongs to the category of multi-sensor perception information post-fusion in the field of computer vision, and specifically involves the application of clustering and graph optimization technology. However, for multi-sensor fusion algorithms, robustness is a basic requirement. Most current post-fusion algorithms tend to be implemented using clustering methods, which sacrifices accuracy and robustness to a certain extent and has obvious limitations; in addition, there is CN117649452A, a method for rapid calibration and post-fusion of multi-sensor information in autonomous driving. Although this method solves the complex problem of calibration conversion between coordinate systems, it may not be accurate enough in obtaining target distance information, and distance information is crucial for the assessment of safe distance. In view of the unique properties of different sensors, there is currently no optimal fusion solution that can fully meet the needs. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to propose a method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors, so as to solve the problems of temporal perception discontinuity and spatial perception fragmentation mentioned in the background technology.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions: a method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors, the overall architecture of which is divided into: data acquisition, construction of time tensor cube, filling in spatial residual cloud and spatiotemporal fusion.

[0007] Data acquisition: ML-T0528-18C industrial camera is used to capture two-dimensional image information. Based on the target detection task of the image, a spatiotemporal feature tensor model with time series is proposed;

[0008] The robosence80 lidar is used to capture three-dimensional depth point cloud information to supplement the target objects that cannot be inspected in the image due to visual occlusion, insufficient light, etc.; it is proposed to use a convolutional neural network to optimize the residual information and combine it with the PRN network structure, with the aim of making the inspected point cloud show a clearer shape and less noise, providing complete point cloud details for the spatial fragmentation of the target object during the subsequent fusion.

[0009] Constructing the Time Tensor Cube: The Tensor Cube demonstrates object detection for multi-object tracking based on spatiotemporal features. The detection network consists of four main parts:

[0010] 1. Based on the Res-Net50 algorithm, a uniform size 512×512 feature map is extracted from the input data to filter out interference information and avoid redundancy;

[0011] 2. Convolution, pooling, and fully connected layers: reduce the dimension of the feature map, retain important information, classify or regress to identify the target object and predict the final output feature map with a size of 2048×16×16;

[0012] 3. Generate the target's motion trajectory by updating the target's position in consecutive frames, and connect the target's position in each frame to form a smooth curve or path;

[0013] Fourth, use the L2,1 norm to model the low-order part and construct the space-time tensor cube. Equate the target detection problem with the low-order sparse decomposition problem and fuse the target perception results with the space-time tensor decomposition.

[0014] The details of the tensor model are as follows:

[0015] Reshape the matrix into a third-order tensor, adding a minimal zero matrix to X for a given integer n2. So that:

[0016] And n3:=(h+l) / n2 is an integer;

[0017] := is an assignment operator, used to define or redefine variables;

[0018] h represents the length of the entire video stream;

[0019] l represents the length error;

[0020] The matrix Reconstructing into tensors So that:

[0021] X (k) =X(:(k-1)n2+1:kn2),k∈[n3]

[0022] Slice the array X, : means select all elements of this dimension;

[0023] The continuity of tensors and the topological information of graphs are used to represent the spatial structural and temporal relationships of images by calculating the similarity between different pixels.

[0024] Completing spatial residual point clouds: The LiDAR point cloud proposed a residual learning iterative refinement method that can take imperfect and noisy low-resolution point cloud patches as input and produce high-fidelity high-resolution point cloud patches as output, instead of using folding operation upsampling in a single shot, which may introduce noise in local areas and lose clear features. Deep learning point clouds are used to filter out noise points to robustly handle noisy and imperfect low-resolution point clouds. Finally, the PRN network is used to cut and locally obtain to obtain more complete point cloud details of the target object.

[0025] Using incomplete point cloud data as input, it is first encoded by an encoder, and then, in the process of iterative refinement, deep restoration is performed simultaneously, during which the original learned information is retained. Finally, the decoder generates an optimized output.

[0026] The PRN residual depth restoration can be expressed as:

[0027] c2=c1+c.

[0028] Space-time fusion: Based on SLAM-LIMO, the camera can capture rich visual information such as the color, texture, and shape of the object, while the radar can accurately measure the distance, speed, and angle of the object. The combination of the two can more accurately determine the driving status and intention of the vehicle;

[0029] The SLAM system process includes the following steps: 1) coordinate transformation; 2) sliding window; 3) graph optimization; 4) global closed-loop detection; 5) fusion mapping.

[0030] The beneficial effects of the present invention are as follows: a method for extracting the target object of each image based on Res-Net50 is proposed for images, and then modeling is performed using the Gamma norm and the L2,1 norm respectively to construct a tensor cube with spatiotemporal characteristics; a residual learning iterative refinement method is proposed for radar point cloud perception, and an improved SLAM-LIMO algorithm is used to collect the joint optimization of multi-frame image data and the depth of feature points obtained by the laser point cloud based on the accurate depth estimation of LiDAR and the feature tracking capability of the camera, and a method for extracting the depth of image feature point cloud from LiDAR is proposed to solve the problem of temporal discontinuity and spatial fragmentation of post-sensor fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The overall structure and principle diagram of the method of the present invention;

[0032] Figure 2 Schematic diagram of modeling principle based on Res-Net50, Gamma and L2,1 norm tensor;

[0033] Figure 3 It is a principle diagram based on the video stream matrix segmentation;

[0034] Figure 4 The schematic diagram of the third-order tensor reconstruction based on the partition matrix;

[0035] Figure 5 The schematic diagram of the residual iterative refinement structure based on the improved PRN network structure;

[0036] Figure 6 This is the multi-sensor fusion flowchart based on the improved SLAM-LIMO algorithm. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be described clearly and completely below in conjunction with the drawings of this specification. It should be noted that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] A method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors includes the following steps:

[0039] Step 1: The vehicle recognizes the target object in the road environment. The camera on the vehicle obtains the original image, takes the original perception result as input, and obtains the target recognition result at the current moment through the target detection network. The calculation process is as follows;

[0040] Specific operations such as Figure 2 As shown:

[0041] 1. Since the acquired images and videos are of different sizes, first unify the input size and scale the t-1 frame image and t frame images To a unified size: 512×512;

[0042] 2. Based on the Res-Net50 algorithm, a uniform size 512×512 feature map is extracted from the input data to filter out interference information and avoid redundancy;

[0043] 3. Convolution, pooling, and fully connected layers: reduce the dimension of the feature map, retain important information, classify or regress to identify the target and predict its location, and finally output a feature map with a size of 2048×16×16;

[0044] 4. Generate the motion trajectory of the target by updating the target's position in consecutive frames, and connect the position of the target in each frame to form a smooth curve or path.

[0045] Step 2: The tensor reconstruction process is as follows:

[0046] like Figure 3 and Figure 4 As shown:

[0047] When n1 and h are very large, for a given integer n2, add a minimal zero matrix to X Make And n3=(h+l) / n2 is an integer, then, the matrix Reconstructing into tensors So that: X (k) =X(:(k-1)n2+1:kn2),k∈[n3];

[0048] For moving target detection, there are existing algorithms based on the low-order sparse decomposition LRSD model, which utilizes the existence of temporal continuity and spatial sparsity of targets across frames. The low-order sparse decomposition model is as follows:

[0049] X = B + T;

[0050] in is the original image, B is the background, T is the target, n1, n2 and n3 are the number of bands and target pixels respectively;

[0051] The low-order part is modeled with L2,1 norm, and the spatiotemporal tensor cube is constructed. The target detection problem is equivalent to the low-order sparse decomposition problem, and the target perception result is fused with the spatiotemporal tensor decomposition. The L2,1 norm is usually used in sparse representation and feature selection. It applies different norms in different dimensions. For the three-dimensional video tensor, a reasonable definition of L2,1 norm may be to first apply the L2 norm to the slices in the time dimension, and then apply the L1 norm to these L2 norm values, which can be expressed as:

[0052]

[0053] The detection results of all pixels are expressed as:

[0054]

[0055] Where m and n are the number of rows and columns of the image to be detected, respectively. The topological information of the graph characterizes the spatial structural relationship of the image by calculating the similarity between different pixels in the image. In order to improve the separation between the target and the background, the graphical representation of the pixel to be detected and the pixels in its four neighborhoods is used to adjust the matching results. The local graph matrix is ​​given by the following formula:

[0056]

[0057] Where h and l are from 1 to 5, T h,l represents the similarity calculation between the h-th pixel and the l-th pixel in the local area, which is given by:

[0058] in represents the root mean square of the h-th pixel value. The local structure of the pixel x(i,j) to be measured is represented by the degree of the pixel node, which is calculated as:

[0059] Where h is the pixel to be measured with coordinates (i, j), and the local representation of all pixels is Forming a matrix Combining the matching strategy with the graph structure representation strategy, we get The target perception result based on multiple frames is:

[0060] Μ(:,:,V)=S;

[0061] Where Μ(:,:,V) represents the perceptual tensor at frame ν; “:” means selecting all elements of this dimension;

[0062] Convolutional layer: extracts image features through multiple convolutional layers, and each convolutional layer is followed by a BN layer. By normalizing the input on each small batch of data, the mean of the data is close to 0 and the variance is close to 1, thereby accelerating the convergence speed of the neural network and improving the stability of training. Then, the ReLU activation function is used to increase the nonlinearity of the neural network, so that the network can learn more complex patterns.

[0063] Pooling layer: Use the maximum pooling layer to gradually reduce the size of the feature map. Global average pooling: Compress the spatial dimension of each channel of the feature map into a single value to generate a global feature vector.

[0064] Fully connected layer: maps the global feature vector to the number of categories to perform tasks such as classification or regression. By updating the position of the target in consecutive frames, the motion trajectory of the target is generated, and the position of the target in each frame is connected to form a smooth curve or path.

[0065] Step 3: Obtain laser point cloud data to check the point cloud integrity and make reasonable supplements;

[0066] like Figure 5 As shown:

[0067] Among them, a residual learning iterative refinement method is proposed for the lidar point cloud. This method aims to take imperfect and noisy low-resolution point cloud patches as input and produce high-fidelity high-resolution point cloud patches as output. It is proposed to use deep learning point cloud to filter out noise points to robustly handle noisy and imperfect low-resolution point clouds, and finally use the PRN network to obtain more complete point cloud details of the target object;

[0068] Among them, the incomplete point cloud is used as input, iteratively refined and deeply repaired at the same time, the original c is retained to learn c1 to obtain c2 as the final output, some specific targets are learned according to the PCN neural network, and then the complete point cloud is output to obtain the complete target model, so that the neural network is iteratively trained to achieve the optimal learning weight;

[0069] Among them, PCN mainly consists of two parts: the encoder and the decoder. The encoder is responsible for encoding the geometric information in the input point cloud data into a feature vector. The encoder is designed with permutation invariance, noise robustness, and the ability to process data with different numbers of input points. The decoder is responsible for generating the output point cloud from the feature vector output by the encoder. The decoder uses a coarse-to-fine point set generator to generate point clouds in stages, from coarse point clouds to fine point clouds.

[0070] The present invention proposes to first train some common targets for specific categories for neural network point cloud completion, use the deep learning model PCN to annotate complete point cloud data as a training set, and train the deep learning model so that it can learn the geometric features and structural information of each category of point cloud, including: monitoring the training process, adjusting model parameters, and optimizing model performance; obtaining residual point cloud data from different sources, including point clouds of specific categories such as people, cars, and street lights, removing noise points, smoothing the point cloud surface, and ensuring data quality; standardizing the point cloud data format for subsequent processing; annotating the point cloud data to clarify the category to which each point cloud belongs, and inputting the residual point cloud into the trained model. The model predicts and completes the missing parts based on the learned features and structural information.

[0071] Before executing the intermediate layers, a part of c1 is cut off to get c, in which the entire input c1 is used to execute the intermediate layers we defined. A copy of c is first kept, and then integrated into c2 to the next layer of Following layers. Through this network structure, the size of c can be adjusted at will, and the sparse point cloud is increased to a dense point cloud, while also refining the geometric details of the target object;

[0072] For the PCN point cloud completion network, multiple loss functions are usually combined to comprehensively evaluate the quality of the generated point cloud;

[0073] Among them, the point cloud similarity measurement indicator Chamfer Distance (CD) is used to calculate the sum of the distances from each point in one point set to the nearest point in another point set, which is defined as:

[0074]

[0075] Among them, ||.||2 represents the L2 norm, |P| and |Q| represent the number of points in the point sets P and Q respectively; the first term calculates the average distance from each point in P to the nearest point in Q, and the second term calculates the average distance from each point in Q to the nearest point in P;

[0076] Earth Mover's Distance (EMD) takes into account the overall distribution differences between point clouds. EMD calculates the minimum transmission cost between two point sets by solving an optimal transmission problem. For two point sets P and Q, their EMD can be defined as:

[0077]

[0078] Among them, φ·P→Q is a mapping from P to Q, c(p) is the cost function associated with point P. In order to more comprehensively evaluate the quality of the generated point cloud, multiple loss functions are usually combined to define the total loss function, which can be combined with CD and EMD;

[0079] textLoss = λ1·CD(P gt ,P pred )+λ2·EMD(P gt ,P pred )

[0080] Among them, P gt shows the real point cloud, P pred represents the generated point cloud, λ1·λ2 is the weight coefficient used to balance the importance of the two loss functions;

[0081] The residual depth restoration can be expressed as: c2=c1+c.

[0082] Multi-sensor fusion processing optimization: LiDAR and camera are two complementary sensors. LiDAR provides high-precision depth information, while the camera captures dense but less accurate depth data. By fusing the data of these two sensors, the depth values ​​of pixels in the image are repaired, thereby improving the accuracy and robustness of overall depth perception. This paper adopts an improved SLAM-LIMO algorithm, which integrates the joint optimization of multi-frame image data and the depth information of laser point cloud feature points, solving the problems of temporal discontinuity and spatial fragmentation of post-sensor fusion.

[0083] The entire improved SLAM-LIMO system process includes the following steps: Figure 6 As shown:

[0084] 1) Coordinate transformation: The radar coordinate system is regarded as the world coordinate system, and the transformation matrix (external parameter matrix) E of the radar coordinate system to the world coordinate system camera is obtained;

[0085]

[0086] World Coordinate System Homogeneous coordinates

[0087] Camera coordinate system Homogeneous coordinates

[0088] Simplified, we can get

[0089] You can get: X world =R -1 ×X cam +T, let the transformation matrix K (camera intrinsic parameter matrix) from the camera coordinate system to the image coordinate system and its inverse matrix K -1 ,

[0090] The same as above gives:

[0091] [M] 3×3 It represents taking the first three rows and the first three columns of the matrix M. From the above formula, we can know that: given the world radar coordinates, they can be accurately projected onto the image coordinates;

[0092] You can get: X world =z cam ×R -1 ×K -1 ×X img_h +T.

[0093] 2) The sliding window method limits the amount of BA calculation by maintaining a fixed-size keyframe window. When a new keyframe is added to the window, the oldest keyframe will be removed from the window. However, directly removing the keyframe will lose the information associated with the keyframe. Therefore, marginalization technology is usually used to process old keyframes. The overall cost function is:

[0094]

[0095] Among them, z ij Is in position ξ i The landmark point p is observed at j The data generated, h(ξ i ,p j ) is the observation model. After linearizing the cost function, the incremental equation HΔx=g is solved. In the sliding window BA, the sparsity of the H matrix is ​​crucial to the solution efficiency.

[0096] 3) Graph optimization is a method of expressing optimization problems in the form of graphs. In SLAM, the vehicle's posture is used as a vertex, and the relationship or observation between postures at different times is used as an edge. The entire graph structure is optimized by continuously adjusting the vertex posture to satisfy the edge constraints. The error function is:

[0097]

[0098] Among them, x i and x j are the positions of vertices i and j respectively, and are T ij The transformation relationship from vertex j to vertex i.

[0099] 4) Global closed-loop detection: Introduce dynamic object detection and elimination mechanism to reduce the impact of dynamic objects on the SLAM system. Through loop detection and optimization technology, correct the accumulated errors and improve the map accuracy. Apply more robust optimization algorithms and filtering methods such as Kalman filtering and particle filtering to improve the stability of the system under the influence of noise and error.

[0100] 5) Post-fusion mapping based on SLAM-LIMO takes advantage of the advantages of each sensor to improve mapping accuracy and real-time environmental perception capabilities.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0102] Parts of the present invention not described in detail are prior art.

Claims

1. A method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors, characterized by: When an autonomous vehicle drives within a certain distance of a complex traffic intersection, it detects objects within its range in combination with multi-sensor driving requirements, including the following steps: S1: Obtain the image information of the camera sensor, extract the target objects of each image based on Res-Net50, important targets such as people, cars, indicator lights, etc., and filter interference information to avoid redundancy; S2: To prevent temporal fragmentation during post-fusion, the L2,1 norm is used to model the parts with weak continuity, separate the target object from the background, fuse the feature targets identified by CNN fast pixels, and construct a spatiotemporal tensor cube; output a tensor feature map with temporal continuity; S3: To prevent spatial fragmentation during post-fusion, the acquired LiDAR sensor point cloud information is iteratively refined using residual learning and a symmetric process is performed to retain the credible information in the input partial point cloud and reconstruct the incomplete point cloud to ensure the stability and integrity of the spatial structure during post-fusion. S4: In order to solve the problems of temporal discontinuity and spatial fragmentation in post-sensor fusion, an improved SLAM-LIMO method is proposed. Based on the accurate depth estimation of LiDAR and the feature tracking capability of the camera, the joint optimization of multi-frame image data and the depth of feature points obtained by the laser point cloud are collected, and a method for extracting the depth of image feature point cloud from LiDAR is proposed.

2. The method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors according to claim 1 is characterized by: The step S1 specifically includes the following steps: S11: Zoom t-1 frame image and t frame images To a unified size: 512×512; S12: Extract a uniform size 512×512 feature map based on the input data based on the Res-Net50 algorithm, filter out interference information, and avoid redundancy; S13: Convolution, pooling, and fully connected layers: reduce the dimension of the feature map, retain important information, and perform classification or regression to identify the target object and predict its location. Finally, the feature map size is output to be 2048×16×16. S14: Generate a motion trajectory of the target object by updating the target object position in consecutive frames, and connect the positions of the target object in each frame to form a smooth curve or path.

3. The method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors according to claim 1 is characterized by: The low-order part is modeled using the L2,1 norm to construct a spatiotemporal tensor cube, the target detection problem is equivalent to the low-order sparse decomposition solution problem, an alternating direction multiplier method ADMM is used to solve the model, and the target perception result is fused with the spatiotemporal tensor decomposition. The step S2 specifically includes the following steps: S21: Reconstruct the matrix into a third-order tensor, adding a minimal zero matrix to X for a given integer n2 So that: And n3:=(h+l) / n2 is an integer; S22: The matrix Refactoring to Tensor So that: X (k) =X(:(k-1)n2+1:kn2),k∈[n3]; S23: using the continuity of the tensor and the topological information of the graph to represent the spatial structural relationship and temporal relationship of the image by calculating the similarity between different pixels; S24: Use L2,1 norm to first apply L2 norm to each slice in the time dimension, and then apply L1 norm to these L2 norm values ​​to maintain the balance of features within the sample and promote feature selection or sparsity:

4. The method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors according to claim 1 is characterized by: Taking imperfect and noisy low-resolution point cloud patches as input and generating high-fidelity high-resolution point cloud patches as output, using deep learning point cloud to filter out noise points to robustly process noisy and imperfect low-resolution point clouds, and using the intermediate layer of the PRN network and the optimization of encoding and decoding to obtain more complete point cloud details of the target object, the step S3 specifically includes the following steps: S31: PCN is similar to the nature of encoding and decoding. It takes the incomplete point cloud encoder as input and decodes the decoder as output. It trains specific point clouds and specific categories, and restores the complex structure of the spatial point cloud according to the trained model, including the height, width and length of the object scanned by the lidar. S32: The residual depth PRN repair can be expressed as: c2 = c1 + c; S33: The loss function of PCN+PRN is: textLoss=λ1·CD(P gt ,P pred )+λ2·EMD(P gt ,P pred ); S34: During the uneven repair, in order to ensure the loss of the original point cloud, the point cloud is copied to the next step, and then the point cloud is processed and supplemented by combining the processed point cloud and the original point cloud.

5. The method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors according to claim 1 is characterized by: When the determination results are obtained in S1, S2 and S3, the step S4 specifically includes the following steps: S41: Coordinate transformation X world =z cam ×R -1 ×K -1 ×X img_h +T; S42: The sliding window uses a more efficient backend optimization algorithm to globally optimize the results of S41 to further improve accuracy and consistency; S43: The optimization problem is expressed in the form of a graph. In SLAM, the posture of the vehicle is used as a vertex. The relationship between the postures at different times is optimized by continuously adjusting the postures of the vertices to satisfy the edge constraints. S44: Global closed-loop detection: In order to solve the problems of moving objects and occlusion in dynamic environments, a dynamic object detection and elimination mechanism is introduced to reduce the impact of dynamic objects on the SLAM system. Through loop detection and optimization technology, the accumulated errors are identified and corrected to improve the accuracy. At the same time, more robust optimization algorithms and filtering methods are used to improve the stability of the system under the influence of noise and errors. S45: Post-fusion mapping based on SLAM-LIMO takes advantage of the advantages of each sensor to improve mapping accuracy and real-time environment perception capabilities.

6. The method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors according to claim 2 is characterized by: When the determination results are obtained in S11 and S12, the step S13 specifically includes the following steps: S131: Convolutional layer: extract image features through multiple convolutional layers, each convolutional layer is followed by a BN layer and a ReLU activation function; S132: Pooling layer: Use the maximum pooling layer to gradually reduce the size of the feature map; S133: Global average pooling: compress the spatial dimension of each channel of the feature map into a single value to generate a global feature vector; S134: Fully connected layer: maps the global feature vector to the number of categories to perform tasks such as classification or regression. By updating the position of the target in consecutive frames, the motion trajectory of the target is generated, and the position of the target in each frame is connected to form a smooth curve.

7. The method for solving the spatiotemporal discontinuity of post-fusion of autonomous driving sensors according to claim 4 is characterized by: When the determination results are obtained in S31 and S32, the step S33 specifically includes the following steps: S331: Calculate the sum of the distances from each point in a point set to the nearest point in another point set by using Chamfer Distance; S332: Earth Mover's Distance is used to consider the overall distribution difference between point clouds, and the minimum transmission cost between two point sets is calculated by solving an optimal transmission problem to balance the importance of the two loss functions; S333:textLoss=λ1·CD(P gt ,P pred )+λ2·EMD(P gt ,P pred )。

Images