A monocular SLAM dynamic point filtering method and device based on deep learning and a medium

Abstract

The application relates to a monocular SLAM dynamic point filtering method and device based on deep learning and a medium, wherein the method comprises the following steps: acquiring an image to be processed; performing feature extraction on the image to be processed to obtain a feature map, and performing semantic segmentation prediction to obtain a binary mask image; using a traversal search algorithm to extract an image boundary contour, calculating a corresponding image moment, and determining the centroid of the target contour; using a Hungarian algorithm and a Kalman filter to track the centroid; adopting a sliding window mechanism, determining the significant difference of the motion of a detection target based on the centroid tracking result, and dividing the target into a dynamic target and a static target; optimizing the dynamic target misjudgment situation by using the centroid tracking result of the static target; filtering the dynamic feature points in the SLAM environment based on the optimized dynamic target, and establishing a map and positioning. Compared with the prior art, the application has the advantages of being capable of eliminating dynamic feature points, maximizing the use of static target information, and significantly improving the precision of a SLAM system.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus and medium for dynamic point filtering in monocular SLAM based on deep learning. Background Technology

[0002] Due to continuous advancements in science and technology, many advanced SLAM algorithms have emerged. Among them, Visual SLAM (VSLAM) systems based on low-cost camera sensors have received significant attention in fields such as robotics, autonomous driving, and augmented reality. After years of research, the theoretical framework of VSLAM has become relatively complete, as exemplified by ORB-SLAM2 and DSO. However, most VSLAM algorithms are developed under the assumption that they operate in static environments. Dynamic targets are unavoidable in the real world, and static assumptions can lead to errors in pose estimation and loop closure detection, thereby degrading the performance of SLAM algorithms.

[0003] With the rapid development of artificial intelligence in computer vision, deep learning has greatly promoted the development of VSLAM. DS-SLAM and DynaSLAM are two classic VSLAM algorithms, mainly used to handle moving objects in dynamic environments. Both algorithms use semantic segmentation in their implementation. However, due to their time-consuming nature, more and more researchers are favoring object detection over semantic segmentation when performing SLAM. YOLO-SLAM combines object detection in RGB images with the RANSAC algorithm in depth images to eliminate dynamic feature points. Since this algorithm assumes that people are usually moving in most scenes, it cannot determine the actual motion state of the person, and this assumption is invalid in real-world scenarios. Therefore, combining object detection with SLAM algorithms to determine the motion state of objects in real-time scenes is a challenging research area. Eliminating dynamic feature points and maximizing the use of static object information can significantly improve the accuracy of SLAM systems.

[0004] In summary, semantic segmentation and object detection-assisted SLAM algorithms have been extensively studied in dynamic environments. However, these methods suffer from a major drawback: if the moving object is stationary, they may mistakenly identify these static feature points as dynamic; conversely, if the static object is moving, they may incorrectly identify it as a dynamic feature point. Furthermore, most SLAM algorithms focusing on dynamic scenes primarily aim to eliminate dynamic objects to optimize performance, failing to fully leverage the potential advantages of the semantic information of static objects. Summary of the Invention

[0005] The purpose of this invention is to provide a method, device, and medium for dynamic point filtering in monocular SLAM based on deep learning. By combining object detection and semantic segmentation techniques and utilizing a sliding window method, it can quickly and effectively filter out real dynamic feature points, thereby greatly reducing the impact of dynamic objects on the SLAM algorithm.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A deep learning-based monocular SLAM dynamic point filtering method includes the following steps:

[0008] S1, Obtain the image to be processed;

[0009] S2: Based on the YOLO segment, feature maps are extracted from the image to be processed, and semantic segmentation prediction is performed. Each pixel in the feature map is assigned a category label to obtain a binarized mask image.

[0010] S3, Centroid estimation based on binary mask image: Extract the image boundary contour using a traversal search algorithm, calculate the corresponding image moments, and determine the centroid of the target contour;

[0011] S4 uses the Hungarian algorithm and Kalman filtering to track the centroid;

[0012] S5 uses a sliding window mechanism to determine significant differences in the motion of the detected target based on the centroid tracking results, and determines whether the target is in motion, classifying the target into dynamic targets and static targets;

[0013] S6, optimize the misjudgment of dynamic targets by using the centroid tracking results of static targets;

[0014] S7, based on the optimized dynamic target, performs dynamic feature point filtering in the SLAM environment.

[0015] In step S3, the extraction of image boundary contours using the traversal search algorithm is specifically as follows: a one-dimensional array is used to record the information of the eight neighboring pixels of the processed pixel; if the gray values ​​of the eight neighboring pixels are the same as the gray value of the center point, the point is considered to be inside the object and is deleted; otherwise, the point is considered to be on the edge of the image and is retained; each pixel in the image is processed in turn to obtain the image contour extraction result.

[0016] In step S3, the method for calculating the corresponding image moments and determining the centroid of the target contour is as follows:

[0017] Calculate the geometric moments of the image, including spatial moments, central moments, and center-normalized moments;

[0018] Calculate the centroid of the target contour based on the geometric zeroth and first moments of the image:

[0019]

[0020] M 10 =∑ I ∑ J i*V(i,j),M 01 =∑ I ∑ J j*V(i,j)

[0021]

[0022] Among them, M 00 It is the zeroth moment, M 01 and M 10 It is a first-order moment, i c and j c These are the centroid coordinates of the image. In a two-dimensional image, the centroid and centroid coincide.

[0023] Step S4 includes the following steps:

[0024] S41, let the centroids of the same dynamic target in multiple consecutive frames form the centroid trajectory of a dynamic target, with λ i ={p i ,v i ,start i end i} represents the trajectory of the centroid motion numbered i, where the trajectory number corresponds one-to-one with the number of the dynamic target, p i ,v i The state x that constitutes the trajectory i p i =[x i ,y i ] T This indicates the centroid position corresponding to the dynamic target mask. The start indicates the velocity of the centroid of the dynamic target. i Indicates the index of the starting frame of the dynamic target trajectory, end i This represents the index of the end frame of the dynamic target trajectory, i.e., the frame number of the dynamic target i at the start time. i Frame to end i Frames are continuously tracked; let the current frame be the k-th frame, and the set of dynamic target trajectories be: The dynamic target detection result is

[0025] S42, using Kalman filtering to predict Λ k Each trajectory in Given the state of the current frame, obtain the prior states of all trajectories in the current frame, thus forming...

[0026] S43, using the Hungarian algorithm to match the dynamic target centroid detection results of the current frame. and After the trajectory in the image is matched, The centroids in the target are divided into two categories: dynamic target centroids that successfully match existing trajectories and dynamic target centroids that fail to match. The trajectories are divided into two categories: trajectories that successfully match the centroid of the dynamic target and trajectories that fail to match the centroid of the dynamic target.

[0027] S44, based on the matching results in step S43, process them respectively to complete the classification of centroids and their tracking between consecutive frames, and obtain the trajectory set. set up The system updates and maintains the trajectory status, while using digital codes to distinguish different detection categories and colors to distinguish different dynamic targets.

[0028] Step S44 includes the following steps:

[0029] S441, For successfully matched dynamic targets and trajectories, establish observations of the trajectory using the dynamic targets:

[0030] set up and Match successful, and The centroid is p o =[x o ,y o ] T The observation equation is established as follows:

[0031]

[0032] The matrix is ​​represented as: p o =Hx, where:

[0033]

[0034] Let R be the covariance matrix of the noise during the observation process, and calculate the Kalman gain K:

[0035]

[0036] The state update of the trajectory is accomplished using Kalman filtering, as shown in the following equation:

[0037]

[0038] While completing the trajectory update, Classified as number k i The dynamic target completes continuous trajectory tracking in the current frame;

[0039] S442, for centroids that fail to match, treat them as newly appearing trajectories and add them to the trajectory:

[0040] exist Add a new trajectory and assign a new number to it;

[0041] Point cloud The centroid is used as the centroid of the point cloud in the trajectory state variables, and the velocity in the trajectory state variables is set to [0,0]. T After completing the trajectory state initialization, The number is set to the newly created trajectory number;

[0042] S443, for unsuccessfully matched trajectories, a delayed deletion trajectories management strategy is adopted:

[0043] When a trajectory fails to match, the number of consecutive unmatched trajectories is recorded. If the number of consecutive unmatched trajectories exceeds a preset threshold, the trajectory is removed from the list. Delete it.

[0044] In step S5, the sliding window mechanism is specifically as follows:

[0045] Trajectory set λ i ={p i ,v i ,start i end i The} includes the motion states of different target centroids. and two-dimensional position p i =[x i ,y i ] T The smoothing process employs a Gaussian weighted moving average filter and a Gaussian distribution function: a filtering window is defined, including the current sample point and several surrounding points; the weight of each point within the window is calculated according to the Gaussian distribution, where the weight of points farther away from the current sample point is smaller; the weights of each point within the window are averaged to obtain the filtering result for the current sample point; the filtering window is moved forward one position, and the above smoothing steps are repeated until all sample points have been processed.

[0046] Step S5 specifically involves:

[0047] Based on the processing results of the sliding window mechanism, the trajectory λ i The corresponding time is start i end iTarget detection bounding box width and height Calculate the average to get w mean h mean Divide the increments dx and dy of the centroid on the x and y axes by w respectively. mean h mean Normalization is performed to obtain the normalized coordinate increments; the corresponding centroid velocity and acceleration are calculated based on the normalized coordinate increments, and targets to which centroids belong with velocity and acceleration greater than preset values ​​are marked as dynamic targets, while targets to which centroids belong with velocity and acceleration less than preset values ​​are marked as static targets.

[0048] Step S6 specifically involves: dividing the element centroid x in the static target pool partitioned in step S5. i y i Normalize the increments dx and dy on the axis, respectively, by dividing The normalized coordinate increment sequence is obtained; the motion component velocity v of the stationary target is calculated based on the normalized coordinate increment. x|i v y|i The average stationary target's motion component velocity is calculated; the motion component velocity of the dynamic target is subtracted from the corresponding velocity of the stationary target to obtain the true two-dimensional moving velocity of the dynamic target, thus filtering out falsely identified dynamic targets.

[0049] A deep learning-based monocular SLAM dynamic point filtering device includes a memory, a processor, and a program stored in the memory. When the processor executes the program, it implements the method described above.

[0050] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] (1) The present invention uses a sliding window and Gaussian weighted moving average filtering method, which can not only effectively eliminate the oscillation of the centroid trajectory and reduce the centroid motion error of static targets, but also accurately capture the motion characteristics of dynamic targets.

[0053] (2) Based on factors such as displacement, velocity, direction and acceleration of the moving target, this invention provides a method for detecting dynamic targets, which can effectively realize the detection of dynamic targets in a static camera environment and filter out some dynamic points;

[0054] (3) The present invention uses static targets to constrain dynamic objects, which effectively reduces the probability of misjudging dynamic objects due to camera movement and improves the accuracy of dynamic point filtering.

[0055] (4) This invention integrates dynamic point detection and filtering schemes into the ORB-SLAM2 system to improve the accuracy of mapping and positioning. Attached Figure Description

[0056] Figure 1 This is a flowchart of the method of the present invention;

[0057] Figure 2 This is a schematic diagram of a high-dynamic target mask binarized image.

[0058] Figure 3 A schematic diagram for visualizing and tracking the centroid of the target;

[0059] Figure 4 A schematic diagram of prior classification for target detection;

[0060] Figure 5 This is a flowchart of the SLAM algorithm for dynamic point filtering based on the present invention. Detailed Implementation

[0061] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0062] This embodiment proposes a deep learning-based monocular SLAM dynamic point filtering method. This method filters dynamic targets using the ORB-SLAM2 framework while utilizing more information from static targets to optimize camera pose. After object detection in the RGB image, a sliding window constraint is applied to detect dynamic objects instead of predefined labels. Furthermore, data associations are established for static objects to add object-level semantic constraints to the system. These object-level semantic constraints are implemented using the centroid distance of objects in the visual sequence.

[0063] Specifically, this embodiment provides a method for dynamic point filtering in monocular SLAM based on deep learning, such as... Figure 1 As shown, it includes the following steps:

[0064] S1, Obtain the image to be processed.

[0065] S2 performs feature extraction and semantic segmentation prediction on the image to be processed based on YOLO segments.

[0066] First, feature extraction is performed using YOLO segments to obtain feature maps. These feature maps contain various information about the image, such as edges, texture, and color. Then, semantic segmentation prediction is performed, effectively converting the feature maps into segmentation results of the same size as the input image. Each pixel in the feature map is assigned a class label, resulting in a binarized mask image, such as... Figure 2 As shown.

[0067] S3, Centroid estimation based on binary mask image: Extract the image boundary contour using a traversal search algorithm, calculate the corresponding image moments, and determine the centroid of the target contour.

[0068] S31, Contour Extraction:

[0069] Binary images alone cannot be used to calculate the centroid of the target object, so boundary contour extraction is required. The first step is contour finding. This patent uses a traversal search algorithm, specifically: a one-dimensional array is used to record the information of the eight neighboring pixels of the processed pixel; if the gray values ​​of the eight neighboring pixels are the same as the gray value of the center point, the point is considered to be inside the object and is deleted; otherwise, the point is considered to be on the edge of the image and is retained; each pixel in the image is processed sequentially to obtain the contour extraction result.

[0070] S32, calculate the corresponding image moments and determine the centroid of the target contour.

[0071] After obtaining the target's contour, the corresponding image moments are calculated. Moments are operators that describe image features. Common moment descriptors can be categorized as follows: geometric moments, orthogonal moments, complex moments, and rotational moments. The set of moments calculated from an image can not only describe the global features of the image shape but also provide a wealth of information about different geometric features of the image, such as size, position, orientation, and shape. The geometric moments of an image include spatial moments, central moments, and center-normalized moments. This embodiment calculates the centroid corresponding to the target contour using the image's zeroth-order and first-order geometric moments:

[0072]

[0073] M 10 =∑ I ∑ J i*V(i,j),M 01 =∑ I ∑ I j*V(i,j)

[0074]

[0075] Among them, M 00 It is the zeroth moment, M 01 and M 10 It is a first-order moment, i c and j c These are the centroid coordinates of the image. In a two-dimensional image, the centroid coincides with the centroid. The centroid estimation result is as follows: Figure 3 As shown.

[0076] Steps S1-S3 above describe the centroid estimation process based on deep learning, while steps S4-S7 describe the dynamic point detection and filtering process based on a sliding window. For dynamic point filtering, development is based on the ORB-SLAM2 framework and YOLOv8, integrating a dynamic object elimination module and semantic constraints for static objects to adapt to dynamic scenes. After calculating and detecting the centroid of instance objects, the Hungarian algorithm and Kalman filtering are used to track the centroid. After tracking the target centroid, a sliding window mechanism is used to determine whether the target is in motion based on significant differences in the detected target's motion. Based on these methods, the dynamic point detection problem under static and shaky camera conditions is addressed.

[0077] S4 uses the Hungarian algorithm and Kalman filtering to track the centroid.

[0078] To perform centroid tracking, it's necessary to combine the target centroid detection results from adjacent frames. Specifically, this includes the following steps:

[0079] S41, let the centroids of the same dynamic target in multiple consecutive frames form the centroid trajectory of a dynamic target, with λ i ={p i ,v u ,start i end i} represents the trajectory of the centroid motion numbered i, where the trajectory number corresponds one-to-one with the number of the dynamic target, p i ,v i The state x that constitutes the trajectory i p i =[x i ,y i ] T This indicates the centroid position corresponding to the dynamic target mask. The start indicates the velocity of the centroid of the dynamic target. i Indicates the index of the starting frame of the dynamic target trajectory, end i This represents the index of the end frame of the dynamic target trajectory, i.e., the frame number of the dynamic target i at the start time. i Frame to end i Frames are continuously tracked; let the current frame be the k-th frame, and the set of dynamic target trajectories be: The dynamic target detection result is

[0080] S42, To combine the trajectory and dynamic target detection results, Kalman filtering is used to predict Λ. k Each trajectory in Given the state of the current frame, obtain the prior states of all trajectories in the current frame, thus forming...

[0081] Let the saved trajectory λ = {p, v, start, end} have a state x = [p, v] T A uniform motion model can be used as the prediction model for Kalman filtering. Let the time interval between two adjacent video frames be Δt, and the state transition process is shown in the equation:

[0082]

[0083] Expressed in matrix form, i.e. in:

[0084]

[0085] Let P be the covariance matrix of state x and Q be the noise covariance matrix of the transition process. The update of the covariance matrix during the state transition process can be obtained as shown in the following equation:

[0086]

[0087] For Λ k After all the trajectories stored in the database have completed state prediction, the prior states of all trajectories in the current frame are obtained, constituting...

[0088] S43, In order to complete the Kalman filter update, the Hungarian algorithm is used to match the dynamic target centroid detection results of the current frame. and After the trajectory in the image is matched, The centroids in the target are divided into two categories: dynamic target centroids that successfully match existing trajectories and dynamic target centroids that fail to match. The trajectories are divided into two categories: trajectories that successfully match the centroid of the dynamic target and trajectories that fail to match the centroid of the dynamic target.

[0089] S44, based on the matching results in step S43, process them respectively to complete the classification of centroids and their tracking between consecutive frames, and obtain the trajectory set. set up The system completes trajectory status updates and maintenance, while using digital codes to distinguish different detection categories and colors to differentiate different dynamic targets. The centroid tracking results are visualized as follows: Figure 4 As shown.

[0090] Specifically, the following further processing is performed on different matching results:

[0091] S441, For a successfully matched dynamic target and trajectory, use the dynamic target to establish observations of the trajectory.

[0092] set up and Match successful, and The centroid is p o =[x o ,y o ] T The observation equation is established as follows:

[0093]

[0094] The matrix is ​​represented as: p o =Hx, where:

[0095]

[0096] Let R be the covariance matrix of the noise during the observation process, and calculate the Kalman gain K:

[0097]

[0098] The state update of the trajectory is accomplished using Kalman filtering, as shown in the following equation:

[0099]

[0100] While completing the trajectory update, Classified as number k i The dynamic target completes continuous tracking of its trajectory in the current frame.

[0101] S442, for centroids that fail to match, treat them as newly emerging trajectories and add trajectories accordingly.

[0102] First of all, Add a new trajectory and assign a new number to it;

[0103] Secondly, point clouds The centroid is used as the centroid of the point cloud in the trajectory state variables, and the velocity in the trajectory state variables is set to [0,0]. T After completing the trajectory state initialization, The number is set to the newly created trajectory number.

[0104] S443, for a trajectory that is not successfully matched, there may be two situations: the first situation is that occlusion occurs during target detection, and thus the image information of the dynamic target represented by the trajectory is not detected in the current frame; the second situation is that the dynamic object represented by the trajectory disappears from the field of view.

[0105] To address these two scenarios, this embodiment employs a delayed deletion trajectory management strategy.

[0106] When a trajectory fails to match, the number of consecutive unmatched trajectories is recorded. If the number of consecutive unmatched trajectories exceeds a preset threshold, the trajectory is removed from the list. Delete it.

[0107] After processing the above three types of objects, for the centroid, the centroid classification was completed, and centroid tracking between consecutive frames was achieved. For the trajectory, the following was obtained: set up The completion of trajectory status updates and maintenance ensures the smooth progress of the next dynamic tracking.

[0108] S5. After completing the tracking of the target centroid, a sliding window mechanism is used to determine the significant differences in the target motion based on the centroid tracking results, determine whether the target is in motion, and classify the target into dynamic targets and static targets.

[0109] A window stores consecutive frames of a fixed length. As frames progress, new frames are stored in the window, while older frames are gradually deleted.

[0110] Trajectory set λ i ={p i ,v i ,start i end i The} includes the motion states of different target centroids. and two-dimensional position p i =[x i ,y i ] T Meanwhile, the centroid trajectory will exhibit a certain degree of oscillation. Therefore, this embodiment employs a Gaussian weighted moving average filter and a Gaussian distribution function for smoothing: First, a filtering window is defined, including the current sample point and several surrounding points; second, the weight of each point within the window is calculated according to the Gaussian distribution, where the weight of points farther away from the current sample point is smaller; then, the weights of each point within the window are weighted and averaged to obtain the filtering result for the current sample point; the filtering window is moved forward one position, and the above smoothing steps are repeated until all sample points have been processed.

[0111] After filtering, dynamic target detection is performed. Considering the lack of scale information in a monocular camera, the motion state of the target cannot be judged solely by the magnitude of displacement or velocity when determining the motion of the centroid. Based on the inherent relationship between the direction of target motion and acceleration, this embodiment presents an algorithm for detecting dynamic targets, specifically including: processing the trajectory λ based on the sliding window mechanism. i The corresponding time is start i end i Target detection bounding box width and height We get the average w mean h mean Divide the increments dx and dy of the centroid on the x and y axes by w respectively. mean h mean Normalization is performed to obtain the normalized coordinate increments; the corresponding centroid velocity and acceleration are calculated based on the normalized coordinate increments, and targets to which centroids belong with velocity and acceleration greater than preset values ​​are marked as dynamic targets, while targets to which centroids belong with velocity and acceleration less than preset values ​​are marked as static targets.

[0112] S6 optimizes the handling of dynamic target misjudgment by utilizing the centroid tracking results of static targets.

[0113] Considering the camera shake in actual sequence data, the following optimization addresses the misjudgment of dynamic targets caused by camera shake. This embodiment employs a constraint method that uses static targets to determine dynamic targets. Building upon step S5, it adds tracking of the centroid of static targets and establishes and maintains a sequence pool containing both static and dynamic targets, such as... Figure 5 As shown. This method enables more effective observation and detection of different types of targets. Specifically, the centroid x of the elements in the static target pool divided in step S5 is... i y i Normalize the increments dx and dy on the axis, respectively, by dividing The normalized coordinate increment sequence is obtained; the motion component velocity v of the stationary target is calculated based on the normalized coordinate increment. x|i v y|i The average velocity component of the stationary target is calculated; the velocity component of the dynamic target is subtracted from the velocity component of the stationary target to obtain the true two-dimensional velocity of the dynamic target, so as to filter out the falsely judged dynamic targets, thereby achieving the purpose of dynamic point detection and filtering.

[0114] S7, based on the optimized dynamic target, performs dynamic feature point filtering in the SLAM environment.

[0115] like Figure 5 As shown, based on the above steps, information from the motion estimation of static objects and the camera itself is fused, and dynamic point detection is applied to the SLAM system to determine dynamic points in the SLAM environment. Then, when a predefined movable object remains static, its static feature points are used, and when a predefined static object moves, its feature points are excluded, thereby achieving pose estimation and loop closure detection. Based on this, map building and localization are performed to improve SLAM accuracy.

[0116] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for dynamic point filtering in monocular SLAM based on deep learning, characterized in that, Includes the following steps: S1, Obtain the image to be processed; S2: Based on the YOLO segment, feature maps are extracted from the image to be processed, and semantic segmentation prediction is performed. Each pixel in the feature map is assigned a category label to obtain a binarized mask image. S3, Centroid estimation based on binary mask image: Extract the image boundary contour using a traversal search algorithm, calculate the corresponding image moments, and determine the centroid of the target contour; S4 uses the Hungarian algorithm and Kalman filtering to track the centroid; S5 uses a sliding window mechanism to determine significant differences in the motion of the detected target based on the centroid tracking results, and determines whether the target is in motion, classifying the target into dynamic targets and static targets; Step S5 specifically involves: Based on the processing results of the sliding window mechanism, the trajectory The corresponding time in China is Target detection bounding box width and height , Calculate the average to get , ; Centroid x , y Increment on axis Excluded , Normalize the coordinates to obtain the normalized coordinate increments; Calculate the corresponding centroid velocity and acceleration based on the normalized coordinate increments. Mark the target whose centroid has a velocity and acceleration greater than the preset value as a dynamic target, and mark the target whose centroid has a velocity and acceleration less than the preset value as a static target. S6, optimize the misjudgment of dynamic targets by using the centroid tracking results of static targets; Step S6 specifically involves: determining the centroids of the elements in the static target pool divided in step S5. , Increment on axis , Normalization, divide separately , The normalized coordinate increment sequence is obtained; the motion component velocity of the stationary target is calculated based on the normalized coordinate increment. , The average stationary target's motion component velocity is calculated; the motion component velocity of the dynamic target is subtracted from the corresponding stationary target's velocity component velocity to obtain the true two-dimensional moving velocity of the dynamic target, thus filtering out falsely identified dynamic targets. S7, based on the optimized dynamic target, performs dynamic feature point filtering in the SLAM environment.

2. The method for dynamic point filtering in monocular SLAM based on deep learning according to claim 1, characterized in that, In step S3, the extraction of image boundary contours using the traversal search algorithm is specifically as follows: a one-dimensional array is used to record the information of the eight neighboring pixels of the processed pixel; if the gray values ​​of the eight neighboring pixels are the same as the gray value of the center point, the point is considered to be inside the object and is deleted; otherwise, the point is considered to be on the edge of the image and is retained; each pixel in the image is processed in turn to obtain the image contour extraction result.

3. The method for dynamic point filtering in monocular SLAM based on deep learning according to claim 1, characterized in that, In step S3, the method for calculating the corresponding image moments and determining the centroid of the target contour is as follows: Calculate the geometric moments of the image, including spatial moments, central moments, and center-normalized moments; Calculate the centroid of the target contour based on the geometric zeroth and first moments of the image: , , in, It is the zeroth moment. and It is a first-order moment. and These are the centroid coordinates of the image. In a two-dimensional image, the centroid and centroid coincide.

4. The method for dynamic point filtering in monocular SLAM based on deep learning according to claim 1, characterized in that, Step S4 includes the following steps: S41, suppose the centroids of the same dynamic target in multiple consecutive frames form the centroid motion trajectory of a dynamic target, and... Indicates the number is The trajectory of the centroid's motion, wherein the trajectory number corresponds one-to-one with the number of the dynamic target. The state that constitutes the trajectory , This indicates the centroid position corresponding to the dynamic target mask. Indicates the velocity of the centroid of a dynamic target. Indicates the index of the starting frame of the dynamic target trajectory. Indicates the index of the end frame of the dynamic target trajectory, i.e., the dynamic target In the Frame to the Frames are tracked continuously; let the current frame be the [number]th frame. The set of frames and dynamic target trajectories is as follows: The dynamic target detection result is ; S42, using Kalman filter prediction Each trajectory in Given the state of the current frame, obtain the prior states of all trajectories in the current frame, thus forming... ; S43, using the Hungarian algorithm to match the dynamic target centroid detection results of the current frame. and After the trajectory in the image is matched, The centroids in the target are divided into two categories: dynamic target centroids that successfully match existing trajectories and dynamic target centroids that fail to match. The trajectories are divided into two categories: trajectories that successfully match the centroid of the dynamic target and trajectories that fail to match the centroid of the dynamic target. S44, based on the matching results in step S43, process them respectively to complete the classification of centroids and their tracking between consecutive frames, and obtain the trajectory set. ,set up It completes the updating and maintenance of trajectory status, and distinguishes different detection categories by digital encoding and different dynamic targets by color.

5. The method for dynamic point filtering in monocular SLAM based on deep learning according to claim 4, characterized in that, Step S44 includes the following steps: S441, For successfully matched dynamic targets and trajectories, establish observations of the trajectory using the dynamic targets: set up and Match successful, and The centroid is The observation equation is established as follows: The matrix is ​​represented as: ,in: Let the covariance matrix of the noise during the observation process be... The Kalman gain was calculated. : The state update of the trajectory is accomplished using Kalman filtering, as shown in the following equation: While completing the trajectory update, Classified as number The dynamic target completes continuous trajectory tracking in the current frame; S442, for centroids that fail to match, treat them as newly appearing trajectories and add them to the trajectory: exist Add a new trajectory and assign a new number to it; Point cloud The centroid is used as the centroid of the point cloud in the trajectory state variables, and the velocity in the trajectory state variables is set to... After completing the trajectory state initialization, The number is set to the newly created trajectory number; S443, for unsuccessfully matched trajectories, a delayed deletion trajectories management strategy is adopted: When a trajectory fails to match, the number of consecutive unmatched trajectories is recorded. When the number of consecutive unmatched trajectories exceeds a preset threshold, the trajectory is removed from the list. Delete it.

6. The method for dynamic point filtering in monocular SLAM based on deep learning according to claim 1, characterized in that, In step S5, the sliding window mechanism is specifically as follows: Trajectory Set This includes the motion states of different target centroids. and two-dimensional position The smoothing process employs a Gaussian weighted moving average filter and a Gaussian distribution function: a filtering window is defined, including the current sample point and several surrounding points; the weight of each point within the window is calculated according to the Gaussian distribution, where the weight of points farther away from the current sample point is smaller; the weights of each point within the window are averaged to obtain the filtering result for the current sample point; the filtering window is moved forward one position, and the above smoothing steps are repeated until all sample points have been processed.

7. A deep learning-based monocular SLAM dynamic point filtering device, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-6.

8. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-6.

Images