Dynamic scene vision SLAM method based on deep learning

Abstract

The invention discloses a dynamic scene visual SLAM method based on deep learning, and the method comprises the following steps: obtaining an original scene image, and extracting ORB feature points in an image scene; based on the improved target detection network, performing dynamic target detection on the original scene image to obtain a dynamic target frame in the image scene; associating the dynamic target frame and the ORB feature points by adopting a key frame synchronization mechanism, and removing the ORB feature points corresponding to the dynamic target frame; obtaining a feature scene image after primary elimination; performing secondary elimination on unmatched ORB feature points in adjacent feature scene images by adopting epipolar geometric constraint to obtain a static feature scene image; and performing pose estimation and / or map construction based on the ORB feature points corresponding to the static feature scene image. According to the method, the dynamic feature points of the dynamic target are extracted by improving the target detection network, and the dynamic feature points are eliminated twice by adopting the epipolar geometric constraint, so that the positioning precision is improved, and accurate map construction is facilitated.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a dynamic scene visual SLAM method based on deep learning. Background Technology

[0002] Visual SLAM is a research area that has attracted much attention in the fields of computer vision and unmanned intelligent systems. It involves using cameras or fusing other sensors to simultaneously determine the location and environmental map of unmanned intelligent systems or mobile devices.

[0003] Currently, traditional visual SLAM methods often fail to meet the localization requirements of unmanned intelligent systems in environments with dynamic interference. When the robot operates in a scene with a large number of dynamic objects, the feature point pairs collected on these objects can severely affect the accuracy of camera pose estimation, and may even lead to system localization failure.

[0004] Methods that effectively address the problem of detecting and filtering dynamic feature points in visual SLAM systems can be categorized into three types: geometry-based methods, deep learning-based methods, and methods combining geometry and deep learning. Geometry-based methods typically require predefined thresholds to identify the state of feature points, easily leading to over- or under-identification. Deep learning-based methods usually use deep learning to generate semantic information for classifying detected feature points, but they heavily rely on network quality, which can affect the detection results. Therefore, methods combining geometry and deep learning leverage the advantages of both approaches.

[0005] However, this dynamic visual SLAM system that combines deep learning still faces two key challenges: first, the balance between dynamic target detection accuracy and dynamic environment adaptability; and second, the impact of increased system computation on network real-time performance.

[0006] Therefore, solving at least one of the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a dynamic scene visual SLAM method based on deep learning, which utilizes the information of dynamic objects extracted by deep learning to further refine geometric constraints and filter feature points of dynamic objects.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A deep learning-based dynamic scene visual SLAM method includes the following steps:

[0010] S1. Obtain the original scene image and extract ORB feature points from the scene image;

[0011] S2. Based on the improved target detection network, dynamic target detection is performed on the original scene image to obtain dynamic target boxes in the image scene; wherein, the improved target detection network adopts a lightweight MobileNetv3 backbone network for deep learning;

[0012] S3. Use a keyframe synchronization mechanism to associate the dynamic target box and ORB feature points, and remove the ORB feature points corresponding to the dynamic target box; to obtain the feature scene image after one removal.

[0013] S4. Using epipolar geometry constraints, mismatched ORB feature points in adjacent feature scene images are removed twice to obtain static feature scene images; and pose estimation and / or map construction are performed based on the ORB feature points corresponding to the static feature scene images.

[0014] Further, step S1 specifically includes:

[0015] The original scene image is acquired, corner points in the image scene are detected by the FAST algorithm, and the corner points are filtered by the Harris corner response value.

[0016] The selected corner points are described in binary form using the BRIEF descriptor; the described corner points are then used as ORB feature points in the image scene.

[0017] Furthermore, the improved target detection network includes: an input end, a backbone network, a neck network, and a head network;

[0018] The input terminal is used for image preprocessing;

[0019] The backbone network adopts the lightweight MobileNetv3 and introduces the deformable convolutional network DCNv2 for multi-scale feature extraction.

[0020] An attention mechanism is introduced into the neck network to integrate attention data from both channel and spatial dimensions for feature optimization.

[0021] The head network is used to predict targets at different scales through the detection head.

[0022] Furthermore, step S2 specifically includes:

[0023] The original scene image is preprocessed through the input terminal;

[0024] The preprocessed image is input into the backbone network, basic features are extracted through convolution operations, and the receptive field is dynamically adjusted through the deformable convolutional network DCNv2. Features of irregularly shaped targets are extracted through learnable sampling offsets and weight coefficients, thus obtaining the corresponding feature representation.

[0025] The corresponding feature representations are input into the neck network, and feature enhancement is performed under the bidirectional feature fusion architecture. The weight distribution of each channel and spatial position in the feature map is adjusted through the attention mechanism of channel and spatial dimensions; further enhancing and capturing multi-target features;

[0026] The head network detects the multi-target features and performs target prediction at different scales to obtain dynamic target bounding boxes in the image scene.

[0027] Furthermore, the deformable convolutional network DCNv2 includes an offset prediction module;

[0028] The offset prediction module uses lightweight convolutional layers to predict the offset on the input feature map corresponding to each output position, and then uses the predicted offset to perform adaptive sampling on the input feature map. Based on the offset, it jumps to a new sampling point and performs weighted summation to generate a feature map for extracting irregularly shaped targets.

[0029] Furthermore, the deformable convolutional network DCNv2 dynamically adjusts its receptive field and extracts features from irregularly shaped targets through learnable sampling offsets and weight coefficients; specifically, it includes:

[0030] The deformable convolutional network DCNv2 enables the feature sampling grid to be dynamically adjusted according to the input content by adaptively learning the spatial offset.

[0031] The calculation process of the deformable convolutional network DCNv2 is expressed by the following formula:

[0032]

[0033] Where y(p0) represents the output feature value of sampling point p0; R represents the convolution kernel element; p n w(p) represents the position of the nth element in R. n ) indicates that in p n The convolution kernel weights at the location, where x is the input feature map to be detected, x(p0+p n ) represents the sampling position of the feature map to be detected; Δp n This indicates that the sampling positions of the convolution kernel are expanded.

[0034] Furthermore, step S3 specifically includes:

[0035] S31. Associate the dynamic target box and ORB feature points by synchronizing keyframe timestamps;

[0036] The formula is expressed as follows:

[0037] t k =min{t y , t o}

[0038] Among them, t k t is the timestamp of the keyframe. y and t o These are the timestamps for the object detection network and ORB feature point extraction, respectively.

[0039] S32. The ORB feature points located within the dynamic target box are taken as dynamic feature points and removed; the feature scene image after one removal is obtained.

[0040] Furthermore, in step S4, the second removal of mismatched ORB feature points in adjacent feature scene images using epipolar geometry constraints specifically includes:

[0041] By calculating the distance d from the point A2 to be matched to the corresponding epipolar line h2 of ORB feature point A1 in adjacent feature scene images, it is determined whether A2 matches A1.

[0042] The formula is expressed as follows:

[0043] a1 = [x1, y1]

[0044] a2 = [x2, y2]

[0045]

[0046] Where A1 and A2 represent feature points in adjacent feature scene images, a1 and a2 represent the homogeneous coordinate forms of A1 and A2 respectively, h2 represents the epipolar vector, and X, Y, and Z represent line vectors;

[0047] If the calculated distance d is less than or equal to the preset threshold, then A2 is retained as the correct matching point;

[0048] If the calculated distance d is greater than the preset threshold, then A2, as a mismatch point, needs to be removed.

[0049] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a dynamic scene visual SLAM method based on deep learning, which has the following beneficial effects:

[0050] This invention improves the target detection network by adopting a lightweight MobileNetv3 backbone network and introducing a deformable convolutional network DCNv2 to detect targets in the environment in real time. It also outputs high-precision detection boxes and semantic information through a neck network enhanced by an attention mechanism. Finally, it outputs accurate dynamic target feature points.

[0051] This invention also detects potential dynamic targets in the environment by constructing thread synchronization and combining feature point extraction from the improved target detection network YOLOv5 and ORB-SLAM3 system, thereby reducing the dynamic object mismatch problem.

[0052] The system uses the target detection results to identify and exclude feature points within the dynamic bounding box; then, based on the epipolar geometric constraint method, it uses the remaining static feature points to detect potential dynamic target feature points; this further improves the positioning accuracy and effectively enhances the system's environmental adaptability and robustness. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0054] Figure 1 A flowchart of a dynamic scene visual SLAM method based on deep learning is provided for an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of a dynamic scene visual SLAM method based on deep learning provided in an embodiment of the present invention.

[0056] Figure 3 This is a schematic diagram of ORB feature points extracted from the original image by ORB-SLAM3, as provided in an embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram of the dynamic target bounding box extracted by the improved target detection network YOLOv5s provided in an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] This invention discloses a dynamic scene visual SLAM method based on deep learning, referring to... Figure 1 The above includes the following steps:

[0060] S1. Obtain the original scene image and extract ORB feature points from the scene image;

[0061] S2. Based on the improved target detection network, dynamic target detection is performed on the original scene image to obtain dynamic target boxes in the image scene; wherein, the improved target detection network adopts a lightweight MobileNetv3 backbone network for deep learning;

[0062] S3. Use a keyframe synchronization mechanism to associate dynamic target boxes and ORB feature points, and remove the ORB feature points corresponding to the dynamic target boxes; obtain the feature scene image after one removal.

[0063] S4. Using epipolar geometry constraints, mismatched ORB feature points in adjacent feature scene images are removed twice to obtain static feature scene images; and pose estimation and / or map construction are performed based on the ORB feature points corresponding to the static feature scene images.

[0064] In this embodiment, when the robot moves indoors, the dynamic SLAM navigation uses ORB-SLAM3 as the basic framework for SLAM, referring to... Figure 2 As shown, the improved object detection network YOLOv5s based on deep learning is combined with ORB-SLAM3 to perform object detection on the read image data. The object detection results are then classified, and the obtained object categories and detection box coordinates are passed to the tracking module of the main framework ORB-SLAM3. In the tracking module, after extracting ORB feature points for the current frame, feature points within the dynamic bounding boxes are judged and excluded based on the object detection results. Then, the purely static feature points left by the epipolar geometric constraint method are used to detect semi-static feature points. In subsequent operations, only stable static feature points are used for tracking and map construction.

[0065] The specific steps of this embodiment are described in detail below:

[0066] The first step is to acquire the original scene image and extract the ORB feature points from the scene.

[0067] In this embodiment, when the visual SLAM system starts running, the system acquires raw data through an RGB-D camera, including RGB images and corresponding depth map information, as the raw scene image; ORB-SLAM3 extracts ORB feature points in the scene, referring to... Figure 3 As shown, Figure 3 In the middle (a), (b), and (c), there is a continuous scene image, and the green annotation shows the feature points extracted by ORB-SLAM3.

[0068] The ORB feature point extraction algorithm (Oriented FAST and Rotated BRIEF) used in this embodiment specifically includes: FAST corner detection and BRIEF descriptor.

[0069] Corner points are points in an image where grayscale changes drastically in both directions. Unlike edges (where grayscale changes drastically in only one direction) or flat areas (where grayscale changes are minimal), corner points possess directionality and stability. The FSAT corner detection algorithm determines whether a pixel is a corner by examining a circular region (typically 16 pixels) surrounding it. If n consecutive pixels (typically n=12) within this region have pixel values ​​that are either higher or lower than the value of the current pixel, then the pixel is considered a corner. This embodiment improves upon this by using Harris response values ​​to filter a fixed number of corner points, thereby enhancing its stability and practicality.

[0070] The original scene image is acquired, corner points in the image scene are detected by the FAST algorithm, and the corner points are filtered by the Harris corner response value.

[0071] The Harris response calculation formula is as follows:

[0072] R = det(M) - k·tr(M) 2

[0073]

[0074] Where M represents the gradient covariance matrix at the corner points, k is an empirical constant, and I x and I y These represent the gradients of the image in the x and y directions, respectively. The larger the Harris response value R, the higher the probability that the point is a corner.

[0075] The BRIEF descriptor is a feature description method based on binary descriptors. It generates descriptors by comparing the grayscale values ​​of pixel pairs.

[0076] In this embodiment, the selected corner points are described in binary form using the BRIEF descriptor; the described corner points are then used as ORB feature points in the image scene.

[0077] For each corner point, select a set of pixel pairs (p i p j ), and compare their grayscale values.

[0078] If p i >p j If the value is 1, the corresponding bit of the descriptor is 1; otherwise, it is 0.

[0079]

[0080] Where f(x) is the output value, I(p) i () represents the grayscale value.

[0081] The second step involves performing dynamic object detection on the original scene image based on the improved object detection network YOLOv5s, thereby obtaining dynamic object bounding boxes in the image scene.

[0082] First, the improved target detection network YOLOv5s performs preprocessing of image data at the input end; the backbone network performs multi-scale feature extraction; the neck network achieves feature fusion to enhance semantic information; and the head network performs target prediction at different scales through three detection heads.

[0083] The input end performs image stitching through operations such as random cropping and scaling transformation, and then inputs the new image into the network for detection. This process effectively improves the diversity of image data.

[0084] The backbone network adopts the lightweight MobileNetv3; a novel lightweight object detection architecture, MobileNetv3-YOLOv5s, is constructed. This improvement significantly reduces model complexity.

[0085] In terms of specific operations, this embodiment integrates key components of MobileNetv3 in the common.py file, including the h_sigmoid activation function, h_swish nonlinear transformation layer, SELayer attention module, conv_bn_hswish convolutional unit, and MobileNet_Block basic structure; modifies the parse_model function in yolo.py to register the above five newly added modules; and creates the yolov5s-mobilenetv3.yaml configuration file, replacing the original YOLOv5s backbone network and feature extraction layer with the corresponding structure of MobileNetv3.

[0086] To address the potential reduction in accuracy after network lightweighting, this embodiment integrates the variable convolution module DCNv2 into the YOLOv5s backbone network, significantly reducing the number of model parameters while maintaining detection accuracy.

[0087] This embodiment introduces the DCNv2 module into the YOLOv5s backbone network and replaces the standard convolution in the C3 module with it, forming a C3-DCNv2 module with deformation adaptability. This improvement enables the network to dynamically adjust its receptive field and significantly enhances its feature extraction capability for irregularly shaped targets through learnable sampling offsets and weight coefficients, making it particularly suitable for detecting targets with variable shapes.

[0088] Deformable Convolutional Network (DCNv2) comprises an offset prediction module and offset-based convolutions. The offset prediction module utilizes a lightweight convolutional layer to predict the offset on the input feature map corresponding to each output location. These offsets reflect the locations of the most relevant feature regions in the current context. Adaptive sampling of the input feature map is performed using the predicted offsets, overcoming the limitations of traditional regular grid sampling. That is, the filter no longer translates with a fixed stride, but jumps to new sampling points based on the offsets and performs weighted summation to generate the final feature map. This improvement allows the network to better adapt to the geometric deformation features of the target, thereby significantly improving the overall performance of the detection model.

[0089] The deformable convolution in this embodiment uses adaptive learning spatial offset to enable the feature sampling grid to dynamically adjust according to the input content. The calculation process of deformable convolution is as follows:

[0090]

[0091] Where y(p0) represents the output feature value of sampling point p0; R represents the convolution kernel element; p n w(p) represents the position of the nth element in R. n ) indicates that in p n The convolution kernel weights at the location, where x is the input feature map to be detected, x(p0+p n ) represents the sampling position of the feature map to be detected; Δp n This indicates that the sampling positions of the convolution kernel are expanded, {Δp n |n=1,2,...,N};Calculate the spatial distance weights based on the fractional part of the offset, and finally fuse the feature response values ​​of each neighboring point by weighted summation.

[0092] To address the issue that dynamic objects can affect key features, this embodiment adds a CBAM attention mechanism module to the neck network of YOLOv5s to update the feature map before fusion, thereby increasing the effective information content of the feature map after fusion.

[0093] This embodiment introduces a channel attention mechanism into the SENet neck network. It acquires channel-level statistical features through global average pooling, learns and generates channel weight coefficients, enabling the network to autonomously strengthen information-rich feature channels. Simultaneously, a dual-path attention architecture is introduced, achieving more comprehensive feature optimization by parallel integration of channel and spatial attention mechanisms. This overcomes the spatial information loss inherent in SENet and significantly enhances the model's representational ability through multi-granularity feature fusion, thus demonstrating superior performance in various visual tasks.

[0094] The head network in this embodiment ultimately outputs prediction results through three detection heads (80×80, 40×40, and 20×20). The input image is processed by Mosaic data augmentation and 640×640 normalization, and then outputs dynamic target detection results through non-maximum suppression (NMS 0.5) and confidence filtering.

[0095] Secondly, this embodiment uses the improved target detection network YOLOv5s to perform dynamic target detection on the original scene image, and obtains dynamic target bounding boxes in the image scene.

[0096] This embodiment preprocesses the acquired original scene image by using the input of the improved target detection network YOLOv5s, and performs normalization processing to adapt it to the rules of the model.

[0097] The preprocessed image is input into the improved target detection network. The backbone network performs basic feature extraction through convolution operations, reducing redundant computation and memory access, and enhancing spatial feature extraction capabilities. The receptive field is dynamically adjusted through the deformable convolutional network DCNv2, and adaptive sampling is achieved through learnable sampling offsets and weight coefficients to extract features from irregularly shaped targets. The corresponding feature representations are obtained, resulting in multi-scale feature maps.

[0098] The corresponding feature representations are input into the neck network, and feature enhancement is performed under the bidirectional feature fusion architecture. The weight distribution of each channel and spatial position in the feature map is adjusted through the attention mechanism of channel and spatial dimensions. Multi-target features are further enhanced and captured to obtain an enhanced feature map with fused attention weights.

[0099] By detecting multi-target features through a head network, target prediction at different scales is performed, and non-maximum suppression is used to filter out overlapping boxes, resulting in dynamic target boxes in the image scene.

[0100] This embodiment refers to Figure 4 As shown, Figure 4 (a)(b)(c) are in Figure 3 Based on a continuous scene image, the red bounding box shows the dynamic object bounding boxes detected by YOLOv5s, which outline the person, keyboard, mouse, chair, and TV monitor.

[0101] The third step involves using a keyframe synchronization mechanism to associate dynamic target boxes and ORB feature points, and then removing the ORB feature points corresponding to the dynamic target boxes; this yields a feature scene image after one removal step.

[0102] This embodiment synchronizes YOLOv5s detection results and ORB-SLAM3 feature points using keyframe rate synchronization. The keyframe synchronization mechanism is a crucial step in ensuring the time alignment of target detection results with the map building process. The synchronization timestamp is:

[0103] t k =min{t y , t o}

[0104] In this embodiment, t k It is the timestamp of the keyframe, t y and t o These are the timestamps for YOLOv5s and ORB-SLAM3, respectively.

[0105] If an ORB feature point is located within the bounding box of a dynamic target, the feature point is considered dynamic, and dynamic feature points associated with the moving target are removed through a keyframe synchronization mechanism.

[0106] The fourth step involves using epipolar geometry constraints to perform secondary removal of mismatched ORB feature points in adjacent feature scene images to obtain static feature scene images; and then performing pose estimation and / or map construction based on the ORB feature points corresponding to the static feature scene images.

[0107] After the sequential elimination in the third step, the fourth step uses the epipolar geometric constraint method to remove unknown category objects and dynamically determine and eliminate objects. That is, after removing some dynamic feature points in the third step, the feature points of unclassified objects are eliminated a second time.

[0108] A1 and A2 are ORB feature points in adjacent feature scene images. In this embodiment, the distance d from the point A2 to be matched to the corresponding epipolar line h2 is calculated to determine whether A2 matches A1.

[0109] The formula is expressed as follows:

[0110]

[0111] a1 = [x1, y1]

[0112] a2 = [x2, y2]

[0113]

[0114] Where A1 and A2 represent feature points in adjacent feature scene images, a1 and a2 represent the homogeneous coordinate forms of A1 and A2 respectively, h2 represents the epipolar vector, and X, Y, and Z represent line vectors.

[0115] If the calculated distance d is less than or equal to the threshold, then A2 is a correct matching point; if it is greater than the threshold, then it is a mismatched point and will be filtered out by the epipolar geometry constraint algorithm.

[0116] This embodiment retains only static feature points after removal and integrates them into ORB-SLAM3 pose estimation. That is, these pure static feature points are used for robot pose estimation and / or map construction. This reduces the interference of dynamic objects on SLAM and utilizes the geometric consistency of SLAM to optimize the localization accuracy of the detection box, thereby achieving robust semantic SLAM in dynamic environments.

[0117] Based on the above method of the present invention, a simulation experiment was conducted in this embodiment.

[0118] The simulation environment in this embodiment is: GPU NVIDIA GeForce RTX 2080Ti, CPU Intel(R)Xeon(R)Platinum 855C, memory 32GB, Ubuntu 20.04, CUDA 11.3.1.

[0119] For dynamic scenes in ORB-SLAM3, the YOLOv5 object detection network is used to label target objects in RGB image frames, obtain labels for different objects, and acquire the coordinates and object categories of image frames to form a bounding box. If the detection result belongs to a predefined dynamic object class, then, based on prior information, the regions where dynamic objects need to be deleted are added to these image frames. Simulation experiments were conducted on object detection for each of the five dynamic sequences, and four of them are extracted for result presentation. The experimental results are referenced... Figure 3 Figure 4 As shown, the figure, keyboard, mouse, chair, and monitor are outlined.

[0120] Within the framework of ORB-SLAM3, this application combines deep learning methods to effectively improve trajectory accuracy. Furthermore, the more dynamic objects and the greater the amplitude of motion in complex dynamic scenes, the more significant the effect of the optimized target detection network becomes.

[0121] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0122] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic scene visual SLAM method based on deep learning, characterized in that, Includes the following steps: S1. Obtain the original scene image and extract ORB feature points from the scene image; S2. Based on the improved target detection network, dynamic target detection is performed on the original scene image to obtain dynamic target boxes in the image scene; wherein, the improved target detection network adopts a lightweight MobileNetv3 backbone network for deep learning; S3. Use a keyframe synchronization mechanism to associate the dynamic target box and ORB feature points, and remove the ORB feature points corresponding to the dynamic target box; to obtain the feature scene image after one removal. S4. Using epipolar geometry constraints, mismatched ORB feature points in adjacent feature scene images are removed twice to obtain static feature scene images; and pose estimation and / or map construction are performed based on the ORB feature points corresponding to the static feature scene images.

2. The deep learning-based dynamic scene visual SLAM method as described in claim 1, characterized in that, Step S1 specifically includes: The original scene image is acquired, corner points in the image scene are detected by the FAST algorithm, and the corner points are filtered by the Harris corner response value. The selected corner points are described in binary form using the BRIEF descriptor; the described corner points are then used as ORB feature points in the image scene.

3. The deep learning-based dynamic scene visual SLAM method as described in claim 1, characterized in that, The improved target detection network includes: an input end, a backbone network, a neck network, and a head network; The input terminal is used for image preprocessing; The backbone network adopts the lightweight MobileNetv3 and introduces the deformable convolutional network DCNv2 for multi-scale feature extraction. An attention mechanism is introduced into the neck network to integrate attention data from both channel and spatial dimensions for feature optimization. The head network is used to predict targets at different scales through the detection head.

4. The deep learning-based dynamic scene visual SLAM method as described in claim 3, characterized in that, Step S2 specifically includes: The original scene image is preprocessed through the input terminal; The preprocessed image is input into the backbone network, basic features are extracted through convolution operations, and the receptive field is dynamically adjusted through the deformable convolutional network DCNv2. Features of irregularly shaped targets are extracted through learnable sampling offsets and weight coefficients, thus obtaining the corresponding feature representation. The corresponding feature representations are input into the neck network, and feature enhancement is performed under the bidirectional feature fusion architecture. The weight distribution of each channel and spatial position in the feature map is adjusted through the attention mechanism of channel and spatial dimensions; further enhancing and capturing multi-target features; The head network detects the multi-target features and performs target prediction at different scales to obtain dynamic target bounding boxes in the image scene.

5. The deep learning-based dynamic scene visual SLAM method as described in claim 4, characterized in that, The deformable convolutional network DCNv2 includes an offset prediction module; The offset prediction module uses lightweight convolutional layers to predict the offset on the input feature map corresponding to each output position, and then uses the predicted offset to perform adaptive sampling on the input feature map. Based on the offset, it jumps to a new sampling point and performs weighted summation to generate a feature map for extracting irregularly shaped targets.

6. The deep learning-based dynamic scene visual SLAM method as described in claim 5, characterized in that, The deformable convolutional network DCNv2 dynamically adjusts the receptive field and extracts features from irregularly shaped targets through learnable sampling offsets and weight coefficients. Specifically, it includes: The deformable convolutional network DCNv2 enables the feature sampling grid to be dynamically adjusted according to the input content by adaptively learning the spatial offset. The calculation process of the deformable convolutional network DCNv2 is expressed by the following formula: Where y(p0) represents the output feature value of sampling point p0; R represents the convolution kernel element; p n w(p) represents the position of the nth element in R. n ) indicates that in p n The convolution kernel weights at the location, where x is the input feature map to be detected, x(p0+p n ) represents the sampling position of the feature map to be detected; Δp n This indicates that the sampling positions of the convolution kernel are expanded.

7. The deep learning-based dynamic scene visual SLAM method as described in claim 4, characterized in that, Step S3 specifically includes: S31. Associate the dynamic target box and ORB feature points by synchronizing keyframe timestamps; The formula is expressed as follows: t k =min{t y ,t o } Among them, t k t is the timestamp of the keyframe. y and t o These are the timestamps for the object detection network and ORB feature point extraction, respectively. S32. The ORB feature points located within the dynamic target box are taken as dynamic feature points and removed; the feature scene image after one removal is obtained.

8. The deep learning-based dynamic scene visual SLAM method as described in claim 7, characterized in that, In step S4, the second removal of mismatched ORB feature points in adjacent feature scene images using epipolar geometry constraints specifically includes: By calculating the distance d from the point A2 to be matched to the corresponding epipolar line h2 of ORB feature point A1 in adjacent feature scene images, it is determined whether A2 matches A1. The formula is expressed as follows: a1 = [x1, y1] a2 = [x2, y2] Where A1 and A2 represent feature points in adjacent feature scene images, a1 and a2 represent the homogeneous coordinate forms of A1 and A2 respectively, h2 represents the epipolar vector, and X, Y, and Z represent line vectors; If the calculated distance d is less than or equal to the preset threshold, then A2 is retained as the correct matching point; If the calculated distance d is greater than the preset threshold, then A2, as a mismatch point, needs to be removed.

Images