Vision slam optimization method in dynamic scene based on improved yolov5 through weighted characteristics

Abstract

The application discloses a kind of based on improved YOLOv5 through weighted characteristics realizes visual SLAM optimization method in dynamic scene, comprising: obtaining the RGB image and depth image of image frame;RGB image is segmented out prior dynamic object object semantic mask by improved YOLOv5 network;Prior dynamic object object semantic mask is segmented boundary optimization based on region growing algorithm, and boundary optimization semantic mask is obtained;Boundary optimization semantic mask is extracted to ORB semantic static feature point;RGB image, depth image is detected potential dynamic object object geometry mask by multi-view geometry;Boundary optimization semantic mask is combined with potential dynamic object object geometry mask to obtain dynamic target detection result;According to dynamic target detection result, ORB semantic static feature point is given weight value;The weight value of camera pose and ORB semantic static feature point is estimated by BA joint optimization.This application improves positioning accuracy and robustness, while still meet certain real-time requirements.

Description

TECHNICAL FIELD

[0001] The application relates to a visual SLAM optimization method based on improved YOLOv5 and realized by weighted features in a dynamic scene, and belongs to the field of visual SLAM positioning. BACKGROUND

[0002] Visual Simultaneous Localization and Mapping (VSLAM) refers to self-position estimation and environment map construction of a robot in an unknown environment through a visual sensor (such as a monocular camera, an RGB-D camera and a depth camera). With the development of robot technology, VSLAM has become a research hotspot in the fields of computer vision, automatic driving, robots, unmanned aerial vehicles and the like. At present, most of visual Simultaneous Localization and Mapping (SLAM) algorithms are designed based on static scenes, and the moving objects in the scene are not considered. Due to the moving objects existing in real scenes, feature point mismatching of a visual odometer is caused, so that the positioning and mapping accuracy of the SLAM system is affected, and the system does not have high robustness in actual application. Prior knowledge such as semantic information is difficult to identify objects without pre-training. In addition to the prior knowledge, frame observation in a short time span is insufficient to determine dynamic visual features, because some dynamic objects can remain stationary in a short time interval and are incorrectly identified as static components. Therefore, it is still challenging to improve the performance of VSLAM in a dynamic scene. SUMMARY

[0003] The application provides a visual SLAM optimization method based on improved YOLOv5 and realized by weighted features in a dynamic scene, so as to combine improved YOLOv5, a region growing algorithm, a multi-view geometry method and weighted features to realize visual SLAM optimization in a dynamic scene.

[0004] The technical scheme of the application is as follows:

[0005] According to an aspect of the present application, a visual SLAM optimization method in a dynamic scene is provided by implementing a weighted feature based on an improved YOLOv5, comprising: step 1: obtaining an RGB image and a depth image of an image frame; step 2: segmenting the RGB image through the improved YOLOv5 network to obtain a prior dynamic object object semantic mask; step 3: performing segmentation boundary optimization on the prior dynamic object object semantic mask based on a region growing algorithm to obtain a boundary-optimized semantic mask; step 4: extracting ORB semantic static feature points from the boundary-optimized semantic mask; step 5: detecting a potential dynamic object object geometric mask from the RGB image and the depth image through multi-view geometry; step 6: combining the boundary-optimized semantic mask and the potential dynamic object object geometric mask to obtain a dynamic target detection result; assigning a weight value to the ORB semantic static feature points according to the dynamic target detection result to obtain an initial weight value of the semantic static feature points; step 7: estimating the camera pose and the weight value of the ORB semantic static feature points through BA joint optimization according to the initial weight value of the semantic static feature points; and step 8: performing map construction and loop detection on the optimized camera pose.

[0006] The improved YOLOv5 network is used for target detection and semantic segmentation at the same time, and comprises an encoder and a decoder. The encoder is composed of a backbone network and a neck network. Five backbone output features are obtained according to the backbone network, and three neck output features are obtained according to the neck network. The decoder comprises a detection head and a segmentation head. The segmentation head comprises three up-sampling modules. The second backbone output feature of the backbone network and the third neck output feature of the neck network are taken as the input of the first up-sampling module. The output of the first up-sampling module and the first backbone output feature of the backbone network are taken as the input of the second up-sampling module. The output of the second up-sampling module is taken as the input of the third up-sampling module. The output of the third up-sampling module is taken as the output of the segmentation head.

[0007] The step 3 comprises:

[0008] Step 3.1, selecting a seed point O' for each prior dynamic object and adding the seed point O' to a seed list. The position of the seed point O' is calculated as:

[0009]

[0010] In the formula, (x j ,y j ) is the jth pixel point in the current prior dynamic object object semantic mask; and n is the total number of pixel points.

[0011] Step 3.2, judging whether the seed point list is empty, if the seed point list is empty, ending the process, if the seed point list is not empty, randomly taking out a seed point from the list;

[0012] Step 3.3, marking the position of the seed point as a grown region, and traversing the points in the seed point field;

[0013] Step 3.4, if the depth difference between the points in the seed point field and the seed point is less than the set first threshold Th, the pixel value of the point is set to 255, and the point is put into the seed point list, steps 3.2-3.4 are repeated until the seed point list is empty, the algorithm ends, and the boundary optimized semantic mask is obtained, otherwise, steps 3.2-3.4 are directly repeated until the seed point list is empty, the algorithm ends, and the boundary optimized semantic mask is obtained.

[0014] The first threshold Th = ω1 * d; wherein ω1 is a depth weight value; and d is a depth map scale factor.

[0015] The BA joint optimization is used to estimate the camera pose and the weight value of the ORB semantic static feature point, specifically, the BA optimization is multiplied by the initial weight value of the semantic static feature point to estimate the camera pose and the weight value of the ORB semantic static feature point.

[0016] According to another aspect of the present application, a visual SLAM optimization system in a dynamic scene based on an improved YOLOv5 through a weighted feature is provided, comprising: a module for acquiring an RGB image and a depth image of an image frame; a module for segmenting a prior dynamic object object semantic mask from the RGB image through an improved YOLOv5 network; a module for performing boundary optimization on the prior dynamic object object semantic mask based on a region growing algorithm to obtain a boundary optimized semantic mask; a module for extracting ORB semantic static feature points from the boundary optimized semantic mask; a module for detecting a potential dynamic object object geometric mask from the RGB image and the depth image through multi-view geometry; a module for combining the boundary optimized semantic mask and the potential dynamic object object geometric mask to obtain a dynamic target detection result; a module for assigning a weight value to the ORB semantic static feature points according to the dynamic target detection result to obtain an initial weight value of the semantic static feature points; a module for estimating the camera pose and the weight value of the ORB semantic static feature points through BA joint optimization according to the initial weight value of the semantic static feature points; and a module for performing map construction and loop detection on the optimized camera pose.

[0017] The beneficial effects of the present application are: the present application increases the semantic segmentation thread and the multi-view geometry thread based on the ORB-SLAM2 framework to detect dynamic feature points, thereby improving the positioning accuracy and robustness of the SLAM system while still meeting certain real-time requirements. Specifically, in the semantic segmentation thread, the improved YOLOv5 of the present application simultaneously performs target detection and semantic segmentation, and then uses the prior dynamic object semantic mask to optimize the dynamic object boundary based on the region growing algorithm to improve the segmentation accuracy of the prior dynamic object detection; in the geometry thread, the present application uses a multi-view geometry method to detect potential dynamic objects; the boundary optimized semantic mask obtained by the region growing algorithm and the geometric mask obtained by the multi-view geometry method are combined to obtain more accurate dynamic targets in the image frame. Further, different weights are given to the semantic static feature points according to the dynamic target detection results, and the camera pose and feature point weight are estimated by BA joint optimization, thereby reducing the feature point pairs participating in the BA optimization, and finally obtaining more accurate camera pose estimation and feature point weight, improving the positioning accuracy and robustness of the traditional visual SLAM system in a dynamic scene, and overcoming the interference of dynamic objects on the traditional SLAM system. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The system framework diagram of the present application is shown in the figure;

[0019] Figure 2 The improved YOLOv5 structure diagram of the present application is shown in the figure;

[0020] Figure 3 The flowchart of the region boundary optimization algorithm of the present application is shown in the figure;

[0021] Figure 4 The multi-view geometry dynamic point detection principle diagram of the present application is shown in the figure;

[0022] Figure 5 The object detection and dynamic feature point removal effect diagram in the present application is shown in the figure;

[0023] Figure 6 The trajectory error comparison diagram and the trajectory error comparison diagram in XYZ direction of the present application method and ORB-SLAM2 using TUM data set walking_xyz sequence for pose estimation are shown in the figure;

[0024] Figure 7 The relative error trajectory comparison diagram of the present application method and ORB-SLAM2 using TUM data set walking_xyz sequence for pose estimation is shown in the figure;

[0025] Figure 8 The camera trajectory comparison of the present application method and ORB-SLAM2 in a real dynamic scene is shown in the figure. DETAILED DESCRIPTION

[0026] The invention will be further described below with reference to the accompanying drawings and embodiments, but the scope of the invention is not limited to the description.

[0027] Example 1: As Figures 1-7 As shown in one aspect of the present invention, a method for optimizing visual SLAM in dynamic scenes based on an improved YOLOv5 network using weighted features is provided, comprising: Step 1: acquiring RGB and depth images of an image frame; Step 2: segmenting the RGB image into a priori dynamic object semantic mask using an improved YOLOv5 network; Step 3: optimizing the segmentation boundary of the priori dynamic object semantic mask using a region growing algorithm to obtain a boundary-optimized semantic mask; Step 4: extracting ORB semantic static feature points from the boundary-optimized semantic mask; Step 5: detecting potential dynamic object geometric masks from the RGB and depth images using multi-view geometry detection; Step 6: combining the boundary-optimized semantic mask with the potential dynamic object geometric mask to obtain a dynamic target detection result; assigning weights to the ORB semantic static feature points based on the dynamic target detection result to obtain the initial weights of the semantic static feature points; Step 7: estimating the camera pose and the weights of the ORB semantic static feature points through joint optimization using BA based on the initial weights of the semantic static feature points; Step 8: performing map construction and loop closure detection on the optimized camera pose.

[0028] Furthermore, the improved YOLOv5 network is used for simultaneous object detection and semantic segmentation; it includes an encoder and a decoder. The encoder consists of a backbone network and a neck network, obtaining five backbone output features based on the backbone network and three neck output features based on the neck network. The decoder includes a detection head and a segmentation head. The segmentation head includes three upsampling modules. The second backbone output feature of the backbone network and the third neck output feature of the neck network are used as inputs to the first upsampling module. The output of the first upsampling module and the first backbone output feature of the backbone network are used as inputs to the second upsampling module. The output of the second upsampling module is used as input to the third upsampling module. The output of the third upsampling module is used as the output of the segmentation head.

[0029] Further, step 3 includes:

[0030] Step 3.1: Select seed point O′ for each prior dynamic object and add it to the seed list; the position of seed point O′ is calculated as follows:

[0031]

[0032] In the formula, (x j ,yj ) is the jth pixel point in the current prior dynamic object object semantic mask; n is the total number of pixel points;

[0033] Step 3.2, judging whether the seed point list is empty, if the seed point list is empty, ending the process, if the seed point list is not empty, randomly taking a seed point from the list;

[0034] Step 3.3, marking the position of the taken seed point as a grown area, and traversing the points in the seed point field;

[0035] Step 3.4, if the depth difference between the points in the seed point field and the seed point is less than the set first threshold Th, the pixel value of the point is set to 255, and the point is put into the seed point list, and steps 3.2-3.4 are repeated until the seed point list is empty, the algorithm ends, and the boundary optimized semantic mask is obtained, otherwise, steps 3.2-3.4 are directly repeated until the seed point list is empty, the algorithm ends, and the boundary optimized semantic mask is obtained.

[0036] Further, the first threshold Th = ω1 * d; wherein ω1 is a depth weight value; d is a depth map scale factor.

[0037] Further, the BA joint optimization is used to estimate the camera pose and the weight value of the ORB semantic static feature point, specifically: multiplying the BA optimization by the initialization weight value of the semantic static feature point, to estimate the camera pose and the weight value of the ORB semantic static feature point.

[0038] According to another aspect of the embodiment of the application, a visual SLAM optimization system in a dynamic scene based on an improved YOLOv5 through a weighted feature is provided, comprising: a module for acquiring an RGB image and a depth image of an image frame; a module for segmenting a prior dynamic object object semantic mask from the RGB image through an improved YOLOv5 network; a module for performing boundary optimization on the prior dynamic object object semantic mask based on a region growing algorithm to obtain a boundary optimized semantic mask; a module for extracting ORB semantic static feature points from the boundary optimized semantic mask; a module for detecting a potential dynamic object object geometric mask from the RGB image and the depth image through multi-view geometry; a module for combining the boundary optimized semantic mask and the potential dynamic object object geometric mask to obtain a dynamic target detection result; a module for assigning a weight value to the ORB semantic static feature points according to the dynamic target detection result to obtain an initialization weight value of the semantic static feature points; a module for estimating the camera pose and the weight value of the ORB semantic static feature points through BA joint optimization according to the initialization weight value of the semantic static feature points; and a module for performing map construction and loop detection on the optimized camera pose.

[0039] Embodiment 2: The following describes an optional specific implementation of the present application in combination with the accompanying drawings, a visual SLAM optimization method based on improved YOLOv5 through weighted features in a dynamic scene, comprising:

[0040] Step 1: Obtain an RGB image and a depth image of an image frame through an RGB camera;

[0041] Step 2: The RGB image obtained in step 1 is segmented by an improved YOLOv5 network to obtain a prior dynamic object object semantic mask;

[0042] Step 3: The prior dynamic object object semantic mask obtained in step 2 is subjected to segmentation boundary optimization based on a region growing algorithm to obtain a boundary-optimized semantic mask;

[0043] Step 4: The boundary-optimized semantic mask obtained in step 3 is subjected to extraction of ORB semantic static feature points;

[0044] Step 5: The RGB image and the depth image obtained in step 1 are subjected to potential dynamic object object geometric mask detection through multi-view geometry;

[0045] Step 6: The boundary-optimized semantic mask obtained by the region growing algorithm in step 3 is fused with the potential dynamic object object geometric mask obtained in step 5 to obtain a dynamic target detection result; the ORB semantic static feature points are given a weight value according to the dynamic target detection result;

[0046] Step 7: The ORB semantic static feature points based on the weight value obtained in step 6 are subjected to BA joint optimization to estimate the camera pose and the weight value of the ORB semantic static feature points;

[0047] Step 8: The optimized camera pose obtained in step 7 is subjected to map construction and loop detection.

[0048] In combination Figure 1 , the embodiment adds a semantic segmentation thread and a geometry thread in the ORB-SLAM2 system, and is a multi-thread parallel semantic dynamic visual SLAM system. The system detects dynamic feature points through the semantic segmentation thread and the geometry thread. In the semantic segmentation thread, the actual edge of the dynamic object depth image is obtained by using the prior information of semantic segmentation to optimize the boundary of the dynamic object region, and then the ORB feature points are extracted through the optimization result. The geometry thread determines the motion state of the object by using a multi-view geometry method. Then, the detection results of the two threads are combined to give the semantic static feature points a weight value, and finally the BA joint optimization is performed in combination with the pose and the weight value.

[0049] In the semantic segmentation thread, target detection and semantic segmentation are simultaneously performed through improved YOLOv5, such as Figure 2For the improved YOLOv5 model, the improved YOLOv5 network globally shares an encoder, which is composed of a Backbone network and a Neck network. The Backbone network adopts the Backbone feature extraction network of YOLOv5s as the backbone network for simultaneous target detection and semantic segmentation. The Neck network mainly consists of two modules. Respectively, they are the fast spatial pyramid pooling (SPPF) module improved on the basis of the spatial pyramid pooling (SPP) module and the feature pyramid network (FPN) module. SPPF generates features of different scales, which are then fused. The FPN module fuses features of different semantic levels, which can make the generated features contain information of multiple scales and multiple semantic levels. At the same time, the two tasks of target detection and semantic segmentation are completed, so the decoder used needs two heads, namely the detection head and the segmentation head. The detection head is the same as the detection head of the traditional YOLOv5, which adopts an anchor-based multi-scale detection method. First, the FPN transmits semantic features from top to bottom; then, the path aggregation network (PAN) transmits positioning features from bottom to top. Combining the FPN and the PAN can obtain a better feature fusion effect. Then, the multi-scale fusion feature map in the PAN is used for target detection, and the bottom layer of the FPN is transmitted to the segmentation branch, which has a size of (W / 8, H / 8, 256). After three times of upsampling, the output feature map has a size of (W, H, 2). The nearest interpolation method is used in the upsampling layer to reduce the calculation cost and obtain the image segmentation head.

[0050] The improved YOLOv5 model uses the MS COCO dataset as the training sample to train the improved YOLOv5 network. The MS COCO dataset contains 80 different object categories, which are sufficient to contain most common objects in indoor environments.

[0051] After the improved YOLOv5 model segments the prior dynamic object object semantic mask, the boundary optimization semantic mask of the dynamic object object is obtained based on the region growing algorithm, such as Figure 3 is the flowchart of the region growing algorithm:

[0052] Step 3.1, select a seed point O' for each prior dynamic object and add it to the seed list. The position of the seed point O' is calculated as:

[0053]

[0054] In the formula, (x j ,y j ) is the jth pixel point in the prior dynamic object object semantic mask; n is the total number of pixel points

[0055] Step 3.2, judging whether the seed point list is empty, if the seed point list is empty, ending the process, if the seed point list is not empty, randomly taking a seed point from the list;

[0056] Step 3.3, marking the seed point position as a grown region, and traversing the points in the seed point field; in the embodiment of the application, the seed point is diffused in a four-connected / eight-connected manner to obtain the seed point field;

[0057] Step 3.4, if the depth difference between the point in the seed point field and the seed point is less than the set first threshold Th, the pixel value of the point is set to 255, and the point is put into the seed point list, steps 3.2-3.4 are repeated until the seed point list is empty, the algorithm ends, and the boundary-optimized semantic mask is obtained; otherwise, steps 3.2-3.4 are directly repeated until the seed point list is empty, the algorithm ends, and the boundary-optimized semantic mask is obtained.

[0058] The first threshold Th = ω1 * d; wherein ω1 is a depth weight, and the application takes 0.03; and d is a scale factor of the depth map.

[0059] Non-prior dynamic objects are processed by multi-view geometry, such as Figure 4 is a principle diagram for detecting dynamic points by multi-view geometry, assuming that p is a key point on a key frame, p' is a point of p projected to a coordinate system of a current frame, and P is a three-dimensional point corresponding to p. For each input frame, 5 key frames with the highest overlap are selected, the perspective value γ CF of each key point in the current frame (CF) is calculated, and the perspective value γ KF of the historical frame (KF) is calculated, by satisfying:

[0060] Δγ = |γ cf - γ kf | > T

[0061] , the key point is determined to be a dynamic point, T is a set second threshold. At the same time, the depth value H CF of the key point in the current frame and the projection depth H proj of the historical frame in the current frame are calculated, the difference between the depth values is ΔH = |H proj - H CF |, if less than a set threshold, the key point is determined to be a static feature point, if greater than the set threshold, the key point is determined to be a dynamic feature point. And the detection result produces a binary geometry mask.

[0062] Further, the step 6 includes:

[0063] Finally, the optimized semantic mask and the potential dynamic object geometry mask obtained by multi-view geometry are combined to determine the complete dynamic target contour; based on the dynamic target detection result, the weight of the semantic static feature point is defined:

[0064] ε i = λmask(u,υ) + ωmask(u,υ)

[0065] In the formula, λ and ω represent the contribution proportion of the semantic mask and the geometry mask to the feature point weight, and λ + ω = 1, and mask(u,υ) is the dynamic target detection result. Through ε i Initialize the weight of the semantic static feature point.

[0066] Further, the step 7 is specifically:

[0067] Multiply the BA optimization by the initialized weight of the semantic static feature point to estimate the camera pose and the weight of the ORB semantic static feature point:

[0068]

[0069] Wherein, {R * ,t * ,ε i *} represents the estimated camera pose and the weight of the semantic static feature point; p i is the projected pixel coordinate; P i is the space point coordinate; and π represents the projection from the three-dimensional coordinate to the pixel coordinate system. The contribution of the feature point to the objective function can be reasonably allocated according to the reliability of the feature point. The feature point with larger weight is more reliable, so it has greater contribution to the objective function, and the feature point with smaller weight has smaller contribution. The optimization variable changes from the original camera pose to the camera pose and the weight of the feature point. The weight of the optimized feature point can more accurately represent the reliability of the feature point.

[0070] Further, the Levenberg-Marquardt method in the g2o library is used to perform BA joint optimization to estimate the camera pose and the weight of the feature point, and the partial derivative of the error term to the optimization variable needs to be calculated manually, so the partial derivative of the feature point weight is:

[0071]

[0072] Wherein, (u i ,υ i ) represents the projected pixel coordinate of the camera, and P = (X, Y, Z) represents the space point coordinate point in the camera coordinate system, wherein f x , f y , c x , c y, bf are parameters of the camera, all known.

[0073] The TUM RGB dataset is used below to evaluate the present embodiment.

[0074] The TUM dataset is a commonly used dataset for evaluating RGB-D SLAM system benchmarks, which contains 39 sequences, each sequence containing RGB images, depth images and related ground truth camera poses. The walking sequence is a dynamic sequence of the TUM dataset. When the SLAM system of the present application is tested using the walking sequence, all prior objects can be identified, the dynamic and static regions of the image are divided, and the feature points in the dynamic region are all deleted. The potential dynamic region needs to be further judged by the multi-view tracking thread to determine the dynamic and static states of the feature points.

[0075] The experiments are all run on a computer with an Intel(R) Core(TM) i7-9750H CPU, 16G of memory, an RTX1650 GPU and 8G of video memory. The system environment is Ubuntu 18.04, CUDA 11.1 is used, and the deep learning framework used for model training is Pytorch v1.8.0.

[0076] Figure 5 The effect of detecting objects in the scene and removing dynamic feature points is shown. Since people are dynamic objects, the feature points in their detection boxes are all dynamic feature points and are all removed, while static objects such as monitors, the feature points in their detection boxes are all retained.

[0077] Table 1 The root mean square error RMSE and the median Median of the ATE index of the present application method and ORB-SLAM2 and DynaSLAM are obtained by experiments on 6 sequences of the TUM dataset, and the percentage of improvement. Compared with ORB-SLAM2, the present application achieves greater improvement on four groups of walking high dynamic sequences. For example, the ATE index of w_xyz (i.e. the walking_xyz sequence), the root mean square error RMSE increases by 98.17%, and at the same time, the present application method has no obvious or negative effect on the improvement of ORB-SLAM2 in low dynamic environment. This shows that the proposed dynamic object removal front end can effectively improve the performance of ORB-SLAM2 in highly dynamic environments.

[0078] Table 1 Absolute trajectory error (ATE) comparison unit (m)

[0079]

[0080] Figure 6(a) and (b) in FIG. 6 show the trajectory error comparison chart of ORB-SLAM2 and the application in the walking_xyz sequence, Figure 6 (c) and (d) in FIG. 6 show the trajectory error comparison chart in XYZ direction. Figure 7 (a) and (b) show the relative trajectory error chart of the application and ORB-SLAM2 running the walking_xyz sequence, the translation error is also within a reasonable range, and the influence on the pose estimation in the indoor dynamic environment can be ignored. Figure 6 (a), 6(c), Figure 7 (a) is the experimental result of ORB-SLAM2, Figure 6 (b), 6(d), Figure 7 (b) is the experimental result of the application. Figure 8 FIG. 6 shows the comparison of camera trajectories of ORB-SLAM2 and the application in the real dynamic scene, and the camera trajectory of the application shows better closed loop effect than ORB-SLAM2. Due to the existence of dynamic ORB feature points in the real scene, the dashed trajectory estimated by ORB-SLAM2 has multiple offset phenomena, and the final closed loop effect is poor.

[0081] Through the test, it is known that the time consumed by the system of the application in the detection and segmentation task is about 15-20 ms, and the time consumed in tracking each frame of image is about 60-80 ms. Compared with the MaskRCNN network model used by DynaSLAM, the YOLOv5 network model used by the application has faster detection speed, and the detection speed is very important for the real-time performance of the system. The application can achieve high positioning accuracy in low dynamic and high dynamic scenes, and has good real-time performance.

[0082] The specific embodiments of the application are described in detail above in combination with the drawings, but the application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.

Claims

1. A method for visual SLAM optimization in a dynamic scene based on improved YOLOv5 by weighting features, characterized in that, The method comprises the following steps: Step 1: obtaining an RGB image and a depth image of an image frame; Step 2: segmenting a prior dynamic object object semantic mask from the RGB image through an improved YOLOv5 network; Step 3: performing segmentation boundary optimization on the prior dynamic object object semantic mask based on a region growing algorithm to obtain a boundary-optimized semantic mask; Step 4: extracting ORB semantic static feature points from the boundary-optimized semantic mask; Step 5: detecting a potential dynamic object object geometric mask from the RGB image and the depth image through multi-view geometry; Step 6: combining the boundary-optimized semantic mask and the potential dynamic object object geometric mask to obtain a dynamic target detection result; assigning a weight value to the ORB semantic static feature points according to the dynamic target detection result to obtain an initial weight value of the semantic static feature points; Step 7: estimating the camera pose and the weight value of the ORB semantic static feature points through BA joint optimization according to the initial weight value of the semantic static feature points; Step 8: performing map construction and loop detection on the optimized camera pose.

2. The method of claim 1, wherein the improved YOLOv5-based visual SLAM optimization method is implemented by weighting characteristics in a dynamic scene. The improved YOLOv5 network is used for simultaneous target detection and semantic segmentation. The improved YOLOv5 network comprises an encoder and a decoder, the encoder comprises a backbone network and a neck network, five backbone output features are obtained from the backbone network, and three neck output features are obtained from the neck network; the decoder comprises a detection head and a segmentation head, the segmentation head comprises three up-sampling modules, the second backbone output feature of the backbone network and the third neck output feature of the neck network are taken as the input of the first up-sampling module, the output of the first up-sampling module and the first backbone output feature of the backbone network are taken as the input of the second up-sampling module, the output of the second up-sampling module is taken as the input of the third up-sampling module, and the output of the third up-sampling module is taken as the output of the segmentation head.

3. The method of claim 1, wherein the improved YOLOv5-based visual SLAM optimization method is implemented by weighting characteristics in a dynamic scene. The step 3 comprises: Step 3.1: selecting a seed point O' for each prior dynamic object and adding the seed point O' to a seed list; the position of the seed point O' is calculated as: where (x j ,y j ) is the jth pixel point in the current prior dynamic object object semantic mask; n is the total number of pixel points; Step 3.2: judging whether the seed list is empty, if the seed list is empty, ending the process, if the seed list is not empty, randomly taking a seed point from the list; Step 3.3: marking the position of the taken seed point as a grown region, and traversing points in the field of the seed point; Step 3.4: if the depth difference between the point in the field of the seed point and the seed point is less than a first threshold Th, the pixel value of the point is set to 255, the point is put into the seed list, steps 3.2-3.4 are repeated until the seed list is empty, the algorithm is ended, and a boundary-optimized semantic mask is obtained, otherwise, steps 3.2-3.4 are directly repeated until the seed list is empty, the algorithm is ended, and a boundary-optimized semantic mask is obtained.

4. The method of claim 3, wherein the improved YOLOv5-based visual SLAM optimization method is implemented by weighting characteristics in a dynamic scene. The first threshold Th = ω1 * d; wherein ω1 is a depth weight value, and d is a scale factor of the depth image.

5. The method of claim 1, wherein the improved YOLOv5-based visual SLAM optimization method is implemented by weighting characteristics in a dynamic scene. The camera pose and the weight of the ORB semantic static feature point are estimated by the BA joint optimization, specifically, the BA optimization is multiplied by the initial weight of the semantic static feature point to estimate the camera pose and the weight of the ORB semantic static feature point.

6. An improved YOLOv5-based visual SLAM optimization system in a dynamic scene by weighting features, characterized in that, Comprise: A module for acquiring an RGB image and a depth image of an image frame; A module for segmenting a prior dynamic object object semantic mask from the RGB image through an improved YOLOv5 network; A module for performing region growing algorithm-based segmentation boundary optimization on the prior dynamic object object semantic mask to obtain a boundary-optimized semantic mask; A module for extracting ORB semantic static feature points from the boundary-optimized semantic mask; A module for detecting a potential dynamic object object geometric mask from the RGB image and the depth image through multi-view geometry; A module for combining the boundary-optimized semantic mask and the potential dynamic object object geometric mask to obtain a dynamic target detection result; A module for assigning a weight to the ORB semantic static feature point according to the dynamic target detection result to obtain an initial weight of the semantic static feature point; A module for estimating the camera pose and the weight of the ORB semantic static feature point by the BA joint optimization according to the initial weight of the semantic static feature point; A module for performing map construction and loop detection on the optimized camera pose.

Images