Indoor vision SLAM positioning method based on dynamic scene

By combining the YOLOv5 object detection model and SLAM algorithm, identifying and eliminating dynamic object feature points, the positioning accuracy and robustness of visual SLAM in dynamic environments is solved, and more efficient dynamic object processing is achieved, improving the stability of the system and Real-time.

CN119027651AInactive Publication Date: 2024-11-26HEFEI UNIV OF TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411163566.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing visual SLAM technology is difficult to accurately distinguish between dynamic objects and static environment feature points in dynamic environments, resulting in a decrease in positioning accuracy and robustness. Traditional geometric methods fail in high dynamic environments, while deep learning algorithms consume high computing resources and poor real-time performance. .

Method used

Combining the YOLOv5 object detection model and SLAM algorithm, through object detection, significance detection and optical flow method, dynamic object feature points are identified and eliminated, the anchor box is adjusted using the HC algorithm, the feature point matching range is narrowed, and the polar line distance is calculated to filter abnormal feature points. Ensure static feature point tracking and local mapping.

Benefits of technology

It improves the positioning accuracy and robustness of visual SLAM in dynamic scenarios, avoids the stability problems of traditional methods, and reduces the computing resource consumption of deep learning algorithms, ensuring the overall stability of the SLAM system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027651A_ABST
    Figure CN119027651A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of dynamic visual positioning, and particularly discloses an indoor visual SLAM positioning method based on a dynamic scene, and the method comprises the steps: deploying a YOLOv5 target detection model generated through pre-training on the basis of a COCO128 data set and an SLAM algorithm on a robot platform; an image sequence is obtained through an image sensor carried by the robot. According to the method, the positioning precision and robustness of the visual SLAM algorithm in a dynamic scene are improved through the YOLOv5 algorithm, and different from the traditional target detection and visual SLAM algorithm, all feature points in a detection frame are not removed, so that the overall stability of an SLAM system is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic visual positioning, and in particular to an indoor visual SLAM positioning method based on a dynamic scene. Background Art

[0002] With the continuous advancement of artificial intelligence technology, simultaneous localization and mapping (SLAM) technology is also constantly updated. SLAM refers to the technology that uses specific sensors to determine the subject's position in real time and build an environmental map during movement in an unknown environment without prior information. This technology is widely used in fields such as autonomous driving, drone navigation, and robot navigation. Visual SLAM is a special SLAM system that mainly uses cameras as sensors to obtain information about the surrounding environment. Compared with other perception methods, visual SLAM cameras have lower cost, simple deployment, and can capture rich image texture and semantic information, so they are more widely used;

[0003] Most of the mainstream visual SLAM frameworks are based on the assumption of a static environment. However, in practical applications, there are often a large number of complex dynamic objects in the environment, which makes it difficult for robots to correctly distinguish between dynamic objects and feature points provided by the static environment. This situation will introduce erroneous observation data and destroy the matching relationship between feature points between frames, which will lead to errors in camera pose estimation and map construction, and reduce the accuracy and robustness of the system. Therefore, how to accurately eliminate dynamic objects in complex environments to ensure the efficiency and robust operation of the system is an important issue that needs to be solved in the current field of visual SLAM;

[0004] In order to achieve stable and efficient operation of the system, there are currently two mainstream solutions for visual SLAM in dynamic environments to remove dynamic objects from static backgrounds: one is based on traditional geometric methods combined with camera motion estimation, and the motion of feature points is calculated through methods such as polar geometric constraints and the camera's own motion, and the matching feature points are screened for consistency to remove outliers. However, the accuracy of this method is affected by the proportion of dynamic objects in the environment. When there are many dynamic objects, the feature points belonging to the static background are outliers. The method has poor stability and is prone to failure. The other is an algorithm based on deep learning, which identifies parts of the environment that may be dynamic objects and removes their corresponding feature points. For example, Dyna-SLAM2 uses an instance segmentation algorithm in deep learning, which can classify each pixel and achieve accurate image segmentation at the pixel level, but instance segmentation requires large computing resources and takes a long time, which is at a disadvantage in the SLAM framework that requires real-time performance. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings in the prior art and propose an indoor visual SLAM positioning method based on dynamic scenes.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An indoor visual SLAM positioning method based on a dynamic scene comprises the following steps: (1) deploying a YOLOv5 target detection model and SLAM algorithm generated by pre-training on a COCO128 dataset onto a robot platform;

[0008] (2) The image sequence is acquired through the image sensor carried by the robot, and the target detection of the scene image is performed in real time through the YOLOv5 algorithm. The classified objects in the model are identified and the position boundaries are described by anchor frames;

[0009] (3) Using the HC algorithm to perform saliency detection on the target anchor box range;

[0010] (4) Adjust the anchor frame according to the boundary of the salient map and reduce the area of ​​the anchor frame so that it covers the salient area more accurately.

[0011] (5) Classify the objects in the current image into high, medium, and low motion probabilities;

[0012] (6) Determine whether a potential dynamic object is detected in the current image;

[0013] Step 61, if not, determine whether there is a dynamic object area in the previous frame; if not, the YOLOv5 target detection algorithm network fails, and the current frame process ends. If the previous frame exists, the previous frame and the current frame are matched by the optical flow method, and the corresponding dynamic object area anchor frame of the previous frame is displaced accordingly and drawn to the current frame, and the sequential container is updated to store the data of the dynamic object feature points matched and tracked;

[0014] Step 62, if present, use a sequential container to store the data of each potential dynamic object;

[0015] (7) Extract feature points from the current frame image;

[0016] Step 71, performing optical flow matching on feature points outside the potential motion area to calculate a basic matrix between two frames;

[0017] Step 72, screening the feature points contained in the potential dynamic area based on the epipolar constraint, judging whether it is an abnormal feature point according to whether the distance from the feature point to the epipolar line exceeds the threshold, and using the above-mentioned sequential container to update and store dynamic object information;

[0018] (8) Use static feature points for subsequent tracking, local mapping, and loop detection threads.

[0019] Preferably, the saliency of the image pixel is represented by the contrast between the pixel and other pixels in the same image:

[0020]

[0021] where c l For example I k The color of the image, n is the total number of colors in the image, f j c j The probability of appearing in image l, D(c l ,c j ) is color c l ,c j The distance between the color spaces Lab;

[0022] For three-dimensional color space, such as RGB color space, the total number of 8-bit data colors is 255×255×255=16581375, which is too much calculation. Therefore, color quantization is performed to reduce the amount of calculation. In the RGB color space, the colors are quantized to 12 colors without notification and some colors with lower frequencies are discarded and replaced with similar colors.

[0023] In order to reduce the noise caused by similar colors being quantized as different values, the saliency value of each color is replaced by the weighted average of the saliency values ​​of similar colors; m = n / 4 nearest colors are selected as the saliency value representing the improved color c:

[0024]

[0025] in, is the sum of the distances between color c and its m closest colors.

[0026] Preferably, the feature points of the dynamic object area of ​​the previous frame are tracked using the optical flow method, and the grayscale invariance assumption is introduced, that is, the grayscale value of the pixel at the same spatial point is fixed in each image; the grayscale of the pixel at the (x, y) position at time t is described as:

[0027] I(x,y,t) (3)

[0028] For the pixel at the (x,y) position at time t, it moves to the (x+dx,y+dy) position at time t+dt. Due to the grayscale invariance assumption, we have:

[0029] I(x+dx,y+dy,t+dt)=I(x,y,t) (4)

[0030] Taylor expansion is performed on the far left, retaining the first-order term, and we get:

[0031]

[0032] Due to the grayscale invariance assumption, the grayscale at the next moment is equal to the previous grayscale:

[0033]

[0034] Divide both sides by dt:

[0035]

[0036] Where dx / dt is the speed of the pixel on the x-axis, and dy / dt is the speed of the pixel on the y-axis, which are denoted as u,v; is the gradient of the image in the x direction at that point, is the gradient in the y direction, denoted by I x ,I y ; The change of image grayscale over time is recorded as I t , then the matrix form is as follows:

[0037]

[0038] For the feature point in the dynamic object area, take the feature point as the center, a w×w pixel window, assuming that the pixels in the window have the same motion, and combine w 2 Equations:

[0039]

[0040] remember:

[0041]

[0042] According to the least squares method, we can get:

[0043]

[0044] Given the speed u and v of the pixels moving between images, calculate the mean displacement change of the feature points in the same moving object area. and The bounding box of the moving object area in the previous frame is displaced by corresponding pixels and regarded as the bounding box of the current frame for subsequent calculation process.

[0045] Preferably, the P 1 and P 2 is a pair of correctly matched feature points outside the potential motion area in two adjacent frames, and is represented by x on the normalized plane. 1 =[u 1 ,v 1 ,1] T and x 2=[u 2 ,v 2 ,1] T represents the coordinates of two pixels, and F represents the basic matrix calculated for the correctly matched feature points; then there is an epipolar line l 2 :

[0046]

[0047] Feature point P 2 To P 1 The corresponding epiline l 2 The distance D is:

[0048]

[0049] For feature points in the potential dynamic area, the epipolar distance D is calculated in combination with the current frame and the previous frame, and the epipolar threshold τ is set; when D>τ, the feature point is considered to be an abnormal feature point, and the subordinate object has movement different from the background; when D<τ, the feature point is considered to be a static point.

[0050] Preferably, all the extracted dynamic feature points are removed, and after all the dynamic feature points are removed, a depth value is assigned to each static point through the acquired depth image, and the disparity of the matching feature points in the right image is calculated according to the depth value.

[0051] Preferably, all static points are transmitted back to the tracking thread, and subsequent local mapping thread and local loop detection thread are performed.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention improves the positioning accuracy and robustness of the visual SLAM algorithm in dynamic scenes through the YOLOv5 algorithm. Unlike the traditional combination of target detection and visual SLAM algorithm, the present invention does not remove all feature points in the detection frame, thereby ensuring the overall stability of the SLAM system. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A schematic diagram of the steps in the embodiment of the present invention;

[0056] Figure 2 It is a schematic diagram of the overall scheme in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0058] Reference Figure 1-2 ,An indoor visual SLAM positioning method based on dynamic scenes, comprising the following steps: (1) deploying the YOLOv5 target detection model and SLAM algorithm generated by pre-training on the COCO128 dataset onto a robot platform;

[0059] (2) The image sequence is acquired through the image sensor carried by the robot, and the target detection of the scene image is performed in real time through the YOLOv5 algorithm. The classified objects in the model are identified and the position boundaries are described by anchor frames;

[0060] The combination of YOLO and visual SLAM usually has three directions: target detection, semantic segmentation and instance segmentation. Target detection aims to identify the target object in the image and draw a bounding box around it, that is, the anchor box. Semantic segmentation is to classify each pixel in the image into a specific category, but does not distinguish between different instances of the same category. Instance segmentation not only identifies the target object and draws a bounding box, but also generates a pixel-level segmentation mask for each target instance. The goal of instance segmentation is to solve the problems of target detection and semantic segmentation at the same time, that is, to detect multiple targets in the image and generate an independent segmentation mask for each target.

[0061] (3) The HC (Histogram-based Contrast) algorithm is used to perform saliency detection on the target anchor box range;

[0062] (4) Adjust the anchor frame according to the boundary of the salient map and reduce the area of ​​the anchor frame so that it covers the salient area more accurately.

[0063] (5) Classify the objects in the current image into high, medium, and low motion probabilities;

[0064] (6) Determine whether a potential dynamic object is detected in the current image;

[0065] Step 61, if not, determine whether there is a dynamic object area in the previous frame; if not, the YOLOv5 target detection algorithm network fails, and the current frame process ends. If the previous frame exists, the previous frame and the current frame are matched by the optical flow method, and the corresponding dynamic object area anchor frame of the previous frame is displaced accordingly and drawn to the current frame, and the sequential container is updated to store the data of the dynamic object feature points matched and tracked;

[0066] Step 62, if present, use a sequential container to store the data of each potential dynamic object;

[0067] (7) Extract feature points from the current frame image;

[0068] Step 71, performing optical flow matching on feature points outside the potential motion area to calculate a basic matrix between two frames;

[0069] Step 72, screening the feature points contained in the potential dynamic area based on the epipolar constraint, judging whether it is an abnormal feature point according to whether the distance from the feature point to the epipolar line exceeds the threshold, and using the above-mentioned sequential container to update and store dynamic object information;

[0070] (8) Use static feature points for subsequent tracking, local mapping, and loop detection threads.

[0071] As a technical optimization solution of the present invention, the saliency of an image pixel is represented by the contrast between the pixel and other pixels in the same image:

[0072]

[0073] where c l For example I k The color of the image, n is the total number of colors in the image, f j c j The probability of appearing in image l, D(c l ,c j ) is color c l ,c j The distance between the color spaces Lab;

[0074] For three-dimensional color space, such as RGB color space, the total number of 8-bit data colors is 255×255×255=16581375, which is too much calculation. Therefore, color quantization is performed to reduce the amount of calculation. In the RGB color space, the colors are quantized to 12 colors without notification and some colors with lower frequencies are discarded and replaced with similar colors.

[0075] In order to reduce the noise caused by similar colors being quantized as different values, the saliency value of each color is replaced by the weighted average of the saliency values ​​of similar colors; m = n / 4 nearest colors are selected as the saliency value representing the improved color c:

[0076]

[0077] in, is the sum of the distances between color c and its m closest colors.

[0078] As a technical optimization solution of the present invention, the feature points of the dynamic object area of ​​the previous frame are tracked using the optical flow method, and the grayscale invariance assumption is introduced, that is, the pixel grayscale value of the same spatial point is fixed in each image; the grayscale of the pixel at the (x, y) position at time t is described as:

[0079] I(x,y,t) (3)

[0080] For the pixel at the (x,y) position at time t, it moves to the (x+dx,y+dy) position at time t+dt. Due to the grayscale invariance assumption, we have:

[0081] I(x+dx,y+dy,t+dt)=I(x,y,t) (4)

[0082] Taylor expansion is performed on the far left, retaining the first-order term, and we get:

[0083]

[0084] Due to the grayscale invariance assumption, the grayscale at the next moment is equal to the previous grayscale:

[0085]

[0086] Divide both sides by dt:

[0087]

[0088] Where dx / dt is the speed of the pixel on the x-axis, and dy / dt is the speed of the pixel on the y-axis, which are denoted as u,v; is the gradient of the image in the x direction at that point, is the gradient in the y direction, denoted by I x ,I y ; The change of image grayscale over time is recorded as I t , then the matrix form is as follows:

[0089]

[0090] For the feature point in the dynamic object area, take the feature point as the center, a w×w pixel window, assuming that the pixels in the window have the same motion, and combine w 2 Equations:

[0091]

[0092] remember:

[0093]

[0094] According to the least squares method, we can get:

[0095]

[0096] Given the speed u and v of the pixels moving between images, calculate the mean displacement change of the feature points in the same moving object area. and The bounding box of the moving object area in the previous frame is displaced by corresponding pixels and regarded as the bounding box of the current frame for subsequent calculation process.

[0097] As a technical optimization solution of the present invention, P 1 and P 2 is a pair of correctly matched feature points outside the potential motion area in two adjacent frames, and is represented by x on the normalized plane. 1 =[u 1 ,v 1 ,1] T and x 2 =[u 2 ,v 2 ,1] T represents the coordinates of two pixels, and F represents the basic matrix calculated for the correctly matched feature points; then there is an epipolar line l 2 :

[0098]

[0099] Feature point P 2 To P 1 The corresponding epiline l 2 The distance D is:

[0100]

[0101] For feature points in the potential dynamic area, the epipolar distance D is calculated in combination with the current frame and the previous frame, and the epipolar threshold τ is set; when D>τ, the feature point is considered to be an abnormal feature point, and the subordinate object has movement different from the background; when D<τ, the feature point is considered to be a static point.

[0102] As a technical optimization solution of the present invention, all extracted dynamic feature points are eliminated. After eliminating all dynamic feature points, a depth value is assigned to each static point through the acquired depth image, and the disparity of the matching feature points in the right image is calculated based on the depth value.

[0103] As a technical optimization solution of the present invention, all static feature points are transmitted back to the tracking thread, and subsequent local mapping thread and local loop detection thread are performed.

[0104] In order to verify the accuracy of this method in indoor dynamic scenes, the TUM RGB-D dataset is used; this dataset contains synchronized RGB images and depth images, and provides ground truth data of the camera position, which is convenient for evaluating the performance of the algorithm. The walking sequence in the TUM dataset is a test sequence commonly used to evaluate visual SLAM. This sequence is a series of images with a resolution of 640×480. The camera moves in a walking manner and is shot in an indoor environment. The scene contains tables, chairs, computers, and researchers. Four high-dynamic scene sequences are selected as image input to run the entire system process.

[0105] Step 1: The camera collects an image sequence and sends it to the dynamic object detection module. The YOLOv5 network model performs target detection on it. The network contains a YOLOv5 target detection model generated by pre-training based on the COCO128 dataset, which can identify more than 80 target objects. The image sequence is input. After recognition, the object category, bounding box position and confidence contained in the image are output. Saliency detection is applied to the bounding box range generated after recognition to further reduce the area of ​​the bounding box to improve the detection accuracy. According to the object category information, the image is divided into potential dynamic areas and static areas, but the YOLOv5 network may fail to detect potential dynamic targets. If the current frame does not detect a potential dynamic area, but there is a dynamic object area in the previous frame, the feature points in the dynamic object area are tracked by the optical flow method, and the displacement change mean dx and dy of the pixel blocks near the above feature points are calculated using the formula, and the bounding box of the moving object area in the previous frame is displaced accordingly as the recognition result of the current frame. Then the dynamic object information is updated in the sequential container and the calculation of step 2 is entered.

[0106] Step 2: Calculate the basic matrix F for the part outside the potential dynamic object area, use the formula to calculate the polar line, use the formula to calculate the distance D from the feature point to the polar line, and enter step 3 for judgment.

[0107] Step 3: Perform an epipolar constraint check on the feature points contained in the potential dynamic object area, and set the threshold τ to 2. When D>τ, the feature point is considered to be an abnormal feature point, and when D<τ, the feature point is considered to be a static point. Count the number of abnormal points in the feature points belonging to the same potential dynamic area. When the number exceeds 50% of the total number of feature points in the area, the area is considered to belong to a dynamic object and the dynamic feature points are removed. Then the remaining static feature points are tracked, locally mapped, and looped back to detect the process to realize the SLAM method based on target detection and geometric constraints in dynamic scenes.

[0108] The performance indicators of visual SLAM systems are often evaluated by absolute trajectory error and relative pose error. The absolute trajectory error aligns the estimated trajectory to the true trajectory through rigid body transformation, and calculates the Euclidean distance between the estimated pose and the true pose at each time point. This indicator is usually used to evaluate the global consistency between the estimated trajectory and the true trajectory. The relative position error defines a fixed time interval Δt and selects multiple time intervals on the trajectory. For each time interval, the pose change within Δt is calculated, and then the translation error and rotation error between each pair of incremental poses are calculated. The relative pose error is used to evaluate the local consistency between the estimated trajectory and the true trajectory.

[0109] The positioning accuracy of this method is compared with that of ORB-SLAM2, a representative of traditional visual SLAM systems. Four high-dynamic scene sequences are selected as image input. Each data set is run 5 times and the average is taken to calculate the corresponding absolute trajectory error indicators, including the mean error, median error and root mean square error.

[0110] Define the percentage η of improvement of this method:

[0111]

[0112] This method ensures the positioning accuracy of the camera in dynamic scenes, and the estimated trajectory maintains a high degree of consistency with the actual trajectory. Compared with traditional visual SLAM solutions, it has significantly improved positioning accuracy and robustness.

[0113] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0114] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An indoor visual SLAM positioning method based on dynamic scenes, characterized in that: The steps include: (1) Deploy the YOLOv5 target detection model and SLAM algorithm pre-trained on the COCO128 dataset onto the robot platform; (2) The image sequence is acquired through the image sensor carried by the robot, and the target detection of the scene image is performed in real time through the YOLOv5 algorithm. The classified objects in the model are identified and the position boundaries are described by anchor frames; (3) Using the HC algorithm to perform saliency detection on the target anchor box range; (4) Adjust the anchor frame according to the boundary of the salient map and reduce the area of ​​the anchor frame so that it covers the salient area more accurately. (5) Classify the objects in the current image into high, medium, and low motion probabilities; (6) Determine whether a potential dynamic object is detected in the current image; Step 61, if not, determine whether there is a dynamic object area in the previous frame; if not, the YOLOv5 target detection algorithm network fails, and the current frame process ends. If the previous frame exists, the previous frame and the current frame are matched by the optical flow method, and the corresponding dynamic object area anchor frame of the previous frame is displaced accordingly and drawn to the current frame, and the sequential container is updated to store the data of the dynamic object feature points matched and tracked; Step 62, if present, use a sequential container to store the data of each potential dynamic object; (7) Extract feature points from the current frame image; Step 71, performing optical flow matching on feature points outside the potential motion area to calculate a basic matrix between two frames; Step 72, screening the feature points contained in the potential dynamic area based on the epipolar constraint, judging whether it is an abnormal feature point according to whether the distance from the feature point to the epipolar line exceeds the threshold, and using the above-mentioned sequential container to update and store dynamic object information; (8) Use static feature points for subsequent tracking, local mapping, and loop detection threads.

2. According to claim 1, a method for indoor visual SLAM positioning based on dynamic scenes is characterized in that: The saliency of an image pixel is expressed using the contrast between that pixel and other pixels in the same image: where c l For example I k The color of the image, n is the total number of colors in the image, f j c j The probability of appearing in image l, D(c l ,c j ) is color c l ,c j The distance between the color spaces Lab; For three-dimensional color space, such as RGB color space, the total number of 8-bit data colors is 255×255×255=16581375, which is too much calculation. Therefore, color quantization is performed to reduce the amount of calculation. In the RGB color space, the colors are quantized to 12 colors without notification and some colors with lower frequencies are discarded and replaced with similar colors. In order to reduce the noise caused by similar colors being quantized as different values, the saliency value of each color is replaced by the weighted average of the saliency values ​​of similar colors; m = n / 4 nearest colors are selected as the saliency value representing the improved color c: in, is the sum of the distances between color c and its m closest colors.

3. The indoor visual SLAM positioning method based on dynamic scenes according to claim 1, characterized in that: The optical flow method is used to track the feature points of the dynamic object area in the previous frame, and the grayscale invariance assumption is introduced, that is, the grayscale value of the pixel at the same spatial point is fixed in each image; the grayscale of the pixel at the (x, y) position at time t is described as: I(x,y,t)(3) For the pixel at the (x,y) position at time t, it moves to the (x+dx,y+dy) position at time t+dt. Due to the grayscale invariance assumption, we have: I(x+dx,y+dy,t+dt)=I(x,y,t)(4) Taylor expansion is performed on the far left, retaining the first-order term, and we get: Due to the grayscale invariance assumption, the grayscale at the next moment is equal to the previous grayscale: Divide both sides by dt: Where dx / dt is the speed of the pixel on the x-axis, and dy / dt is the speed of the pixel on the y-axis, which are denoted as u,v; is the gradient of the image in the x direction at that point, is the gradient in the y direction, denoted by I x ,I y ; The change of image grayscale over time is recorded as I t , then the matrix form is as follows: For the feature point in the dynamic object area, take the feature point as the center, a w×w pixel window, assuming that the pixels in the window have the same motion, and combine w 2 Equations: remember: According to the least squares method, we can get: Given the speed u and v of the pixels moving between images, calculate the mean displacement change of the feature points in the same moving object area. and The bounding box of the moving object area in the previous frame is displaced by corresponding pixels and regarded as the bounding box of the current frame for subsequent calculation process.

4. The indoor visual SLAM positioning method based on dynamic scenes according to claim 1, characterized in that: P1 and P2 are a pair of correctly matched feature points outside the potential motion area in two adjacent frames of images, and are represented on the normalized plane by x1=[u1,v1,1] T and x2=[u2,v2,1] T represents the coordinates of two pixels, and F represents the basic matrix calculated for the correctly matched feature points; then there is the epipolar line l2: The distance D of the epipolar line l2 corresponding to the feature point P2 to P1 is: For feature points in the potential dynamic area, the epipolar distance D is calculated in combination with the current frame and the previous frame, and the epipolar threshold τ is set; when D>τ, the feature point is considered to be an abnormal feature point, and the subordinate object has movement different from the background; when D<τ, the feature point is considered to be a static point.

5. The indoor visual SLAM positioning method based on dynamic scenes according to claim 4 is characterized in that: All the extracted dynamic feature points are removed. After all the dynamic feature points are removed, a depth value is assigned to each static point through the acquired depth image, and the disparity of the matching feature points in the right image is calculated according to the depth value.

6. The indoor visual SLAM positioning method based on dynamic scenes according to claim 4 is characterized in that: All static points are transmitted back to the tracking thread, and the subsequent local mapping thread and local loop detection thread are performed.

Citation Information

Patent Citations

  • Semantic vision SLAM positioning method based on target detection in indoor dynamic scene

    CN114677323A

  • Dynamic scene SLAM positioning method based on target detection algorithm YOLOv4 and geometric constraint

    CN116067374A

  • Dynamic environment SLAM method based on YOLOv8 instance segmentation

    CN117635651A

  • Visual slam method applicable to indoor dynamic environment

    WO2022041596A1