Visual SLAM method based on multi-mask fusion

Abstract

The invention discloses a visual SLAM (Simultaneous Localization and Mapping) method based on multi-mask fusion. The method comprises the following steps: acquiring an RGB (Red, Green and Blue) image and a depth image corresponding to the RGB image to extract feature points and depth information; classifying objects in the RGB image, wherein the objects are C1, C2 and C3 respectively; the method comprises the following steps: firstly, carrying out camera pose preliminary estimation by only using feature points of a C1 region to obtain Tinit, calculating an epipolar constraint error and a depth projection error of the feature points according to the Tinit, fusing the two errors, modeling static feature points belonging to a C1 category, and then judging motion states of other feature points; calculating static weights of the feature points which do not belong to the dynamic points for weighting camera pose estimation so as to obtain a final camera pose estimation result; and calculating according to the dynamic points to obtain a dynamic area mask, fusing the dynamic area mask with the dynamic area mask of the C3 type area to obtain a first dynamic area mask, and reconstructing the static scene. According to the invention, the motion area is accurately identified, and the map only containing static objects is generated.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to a visual SLAM method based on multi-mask fusion. Background Art

[0002] In recent years, due to the continuous progress of sensor technology and computer technology, robotics has been rapidly updated and widely used, covering many fields such as service industry, industry and military. In the field of robotics, Simultaneous Localization and Mapping (SLAM) technology plays a core role. However, most of the current visual SLAM research is based on the static assumption, that is, it is assumed that the robot operating environment is static and the lighting remains constant. In practical applications, the environment often contains dynamic objects, such as moving vehicles and pedestrians, which significantly affect the positioning and mapping accuracy of the visual SLAM system. The presence of dynamic objects may lead to problems such as tracking loss and relocalization failure, and even affect the normal operation of the system. The feature points extracted by the system may be located on the moving object, and its position changes over time, causing the system pose estimation error. Therefore, it is of great significance and practical value to study visual SLAM that can adapt to dynamic scenes. Summary of the invention

[0003] Dynamic objects will cause a significant decrease in the accuracy of visual SLAM positioning and mapping, resulting in problems such as tracking loss and relocalization failure. In order to achieve accurate positioning and mapping of robots in dynamic environments, the present invention proposes a visual SLAM method based on multi-mask fusion, which reduces the impact of dynamic objects in real scenes on the positioning and mapping of the visual SLAM method.

[0004] To achieve the above effects, the technical solution adopted by the present invention is as follows: a visual SLAM method based on multi-mask fusion, comprising the following steps:

[0005] S1. Obtain an RGB image and its corresponding depth image, extract feature points from the RGB image, and obtain depth information from the depth image;

[0006] S2, classify the objects in the RGB image into C1, C2 and C3 respectively; among which C1 represents an absolutely stationary object, C2 represents a potentially moving object, and C3 represents a pre-set moving object;

[0007] S3, use only the feature points in the C1 area to preliminarily estimate the camera pose and obtain T init , according to T init Calculate the epipolar constraint error e of the feature point epi And the depth projection error e depth, the two errors are fused, and the static feature points belonging to the C1 category are modeled, and then the chi-square test is used to determine the motion status of the remaining feature points;

[0008] S4. Calculate the static weight ω of the feature points that are not dynamic points i static , used for weighted camera pose estimation to obtain the final camera pose estimation result T final ;

[0009] S5. Calculate the dynamic area mask M based on the dynamic points g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask Based on the dynamic area detection result, the static scene is reconstructed.

[0010] Furthermore, the ORB method is used to extract feature points in the RGB image and match them with the feature points in the previous key frame for camera pose estimation.

[0011] Furthermore, the objects in the RGB image are classified into C1, C2 and C3, respectively, including:

[0012] The semantic segmentation network is used to preprocess the RGB image, and the objects in the image are classified according to the labels of the semantic segmentation results, and marked as C1, C2 and C3 categories respectively.

[0013] Further, according to T init Calculate the epipolar constraint error e of the feature point epi And the depth projection error e depth , the two errors are fused, and the static feature points belonging to the C1 category are modeled, and then the chi-square test is used to determine the motion state of the remaining feature points, including:

[0014] Assume that the previous key frame and the current frame are and p1 and p2 are the matching points between the two frames, where p1 is the current frame A feature point in , p2 is the previous key frame A feature point in the , the homogeneous coordinates of these two points are p1 = [u1, v1, 1] T and p2 = [u2,v2,1] T , then the polar constraint error of point p1 is:

[0015]

[0016] Among them, u1, v1, u2, v2 represent the coordinates of the feature point in the image coordinate system, X, Y are the vectors of the polar lines of the current frame, F represents the basic matrix, and the superscript T represents the transpose; project the three-dimensional point P corresponding to the feature point p1 to the previous frame Get point p' and get its projection depth value z proj , point p′ in the frame The depth value in is z′; then the depth projection error of point p1 is:

[0017] e depth =z proj -z′

[0018] According to the preliminary estimated pose result T init The average epipolar constraint error and depth projection error of the prior static points are calculated as and And these two errors are independent and identically distributed, then the error distribution that the feature point in a stationary state should obey is:

[0019]

[0020] Among them, Σ is the covariance matrix, which represents the uncertainty of the scale. The calculation formula of the covariance matrix Σ of a feature point is:

[0021]

[0022] Among them, s represents the scale of each layer of the image pyramid extracted by the feature point, p represents the standard deviation of each layer of the image pyramid, and n represents the number of layers of the image pyramid where the feature point is located; the feature point polar constraint error e epi And the depth projection error e depth Combined into an error vector E = [e epi ,e depth ], for static feature points, the error vector composed of its errors should obey the Gaussian distribution, and its error vector E is expressed as:

[0023]

[0024] in, The average error vector value of the feature points belonging to category C1 is represented by calculating the inner product of the error vector E to obtain the feature points p belonging to categories C2 and C3. i Chi-square test value of :

[0025]

[0026] The obtained r i It should follow a chi-square distribution with 2 degrees of freedom:

[0027]

[0028] According to the feature point p i The chi-square test value r i Determine the motion state of the point:

[0029]

[0030] Furthermore, the calculation of the static weights of the feature points that are not dynamic points Used for weighted camera pose estimation to obtain the final camera pose estimation result T final ,include:

[0031] For feature points p that are not dynamic points i , its static weight The calculation formula is:

[0032]

[0033] in Indicates the feature point p that is not a dynamic point i The static weight value, r i Indicates the feature point p that is not a dynamic point i Chi-square test value, Semantic(p i ) represents a feature point p that is not a dynamic point i The label category of ; adding the obtained static weight to the final camera pose estimation is obtained:

[0034]

[0035] Where R, t represent the rotation matrix and translation vector of the camera respectively, m represents the number of feature points that are not dynamic points, and Π(·) represents the camera projection model.

[0036] Furthermore, the dynamic area mask M is obtained by calculating the dynamic points. g ,include:

[0037] Only the dynamic points in the C2 category area are calculated to obtain the dynamic area mask M g .

[0038] Furthermore, the dynamic area mask M is obtained by calculating the dynamic points. g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask include:

[0039] According to the Euclidean distance between dynamic points, the DBSCAN clustering method is used to group them, and the convex hull of each group is obtained to generate the motion area mask M g, and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask

[0040]

[0041] Furthermore, the step S5 further includes:

[0042] First dynamic area mask Divided into T independent masks And calculate each independent mask The average depth value d t :

[0043] According to d t Find the image area whose depth value is within the first preset threshold range

[0044] Will Decompose into K independent regions And calculate and The intersection rate between the two masks; determined by the intersection rate being greater than or equal to a second preset threshold The effective mask in and will and Fusion to obtain the second dynamic area mask H represents the number of effective masks.

[0045] Furthermore, the first dynamic area mask Divided into T independent masks It is expressed as:

[0046]

[0047] The calculation is performed where each independent mask The average depth value d t , the calculation formula is:

[0048]

[0049] Among them, p j Indicated in The jth pixel in the region, D(p j ) represents point p j The depth value of

[0050] According to the t Find the image area whose depth value is within the first preset threshold range The judgment process is:

[0051]

[0052] Wherein, p represents any pixel point in the image, and τ1 represents the first preset threshold;

[0053] Will Decompose into K independent regions It is expressed as:

[0054]

[0055] And calculate and The intersection rate between two masks is given by:

[0056]

[0057] Among them, P(·,·) represents the intersection rate between the two masks, and N(·,·) represents the number of pixels intersecting between the two masks;

[0058] Determine by the intersection rate being greater than or equal to the second preset threshold τ2 The effective mask in q means There are q conditions that meet the conditions area, represents one of the regions that meets the condition, 1≤z≤q;

[0059] The and Fusion to obtain the second dynamic area mask It is expressed as:

[0060]

[0061] Furthermore, the reconstructing the static scene based on the dynamic area detection result includes:

[0062] When building a dense point cloud map, the point cloud in the dynamic area can be removed to obtain a static and reliable point cloud map.

[0063] Compared with existing methods, the beneficial effects of the present invention are mainly reflected in: accurately identifying moving areas and generating maps containing only static objects. These innovations greatly improve the robustness of the system in dynamic scenes, enabling the robot to achieve accurate positioning and mapping functions in dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of a process of a specific embodiment of the present invention;

[0065] Figure 2The figure is a schematic diagram of the multi-mask fusion process in a specific implementation manner of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, design ideas and technical solutions of the embodiments of the present invention clearer, the present invention is further described below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementation methods can be combined with each other.

[0067] Reference Figure 1 As shown, the present invention proposes a visual SLAM method based on multi-mask fusion, and the method comprises the following steps:

[0068] Step S1: Obtain an RGB image and its corresponding depth image, extract feature points from the RGB image, and obtain depth information from the depth image;

[0069] Specifically, the RGB image and depth image of the current environment are obtained through the RGB-D camera, and feature points are extracted from the RGB image. The ORB method (English full name: Oriented Fast and Rotated Brief) is used to extract feature points in the RGB image and match them with the feature points in the previous key frame for subsequent camera pose estimation. Key frames refer to some representative frames selected on the trajectory of the camera. These frames are used to save important information in the environment, such as feature points and poses, for subsequent map construction and positioning.

[0070] Step S2: Classify the objects in the RGB image into C1, C2 and C3, respectively; C1 represents an absolutely stationary object, C2 represents a potentially moving object, and C3 represents a pre-set moving object;

[0071] Specifically, the semantic segmentation network is used to preprocess the RGB image, and the objects in the image are classified according to the labels of the semantic segmentation results, and marked as C1, C2 and C3 categories respectively. In the classification of objects according to labels, C1 represents permanently stationary objects, such as buildings, sky, etc.; C2 represents potentially moving objects, such as vehicles, chairs, balloons, etc.; C3 represents pre-set moving objects, such as people and animals.

[0072] Step S3: Use only the feature points in the C1 region to perform a preliminary estimation of the camera pose and obtain T init , according to T init Calculate the epipolar constraint error e of the feature point epi And the depth projection error e depth , the two errors are fused, and the static feature points belonging to the C1 category are modeled, and then the chi-square test is used to determine the motion status of the remaining feature points.

[0073] Specifically, we first use the feature points belonging to the C1 category to perform initial pose estimation and obtain T init . And according to T init Calculate the epipolar constraint error e of the feature point epi and the depth projection error e depth . Assume that the previous key frame and the current frame are and p1 and p2 are the matching points between the two frames, where p1 is the current frame A feature point in the previous frame, p2 is A feature point in the , the homogeneous coordinates of these two points are p1 = [u1, v1, 1] T and p2 = [u2,v2,1] T , where u1, v1, u2, v2 represent the coordinates of the feature point in the image coordinate system. The epipolar constraint error of point p1 is:

[0074]

[0075] Among them, X, Y are the vectors of the polar lines of the current frame, F represents the basic matrix, and the superscript T represents the transpose. Project the three-dimensional point P corresponding to the feature point p1 to the previous frame Get point p' and get its projection depth value z proj , point p′ in the frame The depth value in is z′. Then the depth projection error of point p1 is:

[0076] e depth =z proj -z′(2)

[0077] According to the semantic classification results, a set of prior static feature points (feature points in the C1 category) is obtained, and according to the preliminary estimated pose result T init , the average epipolar constraint error and depth projection error of the prior static point can be calculated as and Assuming that the two errors are independent and identically distributed, the error distribution that the feature points in a stationary state should obey is:

[0078]

[0079] Among them, Σ is the covariance matrix, which represents the uncertainty of the scale. The calculation formula of the covariance matrix Σ of a feature point is:

[0080]

[0081] Among them, s represents the scale of each layer of the image pyramid extracted by the feature point, p represents the standard deviation of each layer of the image pyramid, and n represents the number of layers of the image pyramid where the feature point is located. epi And the depth projection error e depth Combined into an error vector E = [e epi ,e depth ], for stationary feature points, the error vector composed of its errors should obey the Gaussian distribution, and its error vector E can be expressed as:

[0082]

[0083] in, The average error vector value of the feature points belonging to category C1 is represented by calculating the inner product of the error vector E to obtain the feature points p belonging to categories C2 and C3. i Chi-square test value of :

[0084]

[0085] The obtained r i It should follow a chi-square distribution with 2 degrees of freedom:

[0086]

[0087] According to the feature point p i The chi-square test value r i Determine the motion state of the point:

[0088]

[0089] The motion state of the feature points will be used for camera pose estimation in the next step.

[0090] Step S4: Calculate the static weights of feature points that are not dynamic points Used for weighted camera pose estimation to obtain the final camera pose estimation result T final ;

[0091] In particular, semantics and geometric constraints are combined to calculate the weights of feature points. For feature points p that are not dynamic points, i , its static weight The calculation formula is:

[0092]

[0093] in Indicates that it does not belong to the dynamic point p i The static weight value, r i Indicates that it does not belong to the dynamic point p i Chi-square test value, Semantic(pi ) indicates that it does not belong to the dynamic point p i The label category obtained in semantic segmentation. Add the obtained static weight to the final camera pose estimation to obtain:

[0094]

[0095] Where R, t represent the rotation matrix and translation vector of the camera respectively, m represents the number of feature points that are not dynamic points, Π(·) represents the camera projection model, and i represents the i-th map point

[0096] Step S5: Calculate the dynamic area mask M based on the dynamic points g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask Based on the dynamic area detection result, the static scene is reconstructed.

[0097] Preferably, only the dynamic points in the C2 category area are calculated to obtain the dynamic area mask M g .

[0098] In one embodiment, if Figure 2 As shown, the dynamic area mask M is calculated based on the dynamic points g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask include:

[0099] According to the Euclidean distance between dynamic points, the DBSCAN clustering method is used to group them, and the convex hull of each group is obtained, and then the motion area mask M obtained by geometric information is generated. g , and compare it with the mask M of the C3 category area obtained by semantic segmentation s Fusion is performed to obtain the first dynamic area mask for:

[0100]

[0101] In another embodiment, if Figure 2 As shown, it includes two stages. In the first stage, the first dynamic area mask is obtained from the above embodiment. Phase II includes:

[0102] According to the connectivity between pixels, the first dynamic region mask is divided into T independent masks, so as to calculate the average depth value of each different motion region.

[0103]

[0104] in, express The tth independent mask in , T means there are T masks in total. First, calculate the average depth value of each independent mask in the depth image, assuming that There are J pixels in The average depth value d of the area t for:

[0105]

[0106] Among them, d t express The average depth value of the region, p j Indicated in The jth pixel in the region, D(p j ) represents point p j The depth value of d t Find the image area with a depth value close to it (that is, within the first preset threshold range), and the judgment process is as follows:

[0107]

[0108] Wherein, p represents any pixel in the image; τ1 represents a first preset threshold, also called a depth threshold (in one embodiment, its value is 0.1 m, but it is not limited to this), Indicates that in the depth image Areas with similar depth values. Assume Can be decomposed into K independent regions, and the intersection rate between the two masks is calculated:

[0109]

[0110] in, express The kth independent region in the image, P(·,·) represents the intersection rate between the two masks, and N(·,·) represents the number of pixels that intersect between the two masks. A second preset threshold τ2 is set (in one embodiment, its value is 0.2, but it is not limited to this), and the depth mask obtained based on the intersection rate is judged Assume that There are q conditions that meet the conditions The second dynamic area mask It can be expressed as:

[0111]

[0112]

[0113] in, Represents a valid mask, there are H valid masks in total; H≤T, T is the first dynamic area mask Separate masks Formula (19) indicates that the mask obtained in the first stage is fused with the mask obtained in the second stage to obtain the final mask result

[0114] This gives the first dynamic area mask and the second motion region mask Preferred second motion region mask When establishing a dense point cloud map, the point cloud in the moving area can be removed to obtain a static and reliable point cloud map.

[0115] In the present invention: the present invention adopts a visual SLAM method based on multi-mask fusion, aiming to improve the robustness and accuracy of positioning and mapping of the visual SLAM system in dynamic scenes. The present invention uses a semantic segmentation method to extract a priori static information in the scene, and models the feature points of the static area by combining multiple geometric errors. Thus, a chi-square test is adopted to realize adaptive detection of dynamic feature points under dynamic environments of different degrees. In order to reduce the influence of system noise on pose estimation, a static weight calculation method of feature points is designed, and then a pose optimization method based on static weights is constructed. In addition, in order to realize accurate recognition of dynamic areas, the method fuses the dynamic point detection results with semantic information to preliminarily identify moving objects, and improves the dynamic area mask in combination with depth information, thereby generating a map containing only static objects in mapping. Finally, the present invention enables the visual SLAM method to realize accurate positioning and mapping in unknown dynamic environments. The present invention also combines two geometric errors to model the feature points, thereby realizing accurate estimation of the motion state of the feature points, and adopts a dynamic area detection algorithm based on multi-mask fusion to accurately identify the moving area and generate a map containing only static objects. These innovations greatly improve the robustness of the system in dynamic scenes, enabling the robot to achieve accurate positioning and mapping in dynamic scenes. Therefore, the method of the present invention has important practical application value and can be widely used in the fields of robot navigation, autonomous driving, etc., providing strong support for technological innovation and progress in related industries.

[0116] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A visual SLAM method based on multi-mask fusion, characterized in that: The following steps are involved: S1. Obtain an RGB image and its corresponding depth image, extract feature points from the RGB image, and obtain depth information from the depth image; S2, classify the objects in the RGB image into C1, C2 and C3 respectively; among which C1 represents an absolutely stationary object, C2 represents a potentially moving object, and C3 represents a pre-set moving object; S3, use only the feature points in the C1 area to preliminarily estimate the camera pose and obtain T init , according to T init Calculate the epipolar constraint error e of the feature point epi And the depth projection error e depth , the two errors are fused, and the static feature points belonging to the C1 category are modeled, and then the chi-square test is used to determine the motion status of the remaining feature points; S4. Calculate the static weights of feature points that are not dynamic points Used for weighted camera pose estimation to obtain the final camera pose estimation result T final ; S5. Calculate the dynamic area mask M based on the dynamic points g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask Based on the dynamic area detection result, the static scene is reconstructed.

2. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The ORB method is used to extract feature points in the RGB image and match them with the feature points in the previous key frame for camera pose estimation.

3. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The objects in the RGB image are classified into C1, C2 and C3, respectively, including: The semantic segmentation network is used to preprocess the RGB image, and the objects in the image are classified according to the labels of the semantic segmentation results, and marked as C1, C2 and C3 categories respectively.

4. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: According to T init Calculate the epipolar constraint error e of the feature point epi And the depth projection error e depth , the two errors are fused, and the static feature points belonging to the C1 category are modeled, and then the chi-square test is used to determine the motion state of the remaining feature points, including: Assume that the previous key frame and the current frame are and p1 and p2 are the matching points between the two frames, where p1 is the current frame A feature point in , p2 is the previous key frame A feature point in the , the homogeneous coordinates of these two points are p1 = [u1, v1, 1] T and p2 = [u2,v2,1] T , then the polar constraint error of point p1 is: Among them, u1, v1, u2, v2 represent the coordinates of the feature point in the image coordinate system, X, Y are the vectors of the polar lines of the current frame, F represents the basic matrix, and the superscript T represents the transpose; project the three-dimensional point P corresponding to the feature point p1 to the previous frame Get point p' and get its projection depth value z proj , point p′ in the frame The depth value in is z′; then the depth projection error of point p1 is: yes depth =|with proj -z′| According to the preliminary estimated pose result T init The average epipolar constraint error and depth projection error of the prior static points are calculated as and And these two errors are independent and identically distributed, then the error distribution that the feature point in a stationary state should obey is: Among them, Σ is the covariance matrix, which represents the uncertainty of the scale. The calculation formula of the covariance matrix Σ of a feature point is: Among them, s represents the scale of each layer of the image pyramid extracted by the feature point, p represents the standard deviation of each layer of the image pyramid, and n represents the number of layers of the image pyramid where the feature point is located; the feature point polar constraint error e epi And the depth projection error e depth Combined into an error vector E = [e epi ,e depth ], for static feature points, the error vector composed of its errors should obey the Gaussian distribution, and its error vector E is expressed as: in, The average error vector value of the feature points belonging to category C1 is represented by calculating the inner product of the error vector E to obtain the feature points p belonging to categories C2 and C3. i Chi-square test value of : The obtained r i It should follow a chi-square distribution with 2 degrees of freedom: According to the feature point p i The chi-square test value r i Determine the motion state of the point:

5. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The calculation of the static weight of the feature point that is not a dynamic point Used for weighted camera pose estimation to obtain the final camera pose estimation result T final ,include: For feature points p that are not dynamic points i , its static weight The calculation formula is: in Indicates the feature point p that is not a dynamic point i The static weight value, r i Indicates the feature point p that is not a dynamic point i Chi-square test value, Semantic(p i ) represents a feature point p that is not a dynamic point i The label category of ; adding the obtained static weight to the final camera pose estimation is obtained: Where R, t represent the rotation matrix and translation vector of the camera respectively, m represents the number of feature points that are not dynamic points, and Π(·) represents the camera projection model.

6. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The dynamic area mask M is obtained by calculating the dynamic points g ,include: Only the dynamic points in the C2 category area are calculated to obtain the dynamic area mask M g .

7. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The dynamic area mask M is obtained by calculating the dynamic points g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask include: According to the Euclidean distance between dynamic points, the DBSCAN clustering method is used to group them, and the convex hull of each group is obtained to generate the motion area mask M g , and the dynamic region mask M of the C3 category region s Fusion is performed to obtain the first dynamic area mask 8. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The step S5 further comprises: First dynamic area mask Divided into T independent masks And calculate each independent mask The average depth value d t : According to d t Find the image area whose depth value is within the first preset threshold range Will Decompose into K independent regions And calculate and The intersection rate between the two masks; determined by the intersection rate being greater than or equal to a second preset threshold The effective mask in and will and Fusion to obtain the second dynamic area mask H represents the number of effective masks.

9. A visual SLAM method based on multi-mask fusion according to claim 8, characterized in that: The first dynamic area mask Divided into T independent masks It is expressed as: The calculation is performed where each independent mask The average depth value d t , the calculation formula is: Among them, p j Indicated in The jth pixel in the region, D(p j ) represents point p j The depth value of According to the t Find the image area whose depth value is within the first preset threshold range The judgment process is: Wherein, p represents any pixel point in the image, and τ1 represents the first preset threshold; Will Decompose into K independent regions It is expressed as: And calculate and The intersection rate between two masks is given by: Among them, P(·,·) represents the intersection rate between the two masks, and N(·,·) represents the number of pixels intersecting between the two masks; Determine by the intersection rate being greater than or equal to the second preset threshold τ2 The effective mask in q means There are q conditions that meet the conditions area, represents one of the regions that meets the condition, 1≤z≤q; The and Fusion to obtain the second dynamic area mask It is expressed as:

10. A visual SLAM method based on multi-mask fusion according to claim 1, characterized in that: The step of reconstructing the static scene based on the dynamic area detection result includes: When building a dense point cloud map, the point cloud in the dynamic area can be removed to obtain a static and reliable point cloud map.

Images