A dynamic visual slam method and apparatus

Abstract

The application discloses a dynamic visual SLAM method and device, belongs to the technical field of visual simultaneous localization and mapping, and comprises the following steps: obtaining multi-source input data at the current moment through an improved dynamic visual SLAM system; wherein the multi-source input data comprises an RGB image and a depth image, the improved dynamic visual SLAM system is obtained by additionally arranging a semantic module and a feature-level Bayesian dynamic estimation module in the SLAM system, and the improved dynamic visual SLAM system comprises a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module and a back-end optimization module. The log-likelihood ratio calculation of the semantic prior evidence, the LK optical flow consistency evidence, the epipolar geometry constraint evidence and the depth residual error evidence is completed through the improved dynamic visual SLAM system, then the static posterior probability of the feature points is obtained based on the fused log-likelihood ratio, the front-end observation soft suppression and the back-end local optimization are carried out by adopting the static posterior probability, and the positioning accuracy of the visual SLAM system in a dynamic environment is improved.

Description

Technical Field

[0001] This invention relates to the field of visual simultaneous localization and mapping (SLAM) technology, and in particular to a dynamic visual SLAM method and apparatus. Background Technology

[0002] Visual simultaneous localization and mapping (VSLM) technology acquires continuous image sequences through a camera, estimates the camera's trajectory using image feature extraction, feature matching, geometric constraints, and nonlinear optimization methods, and simultaneously constructs an environmental map. Feature-point SLAM systems, represented by ORB-SLAM, ORB-SLAM2, and ORB-SLAM3, exhibit high localization accuracy and good real-time performance in static environments. Traditional visual SLAM systems are typically based on the assumption of a static scene, meaning that the feature points involved in matching, pose estimation, and map optimization are assumed to originate from a static background structure.

[0003] However, in real-world dynamic scenarios, pedestrian movement, vehicle movement, object relocation, and occlusion changes generate a large number of dynamic features that do not satisfy the static rigid body assumption. If these dynamic features directly participate in front-end feature matching, current frame pose estimation, local mapping, and back-end local BA optimization, it can easily lead to mis-associations of features, degradation of pose estimation, instability in local optimization, and residual dynamic map points, thereby reducing the localization accuracy and mapping reliability of the visual SLAM system in dynamic environments.

[0004] Existing dynamic visual SLAM methods mainly include dynamic feature detection methods based on geometric consistency and dynamic region recognition methods based on semantic information. Geometric methods typically utilize optical flow, epipolar constraints, reprojection errors, or motion consistency to detect dynamic features. These methods do not rely on target category information and have a certain degree of versatility. However, when camera motion is complex, background texture is sparse, or occlusion is severe, they are prone to confusing the geometric residuals caused by camera motion with the residuals caused by independent object motion. Semantic methods typically use semantic segmentation, instance segmentation, or object detection networks to identify potential dynamic targets and remove or downweight feature points within the target region. These methods can use category priors to compensate for the insufficient perception of object category information by pure geometric methods. However, their segmentation results suffer from category misclassification, mask boundary deviation, and confidence uncertainty.

[0005] If all feature points within the semantic mask are directly binary-coded as dynamic and removed, static background features within or near the boundaries of the dynamic target mask may be mistakenly deleted, resulting in insufficient usable geometric constraints. On the other hand, potential movable objects are not necessarily in motion; hard removal based solely on category priors can also lead to the loss of static structural information. Furthermore, existing methods often lack a unified dynamic suppression metric between front-end dynamic feature selection and back-end local BA (Bundle Adjustment) optimization, meaning that observations deemed low-confidence at the front end can still have a significant impact on back-end optimization. Summary of the Invention

[0006] The purpose of this invention is to provide a dynamic visual SLAM method and apparatus. It introduces a semantic module to acquire the category, mask, and confidence information of potential dynamic targets, and a feature-level Bayesian dynamic estimation module to unify multi-source evidence such as semantic prior, LK optical flow consistency, epipolar geometric constraints, and depth residuals into a log-likelihood ratio. Then, it obtains the static posterior probability of feature points through log-odds recursion, and uses this static posterior probability for front-end tracking soft suppression and back-end local BA (Bundle Adjustment) weighted optimization, thereby improving the localization accuracy in dynamic environments. This invention is achieved through the following technical solutions.

[0007] In a first aspect, the present invention provides a dynamic visual SLAM method, comprising the following steps:

[0008] The improved dynamic visual SLAM system acquires multi-source input data at the current moment. The multi-source input data includes RGB images. The improved dynamic visual SLAM system obtains this data by adding a semantic module and a feature-level Bayesian dynamic estimation module to the SLAM system. The improved dynamic visual SLAM system includes a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module, and a back-end optimization module.

[0009] The front-end tracking module is used to extract features from the RGB image at the current moment to obtain the feature points of the current frame; based on the multi-source input data between adjacent RGB images, it obtains the LK optical flow consistency information, epipolar geometric consistency information, and depth residual information of each feature point, and calculates the log-likelihood ratio of the multi-source observation evidence based on the above information; the multi-source observation evidence includes semantic prior evidence, LK optical flow consistency evidence, epipolar geometric constraint evidence, and depth residual evidence.

[0010] The semantic module is used to perform instance segmentation on the RGB image at the current time, obtain the semantic prior information corresponding to each feature point, and calculate the log-likelihood ratio of the semantic prior data based on the semantic prior information.

[0011] The feature-level Bayesian dynamic estimation module is used to fuse the log-likelihood ratio of multi-source observation evidence, calculate the static posterior probability of each feature point, and input the static posterior probability into the local mapping module and the front-end tracking module.

[0012] The front-end tracking module is also used to receive static posterior probabilities to determine keyframes in RGB images and input the keyframes into the local mapping module;

[0013] The local mapping module is used to receive static posterior probabilities to generate map points, insert keyframes, and perform local map maintenance.

[0014] The backend optimization module maps static posterior probabilities to observation weights and works in conjunction with a robust kernel function in the local bundle adjustment optimization process to output the camera motion trajectory.

[0015] In practical applications, traditional visual SLAM systems, or SLAM systems improved upon them, are typically based on a static scene assumption, meaning that the feature points involved in matching, pose estimation, and map optimization are assumed to originate from a static background structure. Therefore, if dynamic feature points exist, these dynamic feature points do not satisfy the static rigid body assumption, and direct use of them can lead to significant errors in localization and mapping. This invention calculates the static posterior probability of each feature point using the log-likelihood ratio of fused multi-source observation data. This static posterior probability is then used for front-end observation soft suppression, overcoming the shortcomings of existing methods that directly remove all features within potential dynamic regions. This is because static feature points may exist even within dynamic regions. Simultaneously, the static posterior probability is used for back-end local optimization, using observation weights and a robust kernel function to jointly suppress dynamic outliers. This ensures that the front-end dynamic discrimination results are consistent with the back-end optimization weights, thereby reducing the interference of dynamic objects on pose estimation and local map construction, further improving the system's localization and mapping accuracy.

[0016] Optionally, the semantic module is used to perform instance segmentation on the RGB image at the current time step and obtain the semantic prior information corresponding to each feature point, including: the semantic module is used to perform instance segmentation on the depth image at the previous time step, output the instance mask, class label, class probability and confidence of the potential dynamic target online, and associate the feature points of the current frame with the instance mask at the pixel level to obtain the semantic prior information corresponding to each feature point.

[0017] Optionally, the multi-source input data may also include depth images, camera intrinsics, the camera pose of the previous frame, historical keyframes, local map points, and feature matching information of adjacent frames.

[0018] The step of obtaining LK optical flow consistency information, epipolar geometric consistency information, and depth residual information for each feature point based on multi-source input data between adjacent RGB images includes: obtaining LK optical flow consistency information, epipolar geometric consistency information, and depth residual information for each feature point based on feature matching information between adjacent RGB images, camera intrinsic parameters, initial camera pose, depth image, and local map points; wherein, camera intrinsic parameters are used to calculate the geometric relationship between image coordinates and projection model, the previous frame camera pose, historical keyframes, and local map points are used to establish the geometric association between the current frame image and the existing map, adjacent frame feature matching information is used for optical flow consistency and epipolar geometric consistency analysis, and the initial camera pose is calculated based on adjacent frame feature matching information, the previous frame camera pose, and local map points.

[0019] In this invention, the depth map is also used to construct depth residual evidence.

[0020] Optionally, the log-likelihood ratio of the semantically prior evidence is calculated using the following formula:

[0021] ,

[0022] ,

[0023] In the formula, Indicates the first The first frame of the RGB image The semantic meaning of each feature point is first verified by the log-likelihood ratio of the data. Indicates the first Frame RGB image The dynamic probability prior of semantic induction for feature points, where the superscript D denotes the dynamic state and the superscript sem denotes semantic evidence. This is to prevent logarithmic divergence. Indicates the first The first frame of the RGB image The mask boundary reliability for each feature point is determined based on the distance from the pixel coordinates of the feature point to the mask boundary of the associated instance. This represents the set of target categories for the instance segmentation model. This represents the category index in the target category set of the instance segmentation model. For category The presupposed a priori mobility, Indicates the relationship with the first Frame number The instance number associated with each feature point. For the first On the RGB image of the first frame Instances of association between feature points Category The class probability, This is a preset probability cutoff constant.

[0024] In this invention, categories This refers to the target category set of the instance segmentation model. One of the target categories, such as pedestrians, cars, and other potential movable target categories.

[0025] Optionally, the log-likelihood ratio of the LK optical flow consistency evidence is calculated using the following formula:

[0026] ,

[0027] In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of LK optical flow consistency evidence corresponding to each feature point, where N is the residual dimension. For static residuals, For dynamic residuals, Indicates the first The first frame of the RGB image The residual energy of each feature point.

[0028] Optionally, the log-likelihood ratio of the epipolar geometric consistency evidence is calculated using the following formula:

[0029] ,

[0030] In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of the epipolar geometric consistency evidence corresponding to each feature point. Represents the static epipolar residual scale. Indicates the dynamic epipolar residual scale. Indicates the first The first frame of the RGB image Normalized epipolar residuals of each feature point.

[0031] In this invention, the normalized epipolar residual is calculated from the homogeneous pixel coordinates of the corresponding feature points in frame t-1 and frame t and the fundamental matrix between the two frames, and is used to measure the degree of deviation of the matched feature point from the epipolar geometric constraint.

[0032] Optionally, the log-likelihood ratio of the deep residual evidence is calculated using the following formula:

[0033] ,

[0034] ,

[0035] In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of the depth residual evidence corresponding to each feature point This represents the Gaussian probability density function. Indicates the first The first frame of the RGB image The depth residual corresponding to each feature point Represents the static depth residual scale. Indicates the dynamic depth residual scale. Indicates the first The first frame of the RGB image Depth measurement values ​​of each feature point Indicates the first The first frame of the RGB image The depth prediction value of each feature point.

[0036] Optionally, the static posterior probability is calculated using the following formula:

[0037] ,

[0038] ,

[0039] ,

[0040] In the formula, The representation is defined as follows: This represents the set of types of multi-source observational evidence. Represents the elements in a set, defined For the first The first frame of the RGB image The static posterior probability of a feature point Static posterior probability The log-dominance ratio is expressed as follows: Forgetting factor, For calibration coefficients, This represents the Sigmoid function. This represents the first digit recovered by the inverse Sigmoid transform. The first frame of the RGB image The static posterior probability of a feature point Represents a graph with the natural constant e as the base and an exponent of e. The exponential function term is used to calculate the static posterior probability using the Sigmoid function.

[0041] Optionally, the observation weights are calculated using the following formula:

[0042] ,

[0043] In the formula, Indicates the keyframe number. Indicates the map point number. This represents a monotonic mapping function from static posterior probability to observed weights. Keyframe With map points The static posterior probability corresponding to the formed observation edge, which is determined by the keyframe. Central observation map points The feature points are obtained by static posterior probability mapping. Indicates the observation edge Backend observation weights, Indicates the minimum observation weight. This parameter represents the suppression strength during the mapping process from static posterior probability to backend observation weights, and is used to adjust the degree of weight reduction for observations with low static posterior probability.

[0044] In a second aspect, the present invention provides a dynamic visual SLAM device, comprising the following:

[0045] The multi-source input data acquisition module is configured to acquire multi-source input data at the current moment through an improved dynamic visual SLAM system. The multi-source input data includes RGB images. The improved dynamic visual SLAM system is obtained by adding a semantic module and a feature-level Bayesian dynamic estimation module to the SLAM system. The improved dynamic visual SLAM system includes a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module, and a back-end optimization module.

[0046] The front-end tracking module is used to extract features from the RGB image at the current moment to obtain the feature points of the current frame; based on the multi-source input data between adjacent RGB images, it obtains the LK optical flow consistency information, epipolar geometric consistency information, and depth residual information of each feature point, and calculates the log-likelihood ratio of the multi-source observation evidence based on the above information; the multi-source observation evidence includes semantic prior evidence, LK optical flow consistency evidence, epipolar geometric constraint evidence, and depth residual evidence.

[0047] The semantic module is used to perform instance segmentation on the RGB image at the current time, obtain the semantic prior information corresponding to each feature point, and calculate the log-likelihood ratio of the semantic prior data based on the semantic prior information.

[0048] The feature-level Bayesian dynamic estimation module is used to fuse the log-likelihood ratio of multi-source observation evidence, calculate the static posterior probability of each feature point, and input the static posterior probability into the local mapping module and the front-end tracking module.

[0049] The front-end tracking module is also used to receive static posterior probabilities to determine keyframes in RGB images and input the keyframes into the local mapping module;

[0050] The local mapping module is used to receive static posterior probabilities to generate map points, insert keyframes, and perform local map maintenance.

[0051] The backend optimization module maps static posterior probabilities to observation weights and works in conjunction with a robust kernel function in the local bundle adjustment optimization process to output the camera motion trajectory.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] The dynamic visual SLAM method introduced in this invention performs instance segmentation on RGB images by introducing a semantic module. The category, mask, and confidence score output by instance segmentation are used as semantic priors and converted into semantic log-likelihood ratios. This prevents semantic information from being used for feature selection through simple binary elimination, thereby reducing the risk of erroneous deletion of effective static features due to semantic mask errors. Through the introduction of a feature-level Bayesian dynamic estimation module, multi-source observation evidence is converted into log-likelihood ratios to obtain the static posterior probability of feature points. This unifies the conversion of heterogeneous multi-source observations into log-likelihood ratio form, allowing various types of evidence to be used as fusion inputs for the same feature-level Bayesian dynamic estimation module and recursively fused in the log-odds space. This enables observation information from different sources and with different dimensions to participate in feature dynamic state estimation under a unified Bayesian framework. Through the above conversion, the differences in data source, physical dimension, and numerical range of various types of observation evidence can be eliminated, allowing different evidence to be weighted and fused within a unified logarithmic domain calculation framework. At the same time, it avoids numerical underflow caused by the continuous multiplication of multiple small conditional likelihoods, improving the numerical stability of the multi-source evidence fusion process.

[0054] This invention introduces a forgetting factor to attenuate the influence of historical observations, enabling the static posterior probability of feature points to be updated as the current observation changes. This avoids the long-term fixation of historical judgments and improves the adaptability of the improved dynamic visual SLAM system to changes in the motion state of dynamic targets, short-term occlusion, and semantic misjudgment.

[0055] This invention employs static posterior probability for front-end observation soft suppression, rather than directly eliminating all features within potential dynamic regions. This approach preserves certain effective constraints in scenarios with high dynamic occlusion ratios or relatively scarce static features, reducing the risk of tracking failure. Furthermore, the static posterior probability of feature points obtained from the front end is passed to the back-end local BA. Dynamic outliers are suppressed jointly by observation weights and robust kernel functions, ensuring consistency between the front-end dynamic discrimination results and the back-end optimization weights. This reduces optimization degradation caused by inconsistencies between front-end and back-end processing, thereby minimizing the interference of dynamic objects on pose estimation and local map construction. This improves the localization accuracy, local mapping stability, and map reliability of the visual SLAM system in dynamic environments. Attached Figure Description

[0056] Figure 1 The diagram shown is a schematic flowchart of a dynamic visual SLAM method in one embodiment of the present invention.

[0057] Figure 2 The diagram shown is a schematic diagram of feature point detection in the ORB-SLAM3 baseline system in one embodiment of the present invention.

[0058] Figure 3 The diagram shown is a schematic diagram of feature point detection in a dynamic visual FBDE-SLAM system according to an embodiment of the present invention.

[0059] Figure 4 The diagram shown is a schematic diagram of feature point detection of the ORB-SLAM3 baseline system in a real environment in one embodiment of the present invention.

[0060] Figure 5 The figure shown is a schematic diagram of feature point detection of the FBDE-SLAM system in a real environment in one embodiment of the present invention;

[0061] Figure 6 The diagram shown is a comparison of camera pose trajectories under a real dataset in one embodiment of the present invention. Detailed Implementation

[0062] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details. In this description, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0063] Example 1

[0064] This embodiment provides a dynamic visual SLAM method, such as Figure 1 As shown, it includes the following steps:

[0065] The improved dynamic visual SLAM system acquires multi-source input data at the current moment. The multi-source input data includes RGB images. The improved dynamic visual SLAM system obtains this data by adding a semantic module and a feature-level Bayesian dynamic estimation module to the SLAM system. The improved dynamic visual SLAM system includes a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module, and a back-end optimization module.

[0066] The front-end tracking module is used to extract features from the RGB image at the current moment to obtain the feature points of the current frame; based on the multi-source input data between adjacent RGB images, it obtains the LK optical flow consistency information, epipolar geometric consistency information, and depth residual information of each feature point, and calculates the log-likelihood ratio of the multi-source observation evidence based on the above information; the multi-source observation evidence includes semantic prior evidence, LK optical flow consistency evidence, epipolar geometric constraint evidence, and depth residual evidence.

[0067] The semantic module is used to perform instance segmentation on the RGB image at the current time, obtain the semantic prior information corresponding to each feature point, and calculate the log-likelihood ratio of the semantic prior data based on the semantic prior information.

[0068] The feature-level Bayesian dynamic estimation module is used to fuse the log-likelihood ratio of multi-source observation evidence, calculate the static posterior probability of each feature point, and input the static posterior probability into the local mapping module and the front-end tracking module.

[0069] The front-end tracking module is also used to receive static posterior probabilities to determine keyframes in RGB images and input the keyframes into the local mapping module;

[0070] The local mapping module is used to receive static posterior probabilities to generate map points, insert keyframes, and perform local map maintenance.

[0071] The backend optimization module maps static posterior probabilities to observation weights and works in conjunction with a robust kernel function in the local bundle adjustment optimization process to output the camera motion trajectory.

[0072] The semantic module is used to perform instance segmentation on the RGB image at the current time step and obtain the semantic prior information corresponding to each feature point. The semantic module is used to perform instance segmentation on the depth image at the previous time step and output the instance mask, class label, class probability and confidence of the potential dynamic target online. The current frame feature points are associated with the instance mask at the pixel level to obtain the semantic prior information corresponding to each feature point.

[0073] Example 2

[0074] Based on Example 1, this example describes a specific implementation process of a dynamic visual SLAM method, which includes the following:

[0075] I. Constructing an improved dynamic visual SLAM system

[0076] In one specific embodiment of the present invention, an improved dynamic visual SLAM system is obtained by adding a semantic module and a feature-level Bayesian dynamic estimation module to the existing ORB-SLAM3. In other embodiments, it can also be added based on other basic systems, as long as the selected basic visual SLAM system can provide several necessary functions and data interfaces.

[0077] II. Acquiring multi-source input data

[0078] In one specific embodiment of the present invention, multi-source input data at the current moment is acquired through an improved dynamic visual SLAM system. The multi-source input data also includes depth images, camera intrinsic parameters, the camera pose of the previous frame, historical keyframes, local map points, and feature matching information from adjacent frames.

[0079] The step of obtaining LK optical flow consistency information, epipolar geometric consistency information, and depth residual information for each feature point based on multi-source input data between adjacent RGB images includes: obtaining LK optical flow consistency information, epipolar geometric consistency information, and depth residual information for each feature point based on feature matching information between adjacent RGB images, camera intrinsic parameters, initial camera pose, depth image, and local map points; wherein, camera intrinsic parameters are used to calculate the geometric relationship between image coordinates and projection model, the previous frame camera pose, historical keyframes, and local map points are used to establish the geometric association between the current frame image and the existing map, adjacent frame feature matching information is used for optical flow consistency and epipolar geometric consistency analysis, and the initial camera pose is calculated based on adjacent frame feature matching information, the previous frame camera pose, and local map points.

[0080] During the current frame tracking process, the improved dynamic visual SLAM system first extracts features from the RGB image to obtain the feature points and descriptors of the current frame, and establishes the initial association between the current frame and the previous frame, and between the current frame and local map points, based on descriptor matching, local map projection matching, or optical flow tracing.

[0081] For RGB image input, the improved dynamic visual SLAM system simultaneously acquires depth measurements corresponding to feature points; when depth observations are missing or unavailable, depth residual evidence is not introduced in the subsequent dynamic feature estimation process.

[0082] Through the above data acquisition process, this invention obtains current frame image observations, depth observations, historical poses, local map points, and initial matching relationships, providing an input basis for subsequent instance segmentation semantic prior extraction and feature-level Bayesian dynamic estimation.

[0083] III. Calculating the static posterior probability

[0084] In one specific embodiment of the present invention, to address the high out-of-point rate problem caused by foreground object movement and occlusion in dynamic environments, a latent state variable is introduced. Multi-source information is fused across frames to continuously update the static posterior probability of each feature point. First, a latent state and static posterior probability are defined for each feature point. The latent state is defined as:

[0085] (1)

[0086] In the formula, Indicates the frame number of the RGB image. Indicates the index of the feature point. Indicates the first Frame number The latent states of each feature point This indicates that the feature point originates from a static scene structure and can be used as an interior point for geometric estimation. This indicates that the feature point originates from a dynamic object or is an external point that does not satisfy the static rigid body assumption. It also represents the feature trajectory number established through adjacent frame feature matching, LK optical flow tracing, or map point association.

[0087] After introducing latent state variables, to avoid information loss caused by hard decisions, posterior probability is used to describe the static nature of features, and it serves as the basis for subsequent front-end suppression and back-end weighting. Therefore, the first... The first frame of the RGB image The static posterior probability of each feature point for:

[0088] (2)

[0089] in, This means "defined as". Indicates the first The feature trajectory from the first time a valid association was established to the current [number]th [item]. The set of historical multi-source observational evidence obtained from the frame, subscript 1: Indicates the current number The continuous historical observation range of the frame. Formula (2) is the static posterior probability. The definition and expression form.

[0090] To unify semantic and geometric information, a multi-source observation evidence vector is constructed:

[0091] (3)

[0092] In the formula, For the first The first frame of the RGB image A multi-source observation evidence vector for each feature point For the first The first frame of the RGB image Semantic observation evidence vector of each feature point For the first The first frame of the RGB image The observation evidence vector of each feature point For the first The first frame of the RGB image The epipolar geometric observation evidence vector of a feature point For the first The first frame of the RGB image The depth residual observation evidence vector of each feature point (if the depth measurement value is missing, it is considered that there is no such evidence).

[0093] In practical applications, the direct summation of the static posterior probabilities expressed by formula (2) may lead to large errors. To make the static posterior probabilities more accurate, in this embodiment, it is transformed into a form in the logarithmic domain to obtain a logarithmic probability form, which is then inversely transformed into a static posterior probability for use in dynamic feature suppression and back-end optimization. The joint likelihood is decomposed into a product of modal likelihoods using the conditional independence approximation, thus achieving additivity in the logarithmic domain, as shown below:

[0094] (4)

[0095] In the formula, This indicates a series of multiplication operations. This represents the set of types of multi-source observational evidence. This represents an element in the set.

[0096] In one specific embodiment of this invention, observation information from different sources such as semantics, optical flow, epipolarity, and depth can be uniformly fused under a Bayesian framework through formula (4), providing a probabilistic model basis for the static posterior probability estimation of feature points. Through the above probabilistic dynamic state model, this invention transforms the question of whether a feature is static into a static posterior probability estimation problem, so that dynamic feature suppression no longer depends on single-frame thresholds or binary elimination, but is based on multi-source evidence to form a continuous probability judgment.

[0097] In one specific embodiment of this invention, since directly fusing multi-source observational evidence in the probability domain may lead to numerical instability and makes it difficult to uniformly process observations with different dimensions, this embodiment introduces the Log-Likelihood Ratio (LLR) as a unified representation of various pieces of evidence. The LLR quantifies the likelihood ratio of a single observational piece of evidence under both static and dynamic hypotheses, characterizing its relative support for the two types of hypotheses.

[0098] (5)

[0099] In the formula, Indicates the first The first frame of the RGB image Log-likelihood ratio of various types of observational evidence for each feature point. This indicates that the evidence tends to the static hypothesis, and vice versa. Formula (5) is the unified definition of the log-likelihood ratio for each type of observational evidence. For specific calculations, see formulas (10)-(14).

[0100] static posterior probability Mapped to the log-odds space, which refers to the log-odds ratio corresponding to the static posterior probability. The logarithmic field serves as the state variable. Its function is to transform probabilistic multiplicative updates into additive updates in the logarithmic field, thereby facilitating cross-frame recursive fusion of observational evidence from different sources and improving numerical computational stability. The resulting modified formula is as follows:

[0101] (6)

[0102] In the formula, Static posterior probability The log-odds representation.

[0103] Under the conditional independent decomposition of formula (4), the Bayesian update can be transformed into the log-cumulative form of the log-likelihood ratio LLR.

[0104] Considering that the state of the tracked target in a dynamic scenario may change over time, and historical evidence is not always valid, a forgetting factor is introduced. To attenuate the influence of historical information; and to balance the scale differences among various modal evidences, a calibration coefficient is introduced. By weighting the various types of evidence, the following recursive form is obtained:

[0105] (7)

[0106] In the formula, Used to balance the contribution strength of different sources of evidence in log-likelihood ratio fusion. Used to control the degree to which historical static states are preserved in the estimation of the current frame. .

[0107] After recursion, the static posterior probability is recovered by inverse Sigmoid transform:

[0108] (8)

[0109] In the formula, This represents the th element recovered after the inverse Sigmoid transform. The first frame of the RGB image The static posterior probability of a feature point For the Sigmoid function, Represents a graph with the natural constant e as the base and an exponent of e. The exponential function term is used to calculate the static posterior probability using the Sigmoid function.

[0110] In the process of calculating the static posterior probability after the inverse Sigmoid transform, the heterogeneous observations are first uniformly mapped to the log-likelihood ratio, and then cross-frame probability accumulation is achieved in the log-odds space, which fundamentally avoids estimation instability caused by semantic missegmentation or short-term occlusion in single-frame hard decision.

[0111] IV. Calculating the log-likelihood ratio

[0112] 3.1 Log-likelihood ratio of semantic prior verification data

[0113] In one specific embodiment of this invention, the semantic segmentation or detection model essentially provides "region mobility" information. If all feature points within the mask are directly binary-classified as dynamic, features that originally belong to the static background within the region will be mistakenly deleted. To address this issue, this embodiment performs probabilistic modeling of the semantic information: firstly, it is used as a soft prior for dynamic probability, converted into LLR through an inverse sensor model, and then injected into the recursive process; simultaneously, a boundary reliability mechanism is introduced to reduce the risk of misjudgment caused by segmentation errors near the mask boundary.

[0114] Construct a dynamic probability prior induced by semantics and convert it into the log-likelihood ratio of the semantic prior data:

[0115] (9)

[0116] (10)

[0117] In the formula, Indicates the first The first frame of the RGB image The semantic meaning of each feature point is first verified by the log-likelihood ratio of the data. Indicates the first Frame RGB image The dynamic probability prior of semantic induction for feature points, where the superscript D denotes the dynamic state and the superscript sem denotes semantic evidence. This is to prevent logarithmic divergence. , Indicates the first The first frame of the RGB image The mask boundary reliability for each feature point is determined based on the distance from the pixel coordinates of the feature point to the mask boundary of the associated instance. This represents the set of target categories for the instance segmentation model. This represents the category index in the target category set of the instance segmentation model. For category The presupposed a priori mobility, Indicates the relationship with the first Frame number The instance number associated with each feature point. For the first On the RGB image of the first frame Instances of association between feature points Category The class probability, Let be the preset probability cutoff constant, and .

[0118] 3.2 Log-likelihood ratio of evidence for LK optical flow consistency

[0119] In one specific embodiment of the present invention, the log-likelihood ratio of the LK optical flow consistency evidence is calculated using the following formula:

[0120] (11)

[0121] In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of LK optical flow consistency evidence corresponding to each feature point, where N is the residual dimension. For static residuals, For dynamic residuals, Indicates the first The first frame of the RGB image The residual energy of each feature point.

[0122] In the formula, and Let represent the LK optical flow residual scale parameters under the static and dynamic state assumptions, respectively, and satisfy . > >0. The photometric residuals of static features are typically concentrated near zero, therefore a relatively small static residual scale is used. The photometric residual distribution of dynamic features, occluded points, or mismatched points is usually wide, therefore a relatively large dynamic residual scale is used. The aforementioned and The optical flow residuals are determined based on the statistical results of pre-collected static and dynamic feature samples, and remain constant during system operation. As the residual energy... Increase the log-likelihood ratio of the LK optical flow consistency evidence. This decreases the likelihood of the hypothesis being supported more strongly.

[0123] 3.3 Log-likelihood ratio of evidence for geometric consistency of polar lines

[0124] In one specific embodiment of the present invention, the log-likelihood ratio of the epipolar geometric consistency evidence is calculated using the following formula:

[0125] (12)

[0126] In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of the epipolar geometric consistency evidence corresponding to each feature point. Represents the static epipolar residual scale. Indicates the dynamic epipolar residual scale. Indicates the first The first frame of the RGB image The normalized epipolar residual of each feature point is used to measure the degree of deviation of the matched feature points in adjacent frames from the epipolar geometric constraints.

[0127] Normalized Pole Residual The larger the value, the more the evidence tends to support the dynamic hypothesis; the smaller the extreme residual, the more the evidence tends to support the static hypothesis.

[0128] 3.4 Log-likelihood ratio of deep residual evidence

[0129] In one specific embodiment of the present invention, the log-likelihood ratio of the deep residual evidence is calculated using the following formula:

[0130] (13)

[0131] (14)

[0132] In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of the depth residual evidence corresponding to each feature point Represents the Gaussian probability density function, for example This indicates that the mean is 0 and the variance is . The Gaussian probability density function is used to describe the probability distribution of depth residuals under static and dynamic assumptions. Indicates the first The first frame of the RGB image The depth residual corresponding to each feature point Represents the static depth residual scale. Represents the static depth residual scale. Indicates the first The first frame of the RGB image Depth measurement values ​​of each feature point Indicates the first The first frame of the RGB image The depth prediction value of each feature point.

[0133] By calculating the log-likelihood ratios of semantic prior evidence, LK optical flow consistency evidence, epipolar geometric consistency evidence, and depth residual evidence, multi-source heterogeneous observations are uniformly converted into log-likelihood ratio form, enabling various types of evidence to serve as fusion inputs for the same feature-level Bayesian dynamic estimation module.

[0134] Since semantics are introduced only as a moving prior and automatically downweighted by reliability factors in the boundary region, while geometric consistency evidence can be accumulated across frames, static background points within the dynamic instance region can still gradually increase their static posterior probability through continuous positive geometric evidence, thereby achieving the preservation of static features within the bounding box and continuous suppression of real dynamic out-of-bounds points.

[0135] V. Back-end beam adjustment and optimization

[0136] After completing the feature-level Bayesian dynamic estimation (i.e., after calculating the static posterior probability of each feature point), this invention uses the static posterior probability obtained by formula (8) as a unified dynamic suppression metric in front-end tracking and back-end local bundle adjustment optimization. Feature points with higher static posterior probabilities are considered to be more likely to come from static scene structures and retain a strong constraint effect in front-end pose estimation and back-end local bundle adjustment optimization. Feature points with lower static posterior probabilities are considered to be more likely to come from dynamic targets, occluded regions, or erroneous association points and are continuously downweighted in the front-end and back-end.

[0137] In the front-end tracking stage, this invention does not employ a hard-decision approach that directly eliminates features from potential dynamic regions. Instead, it uses soft-weighted suppression of feature observations based on the static posterior probability of the features. For feature points that fall into potential dynamic instance regions but still satisfy optical flow consistency, epipolar geometric consistency, or depth consistency (corresponding to small residuals in the evidence, and making the log-likelihood ratio of the corresponding evidence positive or generally supporting the static state hypothesis), their static posterior probability can remain at a high level during cross-frame recursion, thus still serving as an effective constraint in pose estimation. For features on real dynamic targets, since their semantic and geometric evidence continuously support the dynamic hypothesis, their static posterior probability gradually decreases, and is thus suppressed in front-end pose estimation.

[0138] Through the aforementioned front-end soft suppression mechanism, feature points are not simply retained or deleted, but continuously participate in pose estimation based on their static credibility. This approach avoids the problem of erroneous deletion of effective static features caused by traditional semantic mask hard culling strategies, and maintains necessary geometric constraints in scenarios with high dynamic occlusion ratios or relatively scarce static features, thereby improving the stability of front-end tracking.

[0139] Filtering or reducing weights at the front end alone is insufficient to completely prevent dynamic outliers from interfering with local mapping and bundle adjustment optimization. Therefore, this invention further uses the static posterior probability output by the feature-level Bayesian dynamic estimation module as the back-end observation weight, ensuring that the observation reliability enters the optimization process in a consistent manner between the front and back ends. That is, filtering or reducing weights at the front end alone is insufficient to completely prevent dynamic outliers from skewing local mapping and bundle adjustment. Therefore, this invention uses the static posterior probability obtained by formula (8) as the back-end observation weight, ensuring that the observation reliability enters the optimization process in a consistent manner between the front and back ends, thereby maintaining observability and stable convergence even when the dynamic occlusion ratio is large or static features are scarce.

[0140] For observations of keyframes and map points, the reprojection residual is defined as:

[0141] (15)

[0142] In the formula, Keyframe With map points The resulting observation edge corresponds to the two-dimensional reprojection residual, i.e., the map point. In keyframe The difference between the actual observed pixel position and the predicted pixel position calculated based on the keyframe camera pose and the 3D coordinates of the map points. Indicates the keyframe number. Indicates the map point number. Represents map points In keyframe Two-dimensional pixel observation in Keyframe The camera pose; Represents map points 3D coordinates; This represents the camera projection function, which projects 3D map points onto a 2D image plane according to the keyframe camera pose and camera intrinsic parameters.

[0143] To reflect that "the same observation may be a static interior point or a dynamic exterior point", a two-component mixture model is adopted, and the static posterior probability output by the feature-level Bayesian dynamic estimation module is used. As a mixing coefficient:

[0144] (16)

[0145] In the formula, This represents the mixture probability density of the reprojection residual given a static posterior probability. The mixing coefficient representing the static interior component of the observed edge ( The essence is the observation edge The corresponding static posterior probability; in the two-component mixture model, this static posterior probability also serves as the mixing coefficient of the static interior components), which can be output by formula (8). Corresponding to the observation edge Obtain; 1- Indicates the observation edge The mixing coefficient belongs to the dynamic external point component; For static interior point residual covariance, For dynamic exterior point residual covariance, It means greater in the sense of positive definiteness. This indicates that the distribution range of the out-point residuals is greater than that of the in-point residuals. Formula (16) indicates that when the static posterior probability of the observed edge is high, the observation is closer to the static in-point model; when the static posterior probability is low, the observation is closer to the dynamic out-point model.

[0146] When keyframe The first in Feature points and map points When establishing observation associations, the static posterior probability of the feature point is mapped to the keyframe. With map points The observed edge is formed. The static posterior probability of this feature point obtained by equation (8) is... Corresponding to the observation edge , recorded as ,Right now

[0147] Formula (8) defines the first The first frame of the RGB image The static posterior probability of each feature point When the first The frame was selected as a keyframe. At that time, the frame number Specifically, it is recorded as the keyframe number. Therefore, the corresponding probability can be written as .

[0148] Accurately solving for the maximum a posteriori (MAP) estimate of a mixture model requires handling nonlinear terms, and direct optimization is costly. To ensure compatibility with the quadratic residual form in classic business arithmetic (BA), a monotonic mapping function is introduced. Convert the static posterior probability into observation weights:

[0149] (17)

[0150] In the formula, This represents a monotonic mapping function from static posterior probability to observed weights. Indicates the observation edge Backend observation weights; This represents the minimum observation weight, used to avoid numerical degradation caused by a weight of 0. This parameter represents the suppression strength in the mapping process from static posterior probability to back-end observation weights, and is used to adjust the degree of weight reduction for observations with low static posterior probability. .

[0151] The final local BA adopts "FBDE weights ( A unified weighted form of "+robust kernel":

[0152] (18)

[0153] Equation (18) is the objective function for local bundle adjustment optimization. By minimizing this weighted projection error, the optimized set of local keyframe poses and the set of 3D coordinates of local map points are finally obtained. Where, This represents the keyframe—the set of map point observation edges—within a local BA window. For robust kernel functions; The information matrix representing the static interior point residuals.

[0154] The function of this formula is to simultaneously utilize the static posterior probability weights and robust kernel function output by FBDE during the back-end local bundle adjustment process to jointly optimize the keyframe pose and map point coordinates, thereby obtaining the optimized local keyframe pose and local map point 3D coordinates, thus improving the stability of local map and camera pose estimation in dynamic environments.

[0155] VI. Experiment

[0156] In one specific embodiment of this invention, the basic system is defined as the ORB-SLAM3 baseline system, and the improved dynamic visual SLAM system of this invention is defined as the dynamic visual FBDE-SLAM system. Here, FBDE represents the feature-level Bayesian dynamic estimation module. To verify the tracking and mapping capabilities of the dynamic visual FBDE-SLAM system under dynamic disturbances, its evaluation metrics, experimental results, and camera pose trajectories are compared with those of the ORB-SLAM3 baseline system. Experiments are conducted on the publicly available TUM RGB-D dataset and real-world sequences acquired using a RealSense D435i depth camera. To comprehensively cover scenes with different dynamic intensities, the Freiburg3 walking sequence is selected as a representative high-dynamic scene, and the Freiburg3 sitting sequence is selected as a representative low-dynamic scene for quantitative comparison. The experiments are divided into simulation and real-world scene verification.

[0157] 6.1 Simulation

[0158] 6.1.1 Experimental Environment

[0159] The experimental platform was configured as follows: operating system: Ubuntu 20.04; processor: Intel Core i5-10200H (16 GB RAM); GPU: NVIDIA graphics card (driver version 535). To reduce the impact of randomness on the evaluation results, each test sequence was run independently 10 times, and the median of each indicator was taken as the final statistical result.

[0160] 6.1.2 Experimental Results

[0161] This embodiment selects the typical high-dynamic sequence freiburg3 walking and the low-dynamic sequence freiburg3 sitting from the TUM RGB-D dataset for experimental evaluation, and compares them with the ORB-SLAM3 baseline system. The evaluation metric used is Absolute Trajectory Error (ATE) to measure global trajectory consistency and accuracy. The comparison results of the absolute trajectory error (ATE) of the dynamic vision FBDE-SLAM system and the ORB-SLAM3 baseline system on the TUM dataset are shown in Table 1 below. The simulation results are as follows: Figures 2-3 As shown.

[0162] Table 1. Comparison of Absolute Trajectory Error (ATE) between the two systems on the TUM dataset.

[0163] In Table 1, RMSE represents the root mean square error, Mean represents the mean error, and SD represents the standard deviation. As shown in Table 1, FBDE-SLAM achieves significant advantages in high-dynamic walking sequences: the root mean square error (RMSE) of ATE can be reduced by up to 89.90%–95.25% compared to ORB-SLAM3, while the dispersion of the error distribution is significantly reduced. However, in low-dynamic sequences such as fr3_sitting_xyz, since the scene approximately satisfies the static world assumption most of the time, ORB-SLAM3, relying on interior point selection mechanisms such as RANSAC, can already obtain a relatively stable feature set; therefore, the performance improvement of FBDE-SLAM is relatively limited.

[0164] Figures 2 to 5 In the diagram, green dots represent static feature points, and purple dots represent identified dynamic feature points.

[0165] Figure 2 and Figure 3 These diagrams compare the feature point detection and retention results of the ORB-SLAM3 baseline system and the dynamic vision FBDE-SLAM system in dynamic scenes. The retention result refers to the system's determination of whether each feature point continues to participate in camera pose estimation and its weight in pose estimation, based on the static reliability of the feature points after feature point extraction. Figure 2 In the dynamic background, the moving person is considered. Using the ORB-SLAM3 baseline system, feature points from the moving person's surface and the static background will be used together for camera pose estimation (e.g., ...). Figure 2 The feature points identified in the two red boxes are all classified as static feature points, causing dynamic features to interfere with the pose estimation results. Figure 3 In this context, the dynamic visual FBDE-SLAM system can preserve static feature points within a dynamic background within a bounding box. Figure 3 The red box in the middle shows identified static and dynamic feature points; only the identified dynamic feature points are removed. This demonstrates that the method of this invention improves the accuracy of localization and mapping.

[0166] Depend on Figure 2 and Figure 3 It can be seen that the ORB-SLAM3 baseline system detects a large number of matchable feature points in dynamic regions such as moving human bodies. However, it treats all of these feature points as static feature points and directly uses them in camera pose estimation, resulting in a decrease in the accuracy of the generated camera trajectory. In contrast, the dynamic vision FBDE-SLAM system can effectively distinguish and retain static usable feature points in dynamic regions, thus maintaining geometric observability even with large-scale occlusion or scarce static features.

[0167] 6.2 Real-world scenarios

[0168] 6.2.1 Experimental Environment

[0169] To further verify the effectiveness of the dynamic vision FBDE-SLAM system in a real-world environment, this invention designed and conducted experimental tests in an indoor setting. The visual sensor used was a RealSense D435i depth camera with an image resolution of 1280×720 and a frame rate of 20 Hz. The computing platform consisted of an NVIDIA Jetson TX2 embedded module equipped with a dual-core NVIDIA Denver 2 64-bit CPU. The software environment was built on the Ubuntu 20.04 operating system and the ROS Noetic robot operating system framework.

[0170] Before the experiment, the distortion parameters involved in the camera were calibrated. Then, RGB images and depth data sequences were acquired from an experimenter moving freely indoors, and the dynamic visual FBDE-SLAM system was run based on this data to estimate the weights of dynamic / static feature points.

[0171] 6.2.2 Experimental Results

[0172] In one specific embodiment of the present invention, the camera pose trajectory is used instead of the actual path trajectory of the vehicle. For example... Figure 6 As shown in the figure, the motion trajectories of the dynamic visual FBDE-SLAM system and the ORB-SLAM3 baseline system are compared in a real indoor scene. The black dashed line represents the actual motion trajectory of the vehicle, the blue solid line represents the camera pose trajectory generated by the ORB-SLAM3 baseline system, and the green solid line represents the camera pose trajectory generated by the dynamic visual FBDE-SLAM system. The comparison shows that the trajectory estimated by the dynamic visual FBDE-SLAM system is closer to the actual trajectory and has less fluctuation, indicating that this method can estimate the camera pose more stably and accurately in real dynamic environments. The comparison results of the motion trajectory error ATE of the dynamic visual FBDE-SLAM system and the ORB-SLAM3 baseline system in a real indoor scene are shown in Table 2 below, and the experimental results are shown in the figure below. Figures 4-5 As shown.

[0173] Table 2 Comparison of Absolute Trajectory Error (ATE) of the two systems in real-world scenarios

[0174] As can be seen from Table 2, the positioning accuracy (calculated from Mean data) of the method proposed in this invention is improved by 67.2%, while the running speed is improved by 17.6% (calculated from average time).

[0175] Figure 4In the context of movement, the area where the person and the object they are carrying are considered potential dynamic target areas. Due to the lack of an explicit dynamic state discrimination mechanism, the basic visual SLAM system will cause the feature points on the surface of the dynamic target to participate in pose estimation together with the feature points of the static background. The conventional processing method of semantic mask overall removal may accidentally delete effective features near the mask boundary or those that are actually in a static state while deleting dynamic features. Figure 5 In this invention, the static posterior probability of each feature point is calculated by multi-source evidence fusion, and the real dynamic features are continuously weighted. At the same time, effective features that still satisfy geometric consistency in the static background and potential dynamic regions are retained, thus taking into account both dynamic interference suppression and effective geometric constraint retention.

[0176] Depend on Figure 4 and Figure 5 The comparison shows that the present invention can reduce the interference of dynamic features of the moving human body surface on camera pose estimation, while avoiding the loss of effective static constraints caused by directly deleting all feature points in the potential dynamic area, thereby improving the continuity of camera pose estimation and system robustness in dynamic scenes.

[0177] Example 3

[0178] This embodiment describes a dynamic visual SLAM device, including the following:

[0179] The multi-source input data acquisition module is configured to acquire multi-source input data at the current time through an improved dynamic visual SLAM system. The multi-source input data includes RGB images and depth images. The improved dynamic visual SLAM system is obtained by adding a semantic module and a feature-level Bayesian dynamic estimation module to the SLAM system (ORB-SLAM3). The improved dynamic visual SLAM system includes a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module, and a back-end optimization module.

[0180] The front-end tracking module is used to extract features from the RGB image at the current moment to obtain the feature points of the current frame; by observing the feature points on the RGB images of adjacent frames, it obtains the LK optical flow consistency information, epipolar geometric consistency information and depth residual information of each feature point, and calculates the log-likelihood ratio of the LK optical flow consistency evidence, epipolar geometric consistency evidence and depth residual evidence based on the above information respectively.

[0181] The semantic module is used to perform instance segmentation on the depth image of the previous time step, obtain the semantic prior information corresponding to each feature point, and calculate the log-likelihood ratio of the semantic prior information based on the semantic prior information.

[0182] The feature-level Bayesian dynamic estimation module is used to fuse the log-likelihood ratio of multi-source observation evidence, calculate the static posterior probability of each feature point, and input the static posterior probability into the local mapping module and the front-end tracking module; wherein, the multi-source observation evidence includes semantic prior evidence, LK optical flow consistency evidence, epipolar geometric constraint evidence, and depth residual evidence;

[0183] The front-end tracking module is also used to receive static posterior probabilities to determine keyframes of the image and input the keyframes into the local mapping module;

[0184] The local mapping module is used to receive static posterior probabilities to generate map points, insert keyframes, and perform local map maintenance.

[0185] The backend optimization module maps static posterior probabilities to observation weights and works in conjunction with a robust kernel function in the local bundle adjustment optimization process to output the camera motion trajectory.

[0186] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A dynamic visual SLAM method, characterized in that, include: The improved dynamic visual SLAM system acquires multi-source input data at the current moment. The multi-source input data includes RGB images. The improved dynamic visual SLAM system obtains this data by adding a semantic module and a feature-level Bayesian dynamic estimation module to the SLAM system. The improved dynamic visual SLAM system includes a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module, and a back-end optimization module. The front-end tracking module is used to extract features from the RGB image at the current moment to obtain the feature points of the current frame; based on the multi-source input data between adjacent RGB images, it obtains the LK optical flow consistency information, epipolar geometric consistency information, and depth residual information of each feature point, and calculates the log-likelihood ratio of the multi-source observation evidence based on the above information; the multi-source observation evidence includes semantic prior evidence, LK optical flow consistency evidence, epipolar geometric constraint evidence, and depth residual evidence; The semantic module is used to perform instance segmentation on the RGB image at the current time, obtain the semantic prior information corresponding to each feature point, and calculate the log-likelihood ratio of the semantic prior data based on the semantic prior information. The feature-level Bayesian dynamic estimation module is used to fuse the log-likelihood ratio of multi-source observation evidence, calculate the static posterior probability of each feature point, and input the static posterior probability into the local mapping module and the front-end tracking module. The front-end tracking module is also used to receive static posterior probabilities to determine keyframes in RGB images and input the keyframes into the local mapping module; The local mapping module is used to receive static posterior probabilities to generate map points, insert keyframes, and perform local map maintenance. The backend optimization module maps static posterior probabilities to observation weights and works in conjunction with a robust kernel function in the local bundle adjustment optimization process to output the camera motion trajectory.

2. The dynamic visual SLAM method according to claim 1, characterized in that, The semantic module is used to perform instance segmentation on the RGB image at the current time step and obtain the semantic prior information corresponding to each feature point. The semantic module is used to perform instance segmentation on the depth image at the previous time step and output the instance mask, class label, class probability and confidence of the potential dynamic target online. The current frame feature points are associated with the instance mask at the pixel level to obtain the semantic prior information corresponding to each feature point.

3. The dynamic visual SLAM method according to claim 1, characterized in that, The multi-source input data also includes depth images, camera intrinsics, the camera pose of the previous frame, historical keyframes, local map points, and feature matching information of adjacent frames. The step of obtaining LK optical flow consistency information, epipolar geometric consistency information, and depth residual information for each feature point based on multi-source input data between adjacent RGB images includes: obtaining LK optical flow consistency information, epipolar geometric consistency information, and depth residual information for each feature point based on feature matching information between adjacent RGB images, camera intrinsic parameters, initial camera pose, depth image, and local map points; wherein, camera intrinsic parameters are used to calculate the geometric relationship between image coordinates and projection model, the previous frame camera pose, historical keyframes, and local map points are used to establish the geometric association between the current frame image and the existing map, adjacent frame feature matching information is used for optical flow consistency and epipolar geometric consistency analysis, and the initial camera pose is calculated based on adjacent frame feature matching information, the previous frame camera pose, and local map points.

4. The dynamic visual SLAM method according to claim 1, characterized in that, The log-likelihood ratio of the semantic prior evidence is calculated using the following formula: ， ， In the formula, Indicates the first The first frame of the RGB image The semantic meaning of each feature point is first verified by the log-likelihood ratio of the data. Indicates the first Frame RGB image The dynamic probability prior of semantic induction for feature points, where the superscript D denotes the dynamic state and the superscript sem denotes semantic evidence. This is to prevent logarithmic divergence. Indicates the first The first frame of the RGB image The mask boundary reliability for each feature point is determined based on the distance from the pixel coordinates of the feature point to the mask boundary of the associated instance. This represents the set of target categories for the instance segmentation model. This represents the category index in the target category set of the instance segmentation model. For category The presupposed a priori mobility, Indicates the relationship with the first Frame number The instance number associated with each feature point. For the first On the RGB image of the first frame Instances of association between feature points Category The class probability, This is a preset probability cutoff constant.

5. The dynamic visual SLAM method according to claim 4, characterized in that, The log-likelihood ratio of the LK optical flow consistency evidence was calculated using the following formula: ， In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of LK optical flow consistency evidence corresponding to each feature point, where N is the residual dimension. For static residuals, For dynamic residuals, Indicates the first The first frame of the RGB image The residual energy of each feature point.

6. The dynamic visual SLAM method according to claim 5, characterized in that, The log-likelihood ratio of the evidence for epipolar geometric consistency is calculated using the following formula: ， In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of the epipolar geometric consistency evidence corresponding to each feature point. Represents the static epipolar residual scale. Indicates the dynamic epipolar residual scale. Indicates the first The first frame of the RGB image Normalized epipolar residuals of each feature point.

7. The dynamic visual SLAM method according to claim 6, characterized in that, The log-likelihood ratio of deep residual evidence is calculated using the following formula: ， ， In the formula, Indicates the first The first frame of the RGB image The log-likelihood ratio of the depth residual evidence corresponding to each feature point This represents the Gaussian probability density function. Indicates the first The first frame of the RGB image The depth residual corresponding to each feature point Represents the static depth residual scale. Indicates the dynamic depth residual scale. Indicates the first The first frame of the RGB image Depth measurement values ​​of each feature point Indicates the first The first frame of the RGB image The depth prediction value of each feature point.

8. The dynamic visual SLAM method according to claim 7, characterized in that, The static posterior probability is calculated using the following formula: ， ， ， In the formula, The representation is defined as follows: This represents the set of types of multi-source observational evidence. Represents the elements in a set, defined For the first The first frame of the RGB image The static posterior probability of a feature point Static posterior probability The log-dominance ratio is expressed as follows: Forgetting factor, For calibration coefficients, This represents the Sigmoid function. This represents the first digit recovered by the inverse Sigmoid transform. The first frame of the RGB image The static posterior probability of a feature point Represents a graph with the natural constant e as the base and an exponent of e. The exponential function term is used to calculate the static posterior probability using the Sigmoid function.

9. The dynamic visual SLAM method according to claim 8, characterized in that, the observation... The weights are calculated using the following formula: ， In the formula, Indicates the keyframe number. Indicates the map point number. This represents a monotonic mapping function from static posterior probability to observed weights. Keyframe With map points The static posterior probability corresponding to the formed observation edge, which is determined by the keyframe. Central observation map points The feature points are obtained by static posterior probability mapping. Indicates the observation edge Backend observation weights, Indicates the minimum observation weight. This parameter represents the suppression strength during the mapping process from static posterior probability to backend observation weights, and is used to adjust the degree of weight reduction for observations with low static posterior probability.

10. A dynamic visual SLAM device, characterized in that, include: The multi-source input data acquisition module is configured to acquire multi-source input data at the current moment through an improved dynamic visual SLAM system. The multi-source input data includes RGB images. The improved dynamic visual SLAM system is obtained by adding a semantic module and a feature-level Bayesian dynamic estimation module to the SLAM system. The improved dynamic visual SLAM system includes a front-end tracking module, a semantic module, a feature-level Bayesian dynamic estimation module, a local mapping module, and a back-end optimization module. The front-end tracking module is used to extract features from the RGB image at the current moment to obtain the feature points of the current frame; based on the multi-source input data between adjacent RGB images, it obtains the LK optical flow consistency information, epipolar geometric consistency information, and depth residual information of each feature point, and calculates the log-likelihood ratio of the multi-source observation evidence based on the above information; the multi-source observation evidence includes semantic prior evidence, LK optical flow consistency evidence, epipolar geometric constraint evidence, and depth residual evidence; The semantic module is used to perform instance segmentation on the RGB image at the current time, obtain the semantic prior information corresponding to each feature point, and calculate the log-likelihood ratio of the semantic prior data based on the semantic prior information. The feature-level Bayesian dynamic estimation module is used to fuse the log-likelihood ratio of multi-source observation evidence, calculate the static posterior probability of each feature point, and input the static posterior probability into the local mapping module and the front-end tracking module. The front-end tracking module is also used to receive static posterior probabilities to determine keyframes in RGB images and input the keyframes into the local mapping module; The local mapping module is used to receive static posterior probabilities to generate map points, insert keyframes, and perform local map maintenance. The backend optimization module maps static posterior probabilities to observation weights and works in conjunction with a robust kernel function in the local bundle adjustment optimization process to output the camera motion trajectory.

Images