Dynamic environment visual map construction method and system based on OpenSeeD
By combining OpenSeeD and PWC-Net models to construct a dynamic environment visual map, dynamic feature points are eliminated, solving the problems of positioning accuracy and robustness of visual SLAM systems in dynamic environments, and achieving high-precision positioning and mapping results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG NORMAL UNIV
- Filing Date
- 2026-01-17
- Publication Date
- 2026-04-28
AI Technical Summary
Existing visual SLAM systems suffer from decreased positioning accuracy and insufficient robustness in dynamic environments, making it difficult to effectively handle interference from dynamic objects, leading to increased positioning errors and system degradation.
A dynamic environment visual map construction method based on OpenSeeD is adopted. Dynamic masks are obtained through instance segmentation and optical flow estimation. Combined with multiple thresholds and temporal filtering mechanisms, dynamic feature points are removed, and static feature points are retained for pose estimation and map construction.
It significantly improves positioning accuracy and system robustness in dynamic environments, enabling stable and accurate positioning and mapping in complex dynamic scenarios.
Smart Images
Figure CN121937580A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and autonomous robot navigation technology, and more specifically to a method and system for constructing dynamic environment visual maps based on OpenSeeD. Background Technology
[0002] With the continuous development of robotics and intelligent sensing systems, Simultaneous Localization and Mapping (SLAM) has become a key technology supporting autonomous decision-making by intelligent agents, and is widely used in fields such as autonomous driving, drone inspection, and AR / VR virtual-real interaction. Based on the different principles of sensing devices, existing SLAM systems are mainly divided into two branches: visual SLAM and LiDAR SLAM. Visual SLAM relies on high-dimensional semantic and texture information provided by image sensors, eliminating the need for expensive laser hardware, and has received widespread attention from academia and industry in areas such as robot navigation, autonomous driving environmental perception, and AR / VR scene reconstruction. Commonly used visual SLAM sensors include monocular cameras, binocular cameras, and RGB-D cameras, which can simultaneously output color images (RGB) and dense depth maps, directly acquiring 3D environmental information, effectively reducing the complexity of feature matching and depth estimation, and have outstanding application value in various visual SLAM systems.
[0003] Although visual SLAM technology has developed rapidly and been widely applied in various real-world scenarios, it is generally based on the assumption of a "static environment." The numerous dynamic objects present in the real environment (such as pedestrians and vehicles) can interfere with the system's visual observation, leading to decreased positioning accuracy, increased trajectory estimation errors, and in severe cases, even tracking failure. In typical dynamic scenarios such as autonomous driving, if feature points generated by dynamic targets are mistakenly identified as static constraints, it will not only disrupt the system's visual observation but also... Figure 1 Inconsistency can also lead to overall system degradation. Therefore, improving the robustness and accuracy of SLAM in dynamic environments has become an important research direction.
[0004] Therefore, designing a dynamic environment visual map construction method and system based on OpenSeeD to significantly improve the positioning accuracy and system robustness in dynamic environments is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides a method and system for constructing dynamic environment visual maps based on OpenSeeD. By effectively removing dynamic features, it significantly improves the positioning accuracy and system robustness in dynamic environments, and has important practical value in real-world application scenarios such as service robots and autonomous driving.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for constructing dynamic environment visual maps based on OpenSeeD, comprising: Step 1: Obtain a continuous RGB image sequence of the dynamic object; Step 2: Perform instance segmentation and optical flow estimation on the continuous RGB image sequence to obtain the initial mask and optical flow information; Step 3: Fuse and optimize the initial mask and optical flow information, and perform dynamic object recognition on the optimized fusion result to obtain a dynamic mask; Step 4: Remove dynamic feature points from the dynamic mask, retain static feature points in the static region, and input the static feature points into the ORB-SLAM3 library to complete pose estimation and map construction.
[0007] Preferably, the instance segmentation process in step 2 includes: performing end-to-end instance segmentation on a continuous RGB image sequence using the OpenSeeD model based on the Transformer architecture, and retaining instance masks with confidence scores higher than 0.5 through non-maximum suppression to obtain an initial mask.
[0008] Preferably, the optical flow estimation in step 2 uses the PWC-Net model to calculate the dense optical flow field between adjacent frames. The PWC-Net model constructs an image pyramid, iteratively calculates the cost volume and image warping, and outputs the motion vector of each pixel in the adjacent frame to obtain optical flow information.
[0009] Preferably, step 3, based on the motion vectors of pixels in each adjacent frame image, identifies moving objects within the initial mask region obtained from instance segmentation using a dynamic decision-making strategy with multiple thresholds, specifically including: The final dynamic threshold flow_3 is calculated by fusing two intermediate thresholds flow_1 and flow_2, as shown in the following expression: ; The two intermediate thresholds are calculated as follows: ; ; Wherein, flow_median represents the median of the optical flow amplitude of all pixels within the instance mask region; flow_std represents the standard deviation of the optical flow amplitude; and flow_75th represents the 75th percentile of the optical flow amplitude.
[0010] Preferably, the motion state of an object is determined by comparing the relationship between optical flow statistics and the final dynamic threshold flow_3.
[0011] Preferably, after obtaining the initial mask, a temporal filtering mechanism is introduced to smooth the optical flow information of multiple consecutive frames, including: The probability value of each pixel being judged as dynamic in each frame is taken as a temporal sequence, and the dynamic probability of pixel i in the current frame t is... The dynamic probability sequence is smoothed using an exponentially weighted moving average algorithm, calculated according to the following formula:
[0012] in, This indicates the smoothing probability of the pixel in the previous frame. Let be the smoothing factor, and be the smoothed probability. Used to generate dynamic masks.
[0013] Preferably, a dynamic environment visual map construction system based on OpenSeeD includes: Image acquisition module: used to acquire a continuous RGB image sequence of dynamic objects; Dynamic perception module: used to perform instance segmentation and optical flow estimation on continuous RGB image sequences to obtain initial mask and optical flow information; Fusion and optimization module: used to fuse and optimize the initial mask and optical flow information, and to perform dynamic object recognition on the optimized fusion result to obtain a dynamic mask; Map building module: Used to remove dynamic feature points from dynamic masks, retain static feature points in static areas, and input the static feature points into the ORB-SLAM3 library to complete pose estimation and map building.
[0014] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for constructing dynamic environment visual maps based on OpenSeeD, which has the following beneficial effects: 1) A dynamic environment adaptation scheme for OP-SLAM based on ORB-SLAM3 extension is proposed. Test results on public datasets such as TUM RGB-D show that, compared with the original framework and other dynamic SLAM methods, this algorithm has higher trajectory accuracy and robustness in dynamic scenes.
[0015] 2) A dynamic discrimination mechanism combining instance segmentation and optical flow estimation was constructed: OpenSeeD was used to obtain object masks to initially locate potential dynamic regions, and PWC-Net optical flow vectors and motion thresholds were combined to achieve accurate identification and removal of dynamic feature points.
[0016] 3) The proposed dual dynamic detection mechanism effectively reduces the risk of misjudgment that is easily generated by a single method, significantly reduces the interference of dynamic features on the SLAM system, and enhances the algorithm's adaptability in complex dynamic environments. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of the method provided by the present invention.
[0019] Figure 2 The OpenSeeD module structure diagram provided by this invention.
[0020] Figure 3 This is a structural diagram of the PWC-Net module provided by the present invention.
[0021] Figure 4 This is a schematic diagram illustrating the results of a dynamic scenario provided by the present invention.
[0022] Figure 5 This is a schematic diagram of the mask extraction method provided by the OpenSeeD module of this invention.
[0023] Figure 6 This is a visual schematic diagram of the PWC-Net module provided by the present invention.
[0024] Figure 7 This is a schematic diagram of single-person feature point removal provided by the present invention.
[0025] Figure 8 This is a schematic diagram of feature point removal for multiple users provided by the present invention. Detailed Implementation
[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0027] This invention acquires real-time image sequences of the environment; utilizes the OpenSeeD semantic segmentation model to obtain semantic prior masks of potential moving objects, providing category-level guidance for dynamic perception; integrates a PWC-Net optical flow estimation module, accurately identifying real moving objects through motion consistency analysis and an adaptive threshold determination mechanism; further, it introduces an exponentially weighted smoothing temporal filtering strategy to fuse optical flow information from consecutive frames, enhancing the temporal consistency and stability of the dynamic mask. These multimodal dynamic perception modules are seamlessly integrated into the ORB-SLAM3 framework to construct a robust enhanced dynamic SLAM system. This system, through three levels of processing—semantic-guided focusing, optical flow motion verification, and temporal filtering smoothing—precisely eliminates dynamic feature points, effectively resists camera shake and scene change interference, and reconstructs a high-precision static environment map, thereby achieving stable and accurate localization and mapping in drastically dynamic scenes. It has applications in high-dynamic environment perception fields such as autonomous driving, mobile robots, and augmented reality.
[0028] This invention discloses a method for constructing dynamic environment visual maps based on OpenSeeD, comprising: Step 1: Obtain a continuous RGB image sequence of the dynamic object; Step 2: Perform instance segmentation and optical flow estimation on the continuous RGB image sequence to obtain the initial mask and optical flow information; Step 3: Fuse and optimize the initial mask and optical flow information, and perform dynamic object recognition on the optimized fusion result to obtain a dynamic mask; Step 4: Remove dynamic feature points from the dynamic mask, retain static feature points in the static region, and input the static feature points into the ORB-SLAM3 library to complete pose estimation and map construction.
[0029] like Figure 1 As shown, it includes: Data Acquisition and Input: This method acquires a continuous sequence of RGB images, which may contain corresponding depth information, providing raw data input for subsequent processing. In practical deployments, this method supports real-time acquisition of image streams from a standard RGB-D camera.
[0030] Dynamic perception and mask generation: This step executes two key subtasks in parallel and obtains accurate dynamic region localization through information fusion. Feature selection and SLAM core S1 process: Based on the optimized dynamic mask, the system actively removes feature points falling within the dynamic region during the ORB feature extraction stage, retaining only the features from the static region and inputting them into the ORB-SLAM3 tracking, mapping, and loop closure detection threads to complete accurate pose estimation and map construction.
[0031] Specifically, the instance segmentation process in step 2 includes: using the OpenSeeD model based on the Transformer architecture to perform end-to-end instance segmentation on the continuous RGB image sequence, and retaining instance masks with confidence scores higher than 0.5 through non-maximum suppression to obtain the initial mask.
[0032] Specifically, in step 2, the optical flow estimation uses the PWC-Net model to calculate the dense optical flow field between adjacent frames. The PWC-Net model constructs an image pyramid, iteratively calculates the cost volume and image warping, and outputs the motion vector of each pixel in the adjacent frame to obtain optical flow information.
[0033] In a specific embodiment provided by this invention, firstly, the OpenSeeD model is used for instance segmentation to obtain the initial mask of the potential dynamic object. OpenSeeD employs a Transformer architecture and a dynamic mask generation mechanism, and its network structure is as follows: Figure 2 As shown, the model extracts image features through a ResNet-50 backbone network and combines them with a Transformer encoder-decoder structure based on positional encoding. The encoder consists of 6 layers, each containing a multi-head self-attention mechanism and a feedforward network; the decoder also has 6 layers, fusing image features and object queries through a cross-attention mechanism to generate the class probability and pixel-level mask for each instance. Finally, non-maximum suppression is used to remove redundant predictions, retaining the masks of instances with a confidence level higher than 0.5 as the initial localization of potential dynamic regions. Figure 5 The demonstration shows the instance segmentation effect of the OpenSeeD module in a real scene, demonstrating its ability to accurately identify dynamic objects such as pedestrians and vehicles in the scene.
[0034] Secondly, the PWC-Net model is used for optical flow estimation to calculate pixel motion between adjacent frames. PWC-Net is based on a three-stage architecture of "pyramid, warp, and cost volume," and its detailed calculation process is as follows: Figure 3 As shown. The model first constructs a 6-layer pyramid for each of the two input frames, with a scale reduction factor of 0.75 for each layer. At the top of the pyramid, the feature matching cost volume between the two frames is calculated, and the pixel correspondence is learned through a convolutional neural network. The subsequent frame image is then warped to the coordinate system of the previous frame. Then, the optical flow of the higher layers is upsampled to the next layer as the initial value, and the cost volume calculation and warping process are repeated. After 5 iterations of optimization, a dense optical flow field is finally output at the original resolution, obtaining the motion vector of each pixel. , ]. Figure 6 The visualization results of the optical flow field generated by PWC-Net are shown, and the motion of each region in the scene is clearly displayed through color coding.
[0035] Furthermore, after obtaining the instance segmentation mask and optical flow information, the system performs dynamic mask fusion and optimization.
[0036] Within the initial mask region obtained from instance segmentation, a dynamic decision-making strategy using multiple thresholds is employed to identify moving objects. This strategy calculates the final dynamic threshold flow_3 by fusing two adaptive thresholds. The optical flow amplitude of each pixel is calculated based on the optical flow field, using the following formula: .
[0037] Optical flow statistics refer to the statistical characteristics calculated from the optical flow amplitudes of pixels within an instance mask region, including the median (flow_median), standard deviation (flow_std), and 75th percentile (flow_75th). It is directly related to optical flow information, which is the original pixel motion vector, while optical flow statistics are a statistical summary of the amplitudes of these vectors within the region.
[0038] Furthermore, optical flow amplitude is the magnitude of the motion vector of each pixel, and optical flow statistics are statistical features (such as median, standard deviation, etc.) calculated from the optical flow amplitude of all pixels within the instance mask region, used to represent the overall motion of the region; the statistics are calculated based on the optical flow amplitude.
[0039] Optical flow information is the complete motion vector field output by the PWC-Net model, containing the displacement components (Dx, Dy) of each pixel in the horizontal and vertical directions; while the optical flow magnitude is calculated using the formula... The single numerical value calculated from these vectors represents the intensity of pixel motion, but does not include directional information. Simply put, the optical flow amplitude originates from optical flow information, and it is the basic data for calculating optical flow statistics (such as median, standard deviation, etc.) during subsequent dynamic analysis.
[0040] The relationship between optical flow statistics and flow_3 includes: flow_3 is calculated from optical flow statistics, as shown in the following formula: flow_3 = (flow_1 + flow_2) / 2, where flow_1 = flow_median + flow_std, and flow_2 = flow_75th. Therefore, flow_3 is an adaptive threshold based on optical flow statistics.
[0041] Specifically, step 3, based on the motion vectors of pixels in each adjacent frame image, identifies moving objects within the initial mask region obtained from instance segmentation using a dynamic multi-threshold determination strategy, specifically including: Within the initial mask region obtained from instance segmentation, a dynamic decision-making strategy using multiple thresholds is employed to identify moving objects. This dynamic decision-making strategy calculates the final dynamic threshold flow_3 by fusing two adaptive thresholds. The final dynamic threshold flow_3 is calculated by fusing two intermediate thresholds flow_1 and flow_2, and its specific expression is as follows:
[0042] The two intermediate thresholds are calculated as follows:
[0043] In the above formula, flow_median represents the median of the optical flow amplitude of all pixels within the instance mask region, reflecting the central trend of motion in that region. flow_std represents the standard deviation of the optical flow amplitude, used to measure the dispersion of motion amplitude. flow_75th represents the 75th percentile of the optical flow amplitude, used to describe the distribution level of higher amplitudes during motion. To prevent extreme thresholding, the final calculated flow_3 value is restricted to a closed interval of [0.3, 2.0].
[0044] Furthermore, the temporal filtering mechanism employs an exponentially weighted moving average method to temporally smooth the probability results of dynamic discrimination, thereby suppressing dynamic misjudgments caused by abnormal single-frame optical flow estimation or instantaneous occlusion, and improving the stability and consistency of the dynamic mask in the time dimension.
[0045] Specifically, for each pixel in the image sequence, its dynamic probability obtained based on optical flow consistency analysis is used to construct a time series, and then smoothed according to the following formula:
[0046] in, This indicates the smoothing probability of the pixel in the previous frame. The smoothing factor controls the weighting of the current observation and historical information, and its value ranges from (0,1); the smoothed probability. Used to generate a final, stable dynamic object mask.
[0047] In a specific embodiment provided by the present invention, Figure 5 The specific example shows the instance segmentation mask effect extracted by the OpenSeeD module in a real scene, where different colored masks correspond to different dynamic objects in the image (such as pedestrians, moving chairs, etc.).
[0048] Figure 6A visualization of the PWC-Net optical flow estimation results is provided, which uses color coding to intuitively show the motion direction and amplitude of each pixel, where warm colors represent larger motion amplitudes and cool colors represent smaller motion amplitudes.
[0049] Specifically, the motion state of an object is determined by comparing the relationship between optical flow statistics and the final dynamic threshold flow_3.
[0050] In a specific embodiment of the present invention, the motion state of an object is determined by comparing the optical flow statistics (such as the median optical flow amplitude) of the instance mask region with flow_3. If the statistics exceed flow_3, the object is determined to be dynamic; otherwise, it is static. This multi-threshold strategy can adapt to changes in optical flow and improve the accuracy of dynamic recognition.
[0051] Specifically, after obtaining the initial mask, a temporal filtering mechanism is introduced to smooth the optical flow information of multiple consecutive frames, including: The probability value of each pixel being judged as dynamic in each frame is taken as a temporal sequence, and the dynamic probability of pixel i in the current frame t is... The dynamic probability sequence is smoothed using an exponentially weighted moving average algorithm, calculated according to the following formula:
[0052] in, This indicates the smoothing probability of the pixel in the previous frame. Let be the smoothing factor, and be the smoothed probability. Used to generate dynamic masks.
[0053] Specifically, a dynamic environment visual map construction system based on OpenSeeD includes: Image acquisition module: used to acquire a continuous RGB image sequence of dynamic objects; Dynamic perception module: used to perform instance segmentation and optical flow estimation on continuous RGB image sequences to obtain initial mask and optical flow information; Fusion and optimization module: used to fuse and optimize the initial mask and optical flow information, and to perform dynamic object recognition on the optimized fusion result to obtain a dynamic mask; Map building module: Used to remove dynamic feature points from dynamic masks, retain static feature points in static areas, and input the static feature points into the ORB-SLAM3 library to complete pose estimation and map building.
[0054] In another specific embodiment of the present invention, in order to verify the effectiveness of dynamic feature removal, Figure 7 and Figure 8The results of feature point removal are shown in single-person and multi-person scenarios, with green dots representing retained static feature points. It is clear that the system can accurately identify dynamic objects and effectively remove their feature points, while fully preserving static background features, providing high-quality input for subsequent SLAM processes.
[0055] To verify the effectiveness of this invention, this section presents an experimental evaluation and performance demonstration of the OP-SLAM system. The experiments used two publicly available datasets, TUM and Bonn, to evaluate the system's tracking capabilities. The results show that OP-SLAM achieves a competitive performance level compared to advanced SLAM systems such as ORB-SLAM3, DynaSLAM, and Refusion. The experimental program ran on a computing platform equipped with an NVIDIA GeForce RTX 4060 graphics card and Ubuntu 22.04 as the operating system.
[0056] This experiment uses the TUM RGB-D public dataset released by the Computer Vision Group of the Technical University of Munich. This dataset is specifically designed to evaluate the performance of visual odometry and SLAM systems and is widely regarded as an important benchmark in related research. To examine the accuracy and robustness of the OP-SLAM system in dynamic environments, this embodiment focuses on eight sequences from the dynamic object category in this dataset. The first four sequences represent high-dynamic scenes, while the remaining four represent low-dynamic scenes, complementing each other. In SLAM research, trajectory estimation error is typically quantified using two main metrics: absolute trajectory error, which directly measures the global consistency between the estimated trajectory and the true trajectory; and relative pose error, which reflects the gradual accumulation of rotational and translational drift during system operation. This embodiment uses ATE as the core evaluation metric to assess the performance improvement of OP-SLAM compared to its baseline system. This embodiment compares OP-SLAM with ORB-SLAM3 and other SLAM algorithms on the aforementioned eight dynamic sequences, and the relevant results are summarized in Table 1. Experiments show that introducing optical flow and semantic segmentation algorithms into the traditional SLAM architecture can significantly improve the system's positioning accuracy in highly dynamic environments; however, the performance improvement is relatively limited in scenarios with lower dynamic levels. Figure 4 Furthermore, the accuracy advantages of OP-SLAM in dynamic environments are visually demonstrated through ATE trajectory comparison.
[0057] Table 1. Comparison of absolute trajectory errors of OP-SLAM with other SLAM algorithms on the TUM dataset.
[0058] Furthermore, to verify the effectiveness of the improvements made to each module in this study, an ablation experiment was designed in this embodiment. The experiment used the classic ORB-SLAM3 system as the baseline model, and then added the OpenSeeD module and the PWC-Net module sequentially. The results are shown in Table 2, where O represents the addition of the OpenSeeD module and P represents the addition of the PWC-Net module. By comparing their localization accuracy in dynamic scenes, the independent contribution of each module and the superiority of the final OP-SLAM scheme were demonstrated. Table 6 shows that when PWC-Net (P) is used alone, it relies solely on optical flow estimation to determine dynamic regions. Due to the lack of semantic constraints, it is prone to misjudging static objects caused by camera motion as dynamic, and may also miss low-speed or low-texture dynamic objects, leading to the incorrect exclusion of effective static features and limited performance improvement. However, after introducing OpenSeeD (O), O initially locates high-probability dynamic regions (such as pedestrians and vehicles) through semantic segmentation, significantly narrowing the judgment range of P and reducing static interference; simultaneously, P corrects O's misjudgment of "semantically dynamic but actually static" regions (such as stationary people) through optical flow verification. The two form a complementary mechanism of "semantic screening - motion verification", which effectively reduces dynamic judgment error and thus significantly improves overall performance.
[0059] Table 2 Comparison of absolute trajectory errors of OP-SLAM using different methods on the TUM dataset
[0060] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0061] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for constructing dynamic environment visual maps based on OpenSeeD, characterized in that, include: Step 1: Obtain a continuous RGB image sequence of the dynamic object; Step 2: Perform instance segmentation and optical flow estimation on the continuous RGB image sequence to obtain the initial mask and optical flow information; Step 3: Fuse and optimize the initial mask and optical flow information, and perform dynamic object recognition on the optimized fusion result to obtain a dynamic mask; Step 4: Remove dynamic feature points from the dynamic mask, retain static feature points in the static region, and input the static feature points into the ORB-SLAM3 library to complete pose estimation and map construction.
2. The method for constructing a dynamic environment visual map based on OpenSeeD according to claim 1, characterized in that, The instance segmentation process in step 2 includes: using the OpenSeeD model based on the Transformer architecture to perform end-to-end instance segmentation on the continuous RGB image sequence, and retaining instance masks with confidence scores higher than 0.5 through non-maximum suppression to obtain the initial mask.
3. The method for constructing a dynamic environment visual map based on OpenSeeD according to claim 2, characterized in that, In step 2, the optical flow estimation uses the PWC-Net model to calculate the dense optical flow field between adjacent frames. The PWC-Net model constructs an image pyramid, iteratively calculates the cost volume and image warping, and outputs the motion vector of each pixel in the adjacent frame to obtain optical flow information.
4. The method for constructing a dynamic environment visual map based on OpenSeeD according to claim 3, characterized in that, Step 3, based on the motion vectors of pixels in each adjacent frame, identifies moving objects within the initial mask region obtained from instance segmentation using a dynamic multi-threshold judgment strategy. Specifically, this includes: The final dynamic threshold flow_3 is calculated by fusing two intermediate thresholds flow_1 and flow_2, as shown in the following expression: ； The two intermediate thresholds are calculated as follows: ； ； Wherein, flow_median represents the median of the optical flow amplitude of all pixels within the instance mask region; flow_std represents the standard deviation of the optical flow amplitude; and flow_75th represents the 75th percentile of the optical flow amplitude.
5. The method for constructing a dynamic environment visual map based on OpenSeeD according to claim 4, characterized in that, The motion state of an object is determined by comparing the relationship between optical flow statistics and the final dynamic threshold flow_3.
6. The method for constructing a dynamic environment visual map based on OpenSeeD according to claim 3, characterized in that, After obtaining the initial mask, a temporal filtering mechanism is introduced to smooth the optical flow information of multiple consecutive frames, including: The probability value of each pixel being judged as dynamic in each frame is taken as a temporal sequence, and the dynamic probability of pixel i in the current frame t is... The dynamic probability sequence is smoothed using an exponentially weighted moving average algorithm, calculated according to the following formula: in, This indicates the smoothing probability of the pixel in the previous frame. Let be the smoothing factor, and be the smoothed probability. Used to generate dynamic masks.
7. A dynamic environment visual map construction system based on OpenSeeD, characterized in that, include: Image acquisition module: used to acquire a continuous RGB image sequence of dynamic objects; Dynamic perception module: used to perform instance segmentation and optical flow estimation on continuous RGB image sequences to obtain initial mask and optical flow information; Fusion and optimization module: used to fuse and optimize the initial mask and optical flow information, and to perform dynamic object recognition on the optimized fusion result to obtain a dynamic mask; Map building module: Used to remove dynamic feature points from dynamic masks, retain static feature points in static areas, and input the static feature points into the ORB-SLAM3 library to complete pose estimation and map building.