Visual mileage measurement method and system for dynamic object suppression and medium
By introducing optical flow consistency detection and U-Net network into the visual odometer, identifying and removing dynamic areas, the problems of low accuracy and high computational complexity in dynamic scenarios are solved, and more efficient and reliable Camera position estimate.
Patent Information
- Application Number
- CN202411201208.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-29
AI Technical Summary
The existing visual odometer method is difficult to accurately separate static and dynamic objects in dynamic scenarios, resulting in the impact of camera pose estimation accuracy and high computational complexity, making it difficult to achieve real-time performance on resource-constrained devices.
A visual mileage measurement method for dynamic object suppression is designed, and the combination of matching networks, dynamic detection networks and pose estimation networks are used to identify and remove dynamic areas, thereby optimizing optical areas by using optical flow consistency detection and U-Net network architecture. Stream images to improve the accuracy and reliability of camera pose estimation.
This method accurately identifies and removes dynamic objects through the combination of optical flow consistency detection and dynamic detection network, reduces its negative impact on visual odometer, improves the accuracy and reliability of visual odometer measurement in dynamic environments, and reduces the Computing resource requirements.
Smart Images

Figure CN119090922A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a visual mileage measurement method, system and medium for dynamic object suppression. Background Art
[0002] Visual odometry (VO) refers to the process of calculating the camera motion trajectory by analyzing the visual data stream. It is widely used in many fields such as 3D reconstruction, autonomous driving, and virtual reality. At present, the research on monocular visual odometry based on deep learning has been relatively mature and has achieved excellent performance. However, visual odometry is generally based on static assumptions. In dynamic scenes, camera motion and dynamic objects in the scene jointly cause optical flow. Many existing visual odometry methods directly output optical flow maps as visual odometry, which may lead to feature point matching errors, thereby affecting the accurate estimation of camera pose. In order to deal with this problem, the mainstream solution is to use semantic information or motion information to identify the dynamic object area in the video, and then reduce the contribution weight of the dynamic object area in the pose update. The Chinese invention patent with application number 2019110573430 discloses a visual odometry method based on dynamic and static scene separation. Through inter-frame change analysis, the inconsistency of depth information in dynamic scenes and optical flow information are used to achieve dynamic and static separation of scenes. On this basis, more reliable static scene information is used to achieve camera pose estimation. However, the above method still has some problems: on the one hand, although the method can achieve dynamic and static scene separation through inter-frame change analysis, the detection and segmentation of dynamic objects may not be accurate enough in extremely dynamic or complex scenes, which affects the extraction of static scene information and the accuracy of camera pose estimation; on the other hand, the method relies on deep convolutional neural networks for depth estimation, camera pose estimation and dynamic object detection. These calculation processes may be relatively complex and require high computing resources. When running on resource-constrained devices, it may be difficult to ensure real-time performance. Therefore, it is necessary to redesign a visual odometer measurement method to further improve the accuracy and reliability of visual odometer measurement. Summary of the invention
[0003] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a visual mileage measurement method, system and medium with dynamic object suppression, so as to further improve the accuracy and reliability of visual mileage measurement in dynamic environment.
[0004] The purpose of the present invention can be achieved by the following technical solutions:
[0005] The present invention provides a visual mileage measurement method with dynamic object suppression, comprising the following steps:
[0006] Get two consecutive input image frames, input the visual odometry model with dynamic object suppression, and obtain the camera pose change, including camera rotation R and camera displacement T;
[0007] The dynamic object suppression visual odometer measurement model includes a matching network, a dynamic detection network and a pose estimation network connected in sequence. The specific process of the dynamic object suppression visual odometer measurement model processing input data is as follows:
[0008] Two consecutive input image frames I t and I t+1 Input the matching network to estimate the forward optical flow and backward optical flow
[0009] According to the forward optical flow and the backward optical flow Perform optical flow consistency detection, including forward optical flow consistency detection and backward optical flow consistency detection, and obtain the optical flow forward consistency according to the results of the forward optical flow consistency detection and the backward optical flow consistency detection. and optical flow backward consistency They are used to represent the pixels that disappear in the next frame and the pixels that appear in the next frame respectively;
[0010] The optical flow is forward consistent And the optical flow backward consistency Input the dynamic detection network to obtain the probability map It is used to reflect the probability that each pixel in the input image frame is a dynamic pixel. Convert to binary segmentation mask Through the binary segmentation mask Optimize the forward optical flow To remove the forward optical flow Dynamic area in
[0011] The binary segmentation mask Optimized forward optical flow And the internal reference layer is input into the pose estimation network to obtain the camera pose change
[0012] Furthermore, the matching network obtains the forward optical flow through GMFlow estimation and the backward optical flow
[0013] Furthermore, the process of forward optical flow consistency detection is: using the forward optical flow For the backward optical flow Perform deformation, and then transform the forward optical flow The forward optical flow is added to the deformed backward optical flow. If the sum exceeds the set threshold, the forward optical flow consistency detection result is inconsistent.
[0014] Furthermore, the calculation formula for setting the threshold is as follows:
[0015] threshold=α*flow_mag+β
[0016] Among them, threshold is the set threshold, α and β are threshold influence parameters, and flow_mag is the optical flow size.
[0017] Furthermore, the process of the backward optical flow consistency detection is: using the backward optical flow For the forward optical flow Perform deformation, and then transform the backward optical flow The forward optical flow after deformation is added. If the sum exceeds the set threshold, the consistency detection result of the backward optical flow is inconsistent.
[0018] Furthermore, the dynamic detection network is constructed based on a U-Net network architecture, including an encoder and a decoder, and a jump connection is used between the encoder and the decoder.
[0019] Furthermore, the encoder adopts a deep residual network ResNet34.
[0020] Furthermore, the intrinsic parameter layer is constructed based on camera intrinsic parameters, and the camera intrinsic parameters include focal length and principal point position.
[0021] The present invention also provides a visual mileage measurement system for dynamic object suppression, comprising an optical flow estimation module, a dynamic detection module and a posture estimation module connected in sequence;
[0022] The optical flow estimation module is used to estimate the optical flow of two consecutive input image frames I t and I t+1 Estimate the forward optical flow and backward optical flow And for the forward optical flow and the backward optical flow Perform optical flow consistency detection, including forward optical flow consistency detection and backward optical flow consistency detection, and obtain the optical flow forward consistency according to the results of the forward optical flow consistency detection and the backward optical flow consistency detection. and optical flow backward consistency They are used to represent the pixels that disappear in the next frame and the pixels that appear in the next frame respectively;
[0023] The dynamic detection module is used to detect the forward consistency of the optical flow And the optical flow backward consistency Generate Probability Map It is used to reflect the probability that each pixel in the input image frame is a dynamic pixel. Convert to binary segmentation mask Through the binary segmentation mask Optimize the forward optical flow To remove the forward optical flow Dynamic area in
[0024] The pose estimation module is used to segment the binary mask according to the Optimized forward optical flow And the internal reference layer to obtain the camera pose change It includes camera rotation R and camera displacement T.
[0025] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.
[0026] Compared with the prior art, the present invention has the following beneficial effects:
[0027] 1. The present invention proposes a visual odometer measurement method with dynamic object suppression, which is implemented by a visual odometer measurement model with dynamic object suppression including a matching network, a dynamic detection network and a pose estimation network. The specific process is as follows: first, two consecutive input image frames I t and I t+1 Input matching network to estimate the forward optical flow and backward optical flow The forward optical flow can reflect the pixel displacement from the current frame to the next frame, and the backward optical flow can reflect the pixel displacement from the current frame to the previous frame. The forward optical flow and the backward optical flow are the basis for subsequent motion analysis. and backward optical flow Perform optical flow consistency detection to obtain optical flow forward consistency and optical flow backward consistency They represent the pixels that disappear in the next frame and the pixels that appear in the next frame, which can fully reflect the state of the pixels in the image frame; then, the forward consistency of the optical flow and optical flow backward consistency Input the dynamic detection network to obtain the probability map It can reflect the probability that each pixel in the input image frame is a dynamic pixel, identify dynamic objects, and convert the probability map Convert to binary segmentation mask Through the binary segmentation mask Optimizing forward optical flow Forward optical flow can be removed The dynamic area in; Finally, the binary segmentation mask Optimized forward optical flow And the internal reference layer input pose estimation network to obtain the camera pose change The above method can reduce the negative impact of dynamic objects on visual odometry and further improve the accuracy and reliability of visual odometry in dynamic environments.
[0028] 2. In the present invention, the dynamic detection network is constructed based on the U-Net network architecture, including an encoder and a decoder. The encoder adopts the deep residual network ResNet34. U-Net allows the network to capture contextual information in the contraction path and restore image details in the expansion path, so as to accurately detect dynamic objects. ResNet34 can improve the depth of the network by introducing residual blocks and cross-layer connections, thereby enhancing the network's representation and feature extraction capabilities. At the same time, ResNet34 can effectively alleviate the problems of gradient disappearance and gradient explosion, and has high computational efficiency. Combining the precise positioning capability of U-Net with the deep feature extraction capability of ResNet-34 and using them for dynamic object detection can improve the dynamic object detection performance, thereby further optimizing the forward optical flow and improving the accuracy and reliability of visual mileage measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 Schematic diagram of the visual odometry model structure for dynamic object suppression;
[0030] Figure 2 Schematic diagram of optical flow consistency detection;
[0031] Figure 3 Schematic diagram of the output of each network of the visual odometry model for dynamic object suppression.
[0032] Among them, (3a) is the input image at time t, (3b) is the input image at time t+1, (3c) is the forward optical flow estimation of the matching network, (3d) is the backward optical flow estimation of the matching network, (3e) is the forward consistency detection of the optical flow, (3f) is the backward consistency detection of the optical flow, (3g) is the motion mask detected by the dynamic detection network, and (3h) is the visualization of motion segmentation on the image;
[0033] Figure 4 Schematic diagram of qualitative results of different models in challenging sequences.
[0034] Among them, (4a), (4b), (4c) and (4d) are the segmentation results of dynamic target detection of the method of the present invention, and (4e), (4f), (4g), (4h), (4i), (4j), (4k) and (41) are the VO test result trajectories of KITTI odometer sequences 00, 01, 02, 03, 05, 07, 09 and 10 respectively. DETAILED DESCRIPTION
[0035] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0036] Example 1
[0037] The paper TartanVO: A Generalizable Learning-based VO (W. Wang, Y. Hu, and S. Scherer. CoRL, volume 155 of Proceedings of Machine Learning Research, page 1761-1772. PMLR, 2020) designed a TartanVO two-stage network and proposed a method to achieve cross-camera / dataset generalization by directly incorporating camera intrinsic functions into the model, but in dynamic scenes, this method is easily disturbed by dynamic objects. This embodiment designs a three-stage network architecture based on the TartanVO two-stage network, provides a visual mileage measurement method with dynamic object suppression, captures the occlusion information caused by the motion of dynamic objects in the scene through optical flow consistency detection, and directly removes the dynamic optical flow caused by moving objects from the optical flow image. The process is as follows:
[0038] Get two consecutive input image frames and input them into the visual odometry model with dynamic object suppression (MoMaskVO) to obtain the camera pose change, including camera rotation R and camera displacement T.
[0039] like Figure 1 As shown in FIG. 1 , the visual odometer measurement model for dynamic object suppression includes a matching network, a dynamic detection network, and a pose estimation network connected in sequence. The specific process of the visual odometer measurement model for dynamic object suppression processing input data is as follows:
[0040] S1, two consecutive input image frames I t and I t+1 Input matching network, matching network obtains forward optical flow through GMFlow estimation and backward optical flow
[0041] S2, forward optical flow and backward optical flow Perform optical flow consistency detection to obtain optical flow forward consistency and optical flow backward consistency
[0042] Optical flow consistency detection identifies dynamic objects by detecting background changes caused by motion, including forward optical flow consistency detection and backward optical flow consistency detection. Figure 2 As shown, assuming P t is the image frame I t The pixel on P t+1 is the image frame I t+1 If the pixels on satisfy the following formula, the forward optical flow consistency test result is inconsistent:
[0043]
[0044] in, represents the optical flow from time t to time t+1, It means that the pixel at time t plus the forward optical flow finds the backward optical flow of the corresponding pixel in the next frame.
[0045] If the following formula is satisfied, the backward optical flow consistency test result is inconsistent:
[0046]
[0047] The above formula is a theoretical consistency test. In practice, optical flow is always inaccurate, so the specific process of forward optical flow consistency test is as follows: Use forward optical flow Backward Optical Flow Deform and generate a new optical flow field. Then the forward optical flow Theoretically, the forward optical flow and the deformed backward optical flow should be opposite in direction and equal in magnitude, and their sum should be close to zero. Since there is a certain error in practice, if the summation result exceeds the set threshold, there is occlusion in the area or the forward optical flow consistency detection result is inconsistent, and the corresponding pixel is represented by gray.
[0048] The calculation formula for setting the threshold is as follows:
[0049] threshold=α*flow_mag+β
[0050] Among them, threshold is the set threshold, α and β are threshold influence parameters, which can be 0.01 and 0.5 respectively, and flow_mag is the optical flow size.
[0051] Using forward optical flow Backward Optical Flow The specific process of deformation is as follows: establish a two-dimensional grid coordinate in the space of the target image, which represents the position of each pixel in the image. Adding to the original grid coordinates, the position of each pixel in image A in image B can be calculated. By sampling the backward optical flow at these new coordinate points through bilinear interpolation, the deformed backward optical flow can be obtained.
[0052] Similarly, the process of backward optical flow consistency detection is: using backward optical flow Forward optical flow Deform, and then transform the backward optical flow The forward optical flow after deformation is added. If the sum exceeds the set threshold, there is occlusion in the area or the backward optical flow consistency detection result is inconsistent, and the corresponding pixels are represented by red.
[0053] Optical flow forward consistency obtained through optical flow consistency detection It can determine which pixels will disappear in the next frame, and the backward consistency of optical flow It is possible to determine which pixels will appear in the next frame, and accordingly segment the optical flow map into dynamic and static areas. Some existing methods use optical flow to detect dynamic objects, but at corners, scene changes are more significant at faster speeds, and changes in optical flow are obvious in all pixels, causing the vast majority of pixels to be marked as dynamic, resulting in relatively large errors in self-motion estimation at corners. If semantic segmentation is used, it is not only slow, but it can only segment a limited number of categories and cannot generalize to unrecognized categories.
[0054] S3, forward consistency of optical flow and optical flow backward consistency Input the dynamic detection network to obtain the probability map It is used to reflect the probability that each pixel in the input image frame is a dynamic pixel. Convert to binary segmentation mask Through the binary segmentation mask Optimizing forward optical flow To remove the forward optical flow The dynamic area in .
[0055] If there are moving objects in the image, there will be inconsistencies in the forward optical flow consistency and corresponding inconsistencies in the backward optical flow consistency. Camera motion will also produce inconsistencies in the forward and backward optical flow consistency detection, but they are not as obvious as the inconsistencies of dynamic objects. Therefore, the forward and backward optical flow consistency can be input into the dynamic detection network to detect dynamic objects.
[0056] The dynamic detection network is built based on the U-Net network architecture, including an encoder and a decoder, and a jump connection is used between the encoder and the decoder. In a preferred embodiment, the encoder adopts the deep residual network ResNet34. U-Net allows the network to capture contextual information in the contraction path and restore image details in the expansion path, which can accurately detect dynamic objects; ResNet34 can increase the depth of the network by introducing residual blocks and cross-layer connections, thereby enhancing the network's representation and feature extraction capabilities. At the same time, ResNet34 can effectively alleviate the problems of gradient disappearance and gradient explosion, and has high computational efficiency; combining the precise positioning capability of U-Net with the deep feature extraction capability of ResNet-34 and using them for dynamic object detection can improve the dynamic object detection performance, thereby further optimizing the forward optical flow and improving the accuracy and reliability of visual mileage measurement.
[0057] S4, the binary segmentation mask Optimized forward optical flow And the internal reference layer input pose estimation network to obtain the camera pose change
[0058] The intrinsic parameter layer is built based on the camera intrinsic parameters, which include focal length and principal point position. For details, please refer to tartanvo.
[0059] The output of each network of the visual odometry model with dynamic object suppression is as follows: Figure 3 As shown, Figure 3 (3a) is the input image at time t, (3b) is the input image at time t+1, (3c) is the forward optical flow estimation of the matching network, (3d) is the backward optical flow estimation of the matching network, (3e) is the forward consistency detection of the optical flow, (3f) is the backward consistency detection of the optical flow, (3g) is the motion mask detected by the dynamic detection network, and (3h) is the visualization of motion segmentation on the image.
[0060] The above method can directly remove the dynamic optical flow caused by moving objects from the optical flow image by adding optical flow consistency detection, detecting background changes, capturing occlusion information caused by the movement of dynamic objects in the scene, and segmenting and processing through the dynamic detection network, thereby reducing the negative impact of dynamic objects on the visual odometer. This improvement enables the visual odometer system to estimate the camera's position more accurately and maintain high performance even in scenes with dynamic objects.
[0061] To verify the effectiveness of the above method, this embodiment first compares the inference time of TartanVO, DytanVO with 3 iterations, and MoMaskVO (Ours) proposed in the present invention on the KITTI odometry dataset. The experimental results are shown in Table 1.
[0062] Table 1 Comparison of inference time of different methods tested on the KITTI odometry dataset
[0063]
[0064] In Table 1, Flow time is the optical flow time, which indicates the reasoning time of optical flow estimation; Mask time is the mask time, which indicates the reasoning time of the dynamic detection network; Pose time is the posture time, which indicates the reasoning time of the posture estimation network; Toal is the total time. TartanVO and DytanVO both use PWC-Net, but due to the need to estimate the forward and backward optical flows, the present invention uses GMFlow, so the Flow time is longer than TartanVO and DytanVO, but the dynamic detection network of MoMaskVO only takes 7.5 milliseconds to calculate once, while the dynamic detection network of DytanVO takes 37 milliseconds to calculate once, and requires two calculations; the time for all three models to run the posture estimation network once is 5.2 milliseconds, but DytanVO needs to be calculated three times. As can be seen from Table 1, the total time of the present invention is significantly less than that of DytanVO, which is only 42.73% of DytanVO.
[0065] Secondly, this embodiment compares the accuracy of the above three methods tested on the KITTI odometer dataset, and the comparison results are shown in Table 2.
[0066] Table 2 Accuracy comparison of different methods tested on the KITTI odometry dataset
[0067]
[0068] In Table 2, 00-10 represent eleven sequences of the KITTI odometry dataset. The KITTI odometry benchmark measures the performance of visual odometry methods by evaluating trajectory subsequences with lengths ranging from 100 meters to 800 meters in increments of 100 meters. For each trajectory subsequence, the criterion reports two evaluation criteria: the average translation error and the rotation error. The average translation error (T err ): Expressed as a percentage, it measures the difference between the camera position estimated by the algorithm and the actual position. err): Expressed in degrees per 100 meters, it measures the difference between the camera rotation estimated by the algorithm and the actual rotation. These error indicators provide quantitative data for comparing the accuracy of different methods in estimating camera position and rotation. In Table 2, the best result is bold and the second best result is underlined. The data in Table 2 show that the accuracy of the visual odometer method for dynamic object suppression provided by the present invention is significantly better than DytanVO when tested on the KITTI odometer dataset.
[0069] Figure 4 Qualitative results of the three models on challenging sequences are shown. Figure 4 (4a), (4b), (4c) and (4d) show the segmentation results of dynamic target detection of the method of the present invention. It can be seen that even when there are multiple dynamic targets, high-speed vehicles, dynamic targets occupying a large area of the image, and cyclists traveling at a low speed in the image, the method of the present invention can still effectively identify and segment these dynamic targets, which proves the robustness of the method of the present invention in handling highly dynamic and complex scenes. Figure 4 (4e), (4f), (4g), (4h), (4i), (4j), (4k) and (4l) are the VO test result trajectories of KITTI odometer sequences 00, 01, 02, 03, 05, 07, 09 and 10 respectively, and GT is the standard trajectory. It can be seen that the accuracy of the method of the present invention (MoMaskVO) is better.
[0070] This embodiment also conducts experiments on the AirDOS-Shibuya dataset. AirDOS-Shibuya is a highly dynamic scene shot with a handheld camera and has a more complex motion pattern. This embodiment compares the method of the present invention with the most advanced model on this dataset, and the results are shown in Table 3.
[0071] Table 3 Accuracy comparison of different methods tested on the AirDOS-Shibuya dataset
[0072]
[0073] In Table 3, DROID-SLAM greatly surpasses existing traditional SLAM systems on many challenging benchmark datasets; AirDOS, VDO-SLAM and DynaSLAM are three SLAM methods that handle dynamic environmental interference. Among the visual odometry (VO) methods, DeepVO, Trainflow and CC perform poorly because these models are only trained on the KITTI dataset, and the complex motion patterns make it difficult for them to reason correctly. The model proposed in the present invention has significantly improved on TartanVO and achieved comparable results with DytanVO, especially achieving the best performance on Sequence II, Sequence V and Sequence VII. This result demonstrates the effectiveness and robustness of the method of the present invention in dealing with highly dynamic scenes.
[0074] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0075] Example 2
[0076] This embodiment provides a visual odometer system with dynamic object suppression, including an optical flow estimation module, a dynamic detection module, and a posture estimation module connected in sequence.
[0077] The optical flow estimation module is used to estimate the optical flow of two consecutive input image frames I t and I t+1 Estimate the forward optical flow and backward optical flow And for the forward optical flow and backward optical flow Perform optical flow consistency detection, including forward optical flow consistency detection and backward optical flow consistency detection, and obtain the forward consistency of optical flow according to the results of forward optical flow consistency detection and backward optical flow consistency detection. and optical flow backward consistency They are used to represent pixels that disappear in the next frame and pixels that appear in the next frame respectively.
[0078] The dynamic detection module is used to detect the forward consistency of the optical flow and optical flow backward consistency Generate Probability Map It is used to reflect the probability that each pixel in the input image frame is a dynamic pixel. Convert to binary segmentation mask Through the binary segmentation mask Optimizing forward optical flow To remove the forward optical flow The dynamic area in .
[0079] The pose estimation module is used to segment the mask according to the binary value. Optimized forward optical flow And the internal reference layer to obtain the camera pose change It includes camera rotation R and camera displacement T.
[0080] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.
Claims
1. A visual odometry method with dynamic object suppression, characterized in that: The following steps are involved: Get two consecutive input image frames, input the visual odometry model with dynamic object suppression, and obtain the camera pose change, including camera rotation R and camera displacement T; The dynamic object suppression visual odometer measurement model includes a matching network, a dynamic detection network and a pose estimation network connected in sequence. The specific process of the dynamic object suppression visual odometer measurement model processing input data is as follows: Two consecutive input image frames I t and I t+1 Input the matching network to estimate the forward optical flow and backward optical flow According to the forward optical flow and the backward optical flow Perform optical flow consistency detection, including forward optical flow consistency detection and backward optical flow consistency detection, and obtain the optical flow forward consistency according to the results of the forward optical flow consistency detection and the backward optical flow consistency detection. and optical flow backward consistency They are used to represent the pixels that disappear in the next frame and the pixels that appear in the next frame respectively; The optical flow is forward consistent And the optical flow backward consistency Input the dynamic detection network to obtain the probability map It is used to reflect the probability that each pixel in the input image frame is a dynamic pixel. Convert to binary segmentation mask Through the binary segmentation mask Optimize the forward optical flow To remove the forward optical flow Dynamic area in The binary segmentation mask Optimized forward optical flow And the internal reference layer is input into the pose estimation network to obtain the camera pose change 2. The method for visual odometry with dynamic object suppression according to claim 1, characterized in that: The matching network obtains the forward optical flow through GMFlow estimation and the backward optical flow 3. The method for visual odometry with dynamic object suppression according to claim 1, characterized in that: The process of forward optical flow consistency detection is: using the forward optical flow For the backward optical flow Perform deformation, and then transform the forward optical flow The forward optical flow is added to the deformed backward optical flow. If the sum exceeds the set threshold, the forward optical flow consistency detection result is inconsistent.
4. The method for visual odometer measurement with dynamic object suppression according to claim 3, characterized in that: The calculation formula for setting the threshold is as follows: theshold=α*flow_mag+β Among them, theshold is the set threshold, α and β are threshold influence parameters, and flow_mag is the optical flow size.
5. The method for visual odometry with dynamic object suppression according to claim 1, characterized in that: The process of backward optical flow consistency detection is: using the backward optical flow For the forward optical flow Perform deformation, and then transform the backward optical flow The forward optical flow after deformation is added to the original one. If the sum exceeds the set threshold, the consistency detection result of the backward optical flow is inconsistent.
6. The method for visual odometry with dynamic object suppression according to claim 1, characterized in that: The dynamic detection network is constructed based on the U-Net network architecture, including an encoder and a decoder, and a jump connection is adopted between the encoder and the decoder.
7. The method for visual odometry with dynamic object suppression according to claim 6, characterized in that: The encoder adopts the deep residual network ResNet34.
8. The method for visual odometry with dynamic object suppression according to claim 1, characterized in that: The intrinsic parameter layer is constructed based on the camera intrinsic parameters, and the camera intrinsic parameters include focal length and principal point position.
9. A visual odometry system with dynamic object suppression, characterized in that: It includes an optical flow estimation module, a dynamic detection module and a posture estimation module which are connected in sequence; The optical flow estimation module is used to estimate the optical flow of two consecutive input image frames I t and I t+1 Estimate the forward optical flow and backward optical flow And for the forward optical flow and the backward optical flow Perform optical flow consistency detection, including forward optical flow consistency detection and backward optical flow consistency detection, and obtain the optical flow forward consistency according to the results of the forward optical flow consistency detection and the backward optical flow consistency detection. and optical flow backward consistency They are used to represent the pixels that disappear in the next frame and the pixels that appear in the next frame respectively; The dynamic detection module is used to detect the forward consistency of the optical flow And the optical flow backward consistency Generate Probability Map It is used to reflect the probability that each pixel in the input image frame is a dynamic pixel. Convert to binary segmentation mask Through the binary segmentation mask Optimize the forward optical flow To remove the forward optical flow Dynamic area in The pose estimation module is used to segment the binary mask according to the Optimized forward optical flow And the internal reference layer to obtain the camera pose change It includes camera rotation R and camera displacement T.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Visual odometer method based on dynamic and static scene separation
CN110910447A
Non-local filter unsupervised optical flow estimation method based on deep learning
CN111462191A
Visual odometer measuring method
CN112233179A
Visual odometer implementation method based on dynamic visual sensor
CN117649431A
Monocular-vision-based detection of moving objects
EP4174770A1
Cited By
Dynamic scene new view angle synthesis method, system, equipment and medium
CN122265533A
A dynamic scene novel perspective synthesis method, system, device and medium
CN122265533B
