An optical flow estimation method based on occluded object detection and motion continuity

By introducing occlusion object detection and motion continuity processing in the optical flow estimation method, the feature distortion and similarity evaluation module are used to detect occlusion points, and the matching position is optimized through the correlation optimization module, the existing optical flow estimation method is solved in complex scenarios and The problem of low estimation accuracy of occlusion point motion under harsh conditions achieves higher optical flow estimation accuracy and robustness.

CN117495919BActive Publication Date: 2024-11-29SHANGHAI INST OF MICROSYSTEM & INFORMATION TECH CHINESE ACAD OF SCI
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202311561532.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-22
Publication Date
2024-11-29
Estimated Expiration
2043-11-22

AI Technical Summary

Technical Problem

The existing optical flow estimation methods have low accuracy when dealing with occlusion points, especially in complex scenarios and harsh conditions, making it difficult to effectively estimate the movement of occlusion points.

Method used

The optical flow estimation method based on occlusion object detection and motion continuity is adopted to improve the accuracy and robustness of optical flow estimation through steps such as feature extraction, matching, occlusion point estimation and correlation optimization. Specifically, it includes feature distortion module, similarity evaluation module and occlusion estimation module. It uses multi-factor feature similarity judgment function to perform occlusion detection, and optimizes the matching position through Kalman filtering and other methods.

Benefits of technology

The accuracy and robustness of optical flow estimation in complex scenarios and harsh conditions are improved, especially the stability and reliability of optical flow estimation in the occlusion part, reduce the wrong matching relationship and improve the overall accuracy of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117495919B_ABST
    Figure CN117495919B_ABST
Patent Text Reader

Abstract

The present invention relates to an optical flow estimation method based on occluded object detection and motion continuity, comprising: receiving two consecutive frames of images; inputting the two consecutive frames of images into an optical flow estimation model to obtain an optical flow estimation value; wherein the optical flow estimation model comprises: a feature extraction part extracting image features and context features of the two consecutive frames of images; a matching part calculating relevant pixel point matching based on the image features of the two consecutive frames of images to obtain a 4D cost body; an occlusion point estimation part performing feature similarity judgment function processing on the 4D cost body and obtaining occlusion points through a convolutional network; an associated optimization part estimating a potential matching position based on the occlusion point, and calculating and comparing and integrating with the position given by the 4D cost body to obtain an optimized 4D cost body; an iterative optimization part iteratively optimizing based on the optimized 4D cost body and the context features to obtain an optical flow estimation value. The present invention can improve the accuracy of optical flow estimation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an optical flow estimation method based on occluded object detection and motion continuity. Background Art

[0002] Currently, optical flow algorithms include supervised and unsupervised modes. Supervised methods require labeled data, that is, the motion information of each pixel in the optical flow field needs to be manually labeled. Supervised methods are divided into feature extraction, cost volume calculation, and iterative optimization. Unsupervised methods do not require labeled data and can train models by learning the spatiotemporal information in images. The difference between the two is that unsupervised methods follow some hand-designed feature principles, including natural principles such as photometric consistency while optimizing the loss function, while supervised methods usually start with optical flow prediction and then learn to generate optical flow values ​​close to the real annotations by learning data features. Therefore, supervised method models consume more computing resources to obtain better performance.

[0003] In 2021, Shihao Jiang, Dylan Campbell, Yao Lu, Hongdong Li, and Richard Hartle published a paper on the 2021 IEEE / CVF International Conference on Computer Vision (ICCV).<Learning to Estimate Hidden Motions with Global MotionAggregation> In the paper, a definition of occluded points is given. They consider occluded points to be points that are imaged in the reference frame but not in the next frame. They believe that estimating the motion of these points is extremely difficult, especially in a two-frame setting. Their previous work relied on CNN to learn occlusions, without much success, or required multiple frames to infer occlusions using temporal smoothing. They introduced a global motion aggregation module, a Transformer model-based method that finds long-range correlations between pixels in the first image and performs global aggregation on the corresponding motion features. However, this paper only models self-similarity at the image feature level, enhances feature robustness, ensures correct pairwise matching between unoccluded points, and reduces the adverse effects of occlusion. However, most feature matching pairs of occluded points do not exist (missing or occluded behind objects), which leaves much room for improvement in the correction method for occluded point matching (cost volume calculation). Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an optical flow estimation method based on occluded object detection and motion continuity, which can improve the accuracy of optical flow estimation.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide an optical flow estimation method based on occluded object detection and motion continuity, comprising the following steps:

[0006] Receive two consecutive frames of images;

[0007] Inputting the two consecutive frames of images into an optical flow estimation model to obtain an optical flow estimation value;

[0008] Wherein, the optical flow estimation model includes:

[0009] A feature extraction part, used for extracting image features and context features of the two consecutive frames of images;

[0010] A matching part, used for calculating the matching of relevant pixels based on the image features of the two consecutive frames of images to obtain a 4D cost volume;

[0011] The occlusion point estimation part is used to perform feature similarity judgment function processing on the 4D cost volume and obtain the occlusion point through a convolutional network;

[0012] The association optimization part is used to estimate the potential matching position according to the occlusion point, and compare and integrate it with the position calculation given by the 4D cost volume to obtain an optimized 4D cost volume;

[0013] The iterative optimization part is used to divide the optimized 4D cost volume into two 2D features and generate an initial optical flow value. The context feature and the optimized 4D cost volume are used as intermediate features of iterative optimization to perform iterative optimization and obtain the optical flow estimation value.

[0014] The occlusion point estimation part includes:

[0015] The feature distortion module is used to distort the image feature f2 of the latter frame of the two frames of images back to the image feature of the previous frame of the two frames according to the distortion function to obtain the image feature f1 w ;

[0016] Similarity judgment module, used to find image feature f1 based on image structure similarity judgment function w The part that is not similar to the image feature f1 of the previous frame in the two frames is used to obtain the potential occlusion point;

[0017] The occlusion estimation module is used to input the potential occlusion point into an occlusion point estimation model to obtain an occlusion point.

[0018] The image structure similarity judgment function is: Among them, MFS represents the similarity value, b(f1 w ,f1) is the image feature f1 wThe brightness comparison part of the image feature f1 is expressed as: c(f1 w ,f1) is the image feature f1 w The contrast comparison part with the image feature f1 is expressed as: s(f1 w ,f1) is the image feature f1 w The structural similarity comparison part with the image feature f1 is expressed as: in, and Represents image features f1 w and the feature average gray value of image feature f1, and Represents image features f1 w and the standard deviation of image feature f1, Represents image feature f1 w and the covariance between pixels of image feature f1.

[0019] The occlusion point estimation model is composed of multiple convolutional layers.

[0020] When the association optimization part estimates the potential matching position according to the occlusion point, the potential matching position estimated by the position and speed state of the current pixel point is specifically: the design state space is Where (x, y) represents the position coordinates of the occlusion point. Represents the motion state of the occlusion point position coordinates (x, y); Estimate the potential matching position, where P t+1 represents the potential matching position, P t is the position of the occlusion point at the current moment, V t is the motion state of the occlusion point position coordinates at the current moment, A is the displacement increment matrix at the current moment, ε represents the 4D cost volume position matching information, and g t () indicates that z t As the Kalman gain function of the measured disturbance value r, z t Indicates the inaccurate flow estimate of the current location given by the 4D cost volume similarity matching, g t represents the Kalman gain at the current moment, p t represents the motion noise disturbance value predicted at the current moment, and r is the measured disturbance value.

[0021] The association optimization part calculates, compares and integrates the potential matching position with the position given by the 4D cost volume, which means that the pixel point closest to the position given by the 4D cost volume within the eight neighborhoods of the potential matching position replaces the position given by the 4D cost volume to obtain an optimized 4D cost volume.

[0022] The technical solution adopted by the present invention to solve its technical problem is: to provide an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned optical flow estimation method based on occluded object detection and motion continuity when executing the computer program.

[0023] The technical solution adopted by the present invention to solve its technical problem is: providing a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the optical flow estimation method based on occluded object detection and motion continuity are implemented.

[0024] Beneficial Effects

[0025] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art:

[0026] The present invention is based on an end-to-end optical flow estimation framework and an occlusion detection module based on a multi-factor feature similarity judgment function. The module uses the feature similarity judgment function to summarize the mean, variance and covariance of the feature maps of the distorted first frame and the second frame, thereby improving the accuracy and robustness of optical flow estimation in complex scenes and harsh conditions (such as illumination blur, motion blur, and gamma rays), and improving the stability and reliability of optical flow estimation for occluded parts.

[0027] The present invention proposes a strategy for handling occlusion point matching based on motion continuity. Matching updates are performed after the occlusion point is detected based on motion information. Since the matching point of the occlusion point cannot be found directly, the algorithm predicts its potential position based on the motion continuity of the dynamic principle, thereby updating the matching pair, improving the calculation of the cost body, reducing erroneous matching relationships, and improving the algorithm accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of an optical flow estimation method based on occluded object detection and motion continuity according to a first embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of an occlusion estimation part in a first embodiment of the present invention;

[0030] Figure 3 is a rendering of the occlusion motion state estimation model in the first embodiment of the present invention;

[0031] Figure 4 is an example diagram of a test image in an embodiment of the present invention;

[0032] Figure 5 It is an output optical flow estimation result diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0034] The first embodiment of the present invention relates to an optical flow estimation method based on occluded object detection and motion continuity. The method adopts an occlusion estimation part and an association optimization part. The occlusion estimation part improves the accuracy and robustness of optical flow estimation in complex scenes and harsh conditions (such as illumination blur, motion blur, gamma rays), and improves the stability and reliability of optical flow estimation for occluded parts; the association optimization part improves the calculation of the cost volume, reduces erroneous matching relationships, and improves the accuracy of the algorithm. Figure 1 As shown, the specific steps include:

[0035] Step 1: Receive two consecutive frames of images.

[0036] Step 2: input the two consecutive frames of images into an optical flow estimation model to obtain an optical flow estimation value.

[0037] Among them, the optical flow estimation model includes:

[0038] The feature extraction part is used to extract the image features and context features of the two consecutive frames of images; the feature extraction part in this embodiment includes an image feature extraction module and a context feature extraction module. The image feature extraction module extracts image features from the two frames of images respectively to obtain the image feature f1 of the first frame of image and the image feature f2 of the second frame of image. The context feature extraction module is used to extract the context feature f1 of the two frames of image. c The feature extraction modules used here include but are not limited to ResNet, Vision Transformer, Swin Transformer, FPN (Feature Pyramid Network, feature pyramid) and other feature extractors.

[0039] The matching part is used to calculate the matching of related pixels based on the image features of the two consecutive frames of images, that is, to calculate the matching of related pixels based on the image features f1 of the first frame of image and the image features f2 of the second frame of image to obtain a 4D cost volume.

[0040] The occlusion point estimation part is used to perform feature similarity judgment function processing on the 4D cost volume and obtain the occlusion point through a convolutional network. Figure 2 As shown, the occlusion point estimation part of this embodiment includes:

[0041] The feature distortion module is used to distort the image feature f2 of the second frame image in the two frames of images back to the image feature of the first frame image according to the distortion function to obtain the image feature f1 w ;

[0042] Similarity judgment module, used to find image feature f1 based on image structure similarity judgment function w The part that is not similar to the image feature f1 of the previous frame in the two frames is used to obtain the potential occlusion point; wherein the image structure similarity judgment function is: MFS represents the similarity value, and its value range is [0,1]. The larger the value, the more similar the pixel structure of the two images is. w ,f1) is the image feature f1 w The brightness comparison part of the image feature f1 is expressed as: c(f1 w ,f1) is the image feature f1 w The contrast comparison part with the image feature f1 is expressed as: s(f1 w ,f1) is the image feature f1 w The structural similarity comparison part with the image feature f1 is expressed as: in, and Represents image features f1 w and the feature average gray value of image feature f1, and Represents image features f1 w and the standard deviation of image feature f1, Represents image feature f1 w and the covariance between pixels of image feature f1.

[0043] The present invention is based on an end-to-end optical flow estimation framework and an occlusion detection module based on a multi-factor feature similarity judgment function. The module uses the feature similarity judgment function to summarize the mean, variance and covariance of the feature maps of the distorted first frame and the second frame, thereby improving the accuracy and robustness of optical flow estimation in complex scenes and harsh conditions (such as illumination blur, motion blur, and gamma rays), and improving the stability and reliability of optical flow estimation for occluded parts.

[0044] The occlusion estimation module is used to input the potential occlusion point into the occlusion point estimation model to obtain the occlusion point. The occlusion point estimation model is composed of multiple convolutions, and finally the softmax function is used to obtain the probability that each point in the 4D cost volume is an occlusion point, and the occlusion point p is obtained according to the probability. occ .

[0045] The associated optimization part is used to occ The potential matching position is estimated, and compared and integrated with the position calculation given by the 4D cost volume to obtain an optimized 4D cost volume.

[0046] Among them, according to the occlusion point p occ When estimating the potential matching position, Kalman filtering and the assumption that the movement of the same object is continuous are used. Specifically, the design state space is Where (x, y) represents the position coordinates of the occlusion point. Represents the motion state of the occlusion point position coordinates (x, y); Estimate the potential matching position, where P t+1 represents the potential matching position, P t is the position of the occlusion point at the current moment, V t is the motion state of the occlusion point position coordinate at the current moment - speed (optical flow value), A is the displacement increment matrix at the current moment, expressed as: ε x and ε y represents the displacement increment of the x-direction speed and the y-direction speed at the current moment, ε represents the 4D cost volume position matching information, and g t () indicates that z t As the Kalman gain function of the measured disturbance value r, z t Indicates the inaccurate flow estimate of the current location given by the 4D cost volume similarity matching, g t represents the Kalman gain at the current moment, p t represents the motion noise disturbance value predicted at the current moment, and r is the measured disturbance value.

[0047] After the potential matching position estimation is used in the cost volume calculation model, it is used in conjunction with the occlusion estimation module to correct the erroneous cost volume matching generated by the extracted occlusion points according to the motion model to obtain a more accurate matching value. The specific update process is as follows Figure 3 As shown: The occlusion estimation module finds the occluded point x1, and the matching point given by the 4D cost volume is x2, while the actual matching point has been blocked. The potential matching position is estimated to be x′2. Since the estimated position coordinates are not necessarily integer points, the pixel point closest to x2 in its eight-neighborhood is selected. The matching point given by the 4D cost volume is replaced by x2, thereby obtaining a new optimized 4D cost volume.

[0048] The present invention proposes a strategy for handling occlusion point matching based on motion continuity. Matching updates are performed after the occlusion point is detected based on motion information. Since the matching point of the occlusion point cannot be found directly, the algorithm predicts its potential position based on the motion continuity of the dynamic principle, thereby updating the matching pair, improving the calculation of the cost body, reducing erroneous matching relationships, and improving the algorithm accuracy.

[0049] The iterative optimization part is used to divide the optimized 4D cost volume into two 2D features and generate an initial optical flow value. The context feature and the optimized 4D cost volume are used as intermediate features for iterative optimization. 10-15w iterative optimizations are performed to obtain the optical flow estimation value.

[0050] The optical flow estimation model in this embodiment may use L1 loss to supervise all flow predictions, including intermediate and final predictions, which includes an exponentially increasing weight of 0.9.

[0051] Taking the application implementation of the method of this embodiment on the commonly used standard optical flow dataset MPI-Sintel as an example, the implementation on the driving scene dataset KITTI proposed by Karlsruhe Institute of Technology and Chicago Toyota Institute of Technology is similar, and its specific operation process is as follows:

[0052] Step 1: Dataset Introduction

[0053] MPI-Sintel dataset: The MPI-Sintel dataset is a synthetic dataset based on animated movies, containing many large motions, each with up to 400 pixels. It contains a total of 1628 frames of size 1024×436, of which 1064 frames are used for training and 564 frames are used for testing. It has two versions: clean and final. Clean contains realistic lighting and reflections. Finally, rendering effects such as motion, defocus blur, and atmospheric effects are added. MPI-Sintel uses EPE as a metric to measure the error distribution of occlusion and large motions through different thresholds.

[0054] Step 2: Preprocess the dataset

[0055] 2.1) Input the original RGB image with a resolution of 1024×436. Set the division ratio of the training set and the validation set (such as 8:2), obtain the training set and the validation set. Since the test set has been divided in advance, it is no longer divided. The image input to the network is cropped to 512×384. Reasons and methods for cropping: In optical flow estimation, the image is cropped to 512×384 in order to reduce the amount of calculation and improve the efficiency of the algorithm. The width of the image is scaled proportionally to 512 by using width cropping. This method is suitable for situations where the image has a large height and a small width. Due to limited computing resources and appropriate up and down sampling methods, its accuracy can be guaranteed not to be greatly damaged.

[0056] 2.2) Before inputting into the training network, two consecutive frames of images need to be divided into a group and used as input in groups.

[0057] Step 3: Construct a network framework based on occluded object detection and motion continuity in this embodiment, set relevant parameters for training the model, train and obtain the best training model, and use the final trained model as the optical flow estimation model.

[0058] 3.1) The network of panoptic segmentation in this embodiment mainly includes Swin Transformer as the backbone network, feature pyramid, occlusion estimation model, motion joint optimization module and iterative optimization module. The backbone network consists of two paths, one branch extracts the image features of the frame group, and the other extracts the context features of the first frame image of the frame group. The output of each branch is added to the iterative optimization part accordingly as the iterative body and intermediate optimization input.

[0059] 3.2) Optimizer and training parameter settings: The Adam optimizer is used during training, with an initial learning rate of 0.00002 and a learning rate decay of 0.00005. The total training iterations are 100,000-150,000 times. The optimal model parameters learned during training are saved for subsequent optical flow prediction.

[0060] Step 4: Send the test RGB image data into the optical flow estimation model and output the optical flow estimation result.

[0061] The test image group (see Figure 4 ) is fed into the optical flow estimation model, the prediction result is output, and the inferred result is colored by a pre-set color palette to obtain a visual optical flow estimation result (see Figure 5 ).

[0062] Step 5: Camera deployment, real-world scenario testing

[0063] The video frames captured by the external camera are extracted, and the RGB image groups are transferred to the GPU in groups of two frames. The input images are then sent to the model for inference in the GPU, and the results are visualized and input to the display.

[0064] It is not difficult to find that the present invention addresses the occlusion problem in optical flow estimation by detecting the occluded area method and considering the basic principle of motion continuity, modeling the motion characteristics of the occluded points, and reasonably correcting the association of the occluded points by ignoring the feature similarity in the cost body, thereby improving the accuracy of optical flow estimation.

[0065] The second embodiment of the present invention relates to an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the optical flow estimation method based on occluded object detection and motion continuity of the first embodiment when executing the computer program.

[0066] The third embodiment of the present invention relates to a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the optical flow estimation method based on occluded object detection and motion continuity of the first embodiment are implemented.

[0067] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction method, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0071] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An optical flow estimation method based on occluded object detection and motion continuity, characterized in that: The following steps are involved: Receive two consecutive frames of images; Inputting the two consecutive frames of images into an optical flow estimation model to obtain an optical flow estimation value; Wherein, the optical flow estimation model includes: A feature extraction part, used to extract image features and context features of the two consecutive frames of images; A matching part, used for calculating the matching of relevant pixels based on the image features of the two consecutive frames of images to obtain a 4D cost volume; The occlusion point estimation part is used to perform feature similarity judgment function processing on the 4D cost volume and obtain the occlusion point through a convolutional network; the occlusion point estimation part includes: The feature distortion module is used to distort the image feature f2 of the latter frame of the two frames of images back to the image feature of the previous frame of the two frames according to the distortion function to obtain the image feature f1 w ; Similarity judgment module, used to find image feature f1 based on image structure similarity judgment function w The part that is not similar to the image feature f1 of the previous frame in the two frames is used to obtain the potential occlusion point; wherein the image structure similarity judgment function is: Among them, MFS represents the similarity value, b(f1 w ,f1) is the image feature f1 w The brightness comparison part of the image feature f1 is expressed as: c(f1 w ,f1) is the image feature f1 w The contrast comparison part with the image feature f1 is expressed as: s(f1 w ,f1) is the image feature f1 w The structural similarity comparison part with the image feature f1 is expressed as: in, and Represents image features f1 w and the feature average gray value of image feature f1, and Represents image features f1 w and the standard deviation of image feature f1, Represents image feature f1 w and the covariance between pixels of image feature f1; An occlusion estimation module, used for inputting the potential occlusion point into an occlusion point estimation model to obtain an occlusion point; The associated optimization part is used to estimate the potential matching position according to the occlusion point, and compare and integrate it with the position calculation given by the 4D cost volume to obtain an optimized 4D cost volume; wherein, when estimating the potential matching position according to the occlusion point, the potential matching position estimated by the position and speed state of the current pixel point is specifically: the design state space is Where (x, y) represents the position coordinates of the occlusion point. Represents the motion state of the occlusion point position coordinates (x, y); Estimate the potential matching position, where P t+1 represents the potential matching position, P t is the position of the occlusion point at the current moment, V t is the motion state of the occlusion point position coordinates at the current moment, A is the displacement increment matrix at the current moment, ε represents the 4D cost volume position matching information, and g t () indicates that z t As the Kalman gain function of the measured disturbance value r, z t Indicates the inaccurate flow estimate of the current location given by the 4D cost volume similarity matching, g t represents the Kalman gain at the current moment, p t represents the motion noise disturbance value predicted at the current moment, and r is the measured disturbance value; the position calculation comparison integration with the position given by the 4D cost body refers to replacing the position given by the 4D cost body with the pixel point closest to the position given by the 4D cost body in the eight neighborhoods of the potential matching position to obtain an optimized 4D cost body The iterative optimization part is used to divide the optimized 4D cost volume into two 2D features and generate an initial optical flow value. The context feature and the optimized 4D cost volume are used as intermediate features of iterative optimization to perform iterative optimization and obtain the optical flow estimation value.

2. The optical flow estimation method based on occluded object detection and motion continuity according to claim 1, characterized in that: The occlusion point estimation model is composed of multiple convolutional layers.

3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the optical flow estimation method based on occluded object detection and motion continuity as described in any one of claims 1-2 are implemented.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the optical flow estimation method based on occluded object detection and motion continuity as described in any one of claims 1-2 are implemented.

Citation Information

Patent Citations

  • In-depth extracting method of three-viewpoint stereoscopic video restrained by time-space domain

    CN103702098A

  • Multi-feature fusion multi-target tracking method based on Kalman filtering assistance

    CN109919981A

  • Visual odometer measuring method

    CN112233179A

  • Video frame insertion method and system based on full-to-multi-field transformation

    CN116546237A

  • Super resolution processing method and apparatus using luminance masking and luminance compensation

    JP2007329678A