A method, apparatus, and device for optical flow calculation based on contrastive learning

By employing a contrastive learning-based optical flow computation method, which utilizes convolutional networks and momentum contrastive training, the data dependency and redundancy issues in existing optical flow computations are resolved. This approach achieves efficient optical flow computation, reduces the need for real datasets, and improves computational accuracy.

CN121837318BActive Publication Date: 2026-11-10SUZHOU IND PARK SERVICE OUTSOURCING VOCATIONAL COLLEGE (SUZHOU SERVICE OUTSOURCING TALENT TRAINING & TRAINING CENT)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511975453.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-12-25
Publication Date
2026-11-10
Estimated Expiration
2045-12-25

AI Technical Summary

Technical Problem

Existing optical flow calculation methods rely on small-scale artificially synthesized datasets, which suffer from overfitting problems. Furthermore, it is difficult to obtain optical flow label data in real-world scenarios, resulting in a lot of redundant information and a lack of effective filtering and purification network structures.

Method used

We employ a contrastive learning approach, using convolutional networks to extract image features, construct relevant volumes, perform background attention calculations and 2D motion vector extraction, and utilize momentum contrast training between online and target networks to achieve optical flow calculations, thereby reducing reliance on real datasets.

Benefits of technology

It overcomes the overfitting problem, requiring only one target image and one original image to perform optical flow calculations, thus improving calculation accuracy and reducing the need for real-world scene datasets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121837318B_ABST
    Figure CN121837318B_ABST
Patent Text Reader

Abstract

This application discloses an optical flow calculation method, apparatus, device, and readable storage medium based on contrastive learning, relating to the field of optical flow calculation technology. The method includes: firstly, extracting features from the original image and the target image using a convolutional network to obtain the original feature map, the background features of the original image, and the target feature map of the target image; then, constructing a correlation volume based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching; next, performing attention calculation on the background features to obtain background attention, and extracting 2D motion vectors based on the correlation volume; finally, inputting the background attention and the 2D motion vectors into an online network to obtain the optical flow calculation result. This method increases the accuracy of the optical flow calculation result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of optical flow computing technology, and more specifically to an optical flow computing method, apparatus, device, and readable storage medium based on contrastive learning. Background Technology

[0002] Optical flow computation aims to find the pixel-to-pixel motion relationships between images and has wide applications in tasks such as autonomous driving, visual odometry, video restoration, and scene reconstruction. Classical optical flow computation is considered an optimization problem. Since 2015, many end-to-end deep learning networks have emerged to directly compute optical flow values. In recent years, optical flow computation methods based on visual attention have become dominant.

[0003] Optical flow computation faces two main challenges: dataset dependency and correlation volume redundancy. Existing optical flow datasets are often small and artificially synthesized, such as Sintel's collection of images edited from a video, or FlyingChairs' rendering of 3D chair models onto images. This leads to overfitting in most existing optical flow computation methods, resulting in drooping performance on the KITTI dataset test sequences. To bridge the significant gap between synthetic and real-world data, more real-world scene datasets are needed. However, obtaining optical flow label data from real-world scenes is extremely difficult, requiring motion capture systems to track the movement of each pixel. Correlation volumes are crucial nodes in optical flow computation, recording the similarity relationships between pixel pairs, but they also contain a large amount of redundancy and erroneous information. Existing optical flow computation methods primarily focus on image feature extraction and correlation volume computation, with few network structures specifically designed for filtering and refining correlation volumes.

[0004] Therefore, there is an urgent need for an optical flow calculation method based on contrastive learning that can solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide an optical flow calculation method, apparatus, device, and readable storage medium based on contrastive learning. The optical flow calculation is achieved through a neural network. In this process, there is no need to artificially synthesize datasets, which overcomes the problem of overfitting. Moreover, optical flow calculation can be achieved based on only a target image and an original image, without the need to obtain a large amount of real scene datasets. This invention achieves optical flow calculation by overcoming the limitations of existing technologies.

[0006] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides an optical flow calculation method based on contrastive learning, the method comprising: The convolutional network is used to extract features from the original image and the target image, resulting in the original feature map, the background features of the original image, and the target feature map of the target image. A correlation volume is constructed based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching. Attention is calculated on background features to obtain background attention, and 2D motion vectors are extracted based on the relevant volume. Background attention and 2D motion vectors are input into an online network to obtain optical flow calculation results.

[0007] In some embodiments, background attention and 2D motion vectors are input into an online network to obtain optical flow calculation results, including: Based on the agent task, background attention and 2D motion vectors are transformed into sample pairs; The sample pairs are input into the encoding network for feature extraction to obtain the sample pair features; The optical flow calculation results are obtained by mapping the features of sample pairs to a new feature space through a multi-layer MLP and making predictions.

[0008] In some embodiments, the training method for the online network includes: Obtain the preset occlusion image; Momentum comparison is performed based on sample background attention, sample 2D motion vectors, and preset occlusion maps to obtain optical flow increments; sample background attention and sample 2D motion vectors are obtained based on the original sample image and the sample target image. Adjusting network parameters of the online network based on optical flow increments.

[0009] In some embodiments, momentum comparison is performed based on sample background attention, sample 2D motion vectors, and a preset occlusion map to obtain optical flow increments, including: The sample background attention and sample 2D motion vectors are input into the online network to obtain query samples of 2D motion vectors; The sample 2D motion vector and the preset occlusion map are input into the target network to obtain the positive sample vector; The optical flow increment is calculated based on the query sample and positive sample vector of the 2D motion vector.

[0010] In some embodiments, adjusting the network parameters of the online network based on optical flow increments includes: Preset occlusion map is adjusted based on optical flow increment; The online network is iteratively trained based on the adjusted preset occlusion map.

[0011] In some embodiments, the method further includes: The contrastive loss is calculated based on the query sample and positive sample vectors using 2D motion vectors; The network parameters of the online network are adjusted based on the contrast loss, and the network parameters of the target network are updated according to the momentum update principle.

[0012] In a second aspect, the present invention also provides an optical flow computing device based on contrastive learning, the device comprising: The feature extraction module is used to extract features from the original image and the target image through a convolutional network, resulting in the original feature map of the original image, the background features of the original image, and the target feature map of the target image. The relevant building block is used to construct a correlation volume based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching. The feature processing module is used to calculate attention to background features, obtain background attention, and extract 2D motion vectors based on the relevant volume. The optical flow calculation module is used to input background attention and 2D motion vectors into the online network to obtain optical flow calculation results.

[0013] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the optical flow calculation method based on contrastive learning provided in the first aspect.

[0014] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the optical flow calculation method based on contrastive learning provided in the first aspect.

[0015] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the optical flow calculation method based on contrastive learning provided in the first aspect.

[0016] The beneficial effects of this invention are as follows: The optical flow calculation method based on contrastive learning in this invention first extracts features from the original image and the target image through a convolutional network to obtain the original feature map, background features of the original image, and target feature map of the target image; then, a correlation volume is constructed based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching; then, attention is calculated on the background features to obtain background attention, and 2D motion vectors are extracted based on the correlation volume; finally, the background attention and 2D motion vectors are input into an online network to obtain the optical flow calculation result. The above method realizes the calculation of optical flow through a neural network. In this process, there is no need to artificially synthesize datasets, overcoming the problem of overfitting. Moreover, optical flow calculation can be achieved based on only one target image and one original image, without the need to obtain a large amount of real scene datasets. This achieves optical flow calculation while overcoming the limitations of existing technologies.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an optical flow calculation method based on contrastive learning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a proxy task according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating an online network training method according to an embodiment of the present invention. Figure 4 This is a flowchart illustrating another optical flow calculation method based on contrastive learning, as shown in an embodiment of the present invention. Figure 5 This is a schematic diagram of the structure of an optical flow computing device based on contrastive learning according to an embodiment of the present invention; Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0019] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. It should be noted that references to "an embodiment," "embodiment," "example embodiment," etc., in this specification refer to the described embodiment including specific features, structures, or characteristics; however, not every embodiment must include these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge scope of those skilled in the art.

[0020] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] In some embodiments, such as Figure 1 As shown, a method for optical flow calculation based on contrastive learning is provided, which includes: S101 uses a convolutional network to extract features from the original image and the target image, obtaining the original feature map of the original image, the background features of the original image, and the target feature map of the target image.

[0022] Specifically, optical flow calculation refers to the optical flow changes between two images at different times. Therefore, it requires an original image and a target image, with the target image corresponding to a later time point than the original image. The original image and the target image are respectively input into a convolutional network to extract the original feature map, the background features of the original image, and the target feature map of the target image.

[0023] S102, construct the correlation body based on the original feature map and the target feature map.

[0024] Among them, the correlation volume is used to record the degree of pixel matching.

[0025] The product of each position in the original feature map and the target feature map is calculated to obtain the correlation volume, which retains the similarity information of each point in the two feature maps.

[0026] S103, perform attention calculation on background features to obtain background attention, and extract 2D motion vectors based on relevant volumes.

[0027] The 2D motion vectors reflect the motion information of each point in the two feature maps.

[0028] Specifically, background attention is calculated by performing attention calculations on background features, and 2D motion vectors are extracted from the relevant volumes.

[0029] S104 inputs background attention and 2D motion vectors into the online network to obtain optical flow calculation results.

[0030] Specifically, by inputting background attention and 2D motion vectors into the online network, the online network can predict the optical flow calculation results.

[0031] Optionally, the background attention and 2D motion vectors can be converted into sample pairs based on the agent task; then the sample pairs can be input into the encoding network for feature extraction to obtain sample pair features; finally, the sample pair features can be mapped to a new feature space through a multi-layer MLP and prediction can be performed to obtain the optical flow calculation results.

[0032] Specifically, the 2D motion vectors extracted from the relevant data are not entirely accurate. Therefore, further refinement of the 2D motion vectors using an online network is necessary to obtain accurate optical flow calculation results. This can be achieved by using a proxy task to transform background attention and 2D motion vectors into sample pairs. Here, the proxy task can be motion aggregation, which uses the correlation of background features to infer the motion characteristics of occluded pixels, thereby obtaining sample pairs. Figure 2 As shown, Q and K are first extracted from the background features, and then the background attention matrix A is calculated using formula (1). t It records the degree of similarity between feature points at different locations: (1); Among them, Q i These are query samples generated by the online network, K j These are the positive samples generated by the target network, where N represents the total number of negative samples in the Memory Bank, D is the dimension of the sample vector, and A... t '(i,j) is the background attention matrix.

[0033] Based on the principle that similar pixels have similar motions, the motion of occluded pixels can be deduced from the motion vectors of unoccluded pixels. As shown in formula (2), vj is the 2D motion vector corresponding to other pixels, and yi and yˆi are the 2D motion vectors before and after the proxy task, respectively: (2); Among them, y i yˆ i These are the 2D motion vectors before and after the proxy task, v j It is the 2D motion vector corresponding to all pixels other than this pixel.

[0034] After feature extraction (RestNet), motion mapping (MLP), and motion prediction (MLP), the online network outputs the optical flow calculation results.

[0035] The optical flow calculation method based on contrastive learning in the above embodiments first extracts features from the original image and the target image using a convolutional network to obtain the original feature map, background features of the original image, and target feature map of the target image. Then, a correlation volume is constructed based on the original and target feature maps; the correlation volume records the degree of pixel matching. Next, attention is calculated on the background features to obtain background attention, and 2D motion vectors are extracted based on the correlation volume. Finally, the background attention and 2D motion vectors are input into an online network to obtain the optical flow calculation result. This method achieves optical flow calculation through a neural network. In this process, it does not require artificially synthesized datasets, overcoming the overfitting problem. Furthermore, it only requires a target image and an original image to perform optical flow calculation, eliminating the need for large amounts of real-world scene datasets. This achieves optical flow calculation while overcoming the limitations of existing technologies.

[0036] In another embodiment, such as Figure 3 As shown, a method for training an online network is provided, which includes: S201, Obtain the preset occlusion map.

[0037] The occlusion information contained in the occlusion map represents the confidence level of pixel matching (the confidence level of matching in unoccluded areas is high). This allows the prediction results to minimize the aggregation of occluded areas during the training process and extract better 2D motion vector feature representations.

[0038] S202, based on the sample background attention, sample 2D motion vector and preset occlusion map, momentum comparison is performed to obtain the optical flow increment.

[0039] The sample background attention and sample 2D motion vector are obtained based on the original sample image and the sample target image. The methods for obtaining the sample background attention and sample 2D motion vector are described in S101-S103, and will not be repeated here.

[0040] Specifically, the sample background attention and sample 2D motion vectors can be input into the online network to obtain query samples of 2D motion vectors; then the sample 2D motion vectors and the preset occlusion map can be input into the target network to obtain positive sample vectors; finally, the optical flow increment is calculated based on the query samples of 2D motion vectors and the positive sample vectors.

[0041] During the training of an online network, a target network is required for assistance; however, it is not needed during the actual use of the online network. For the online network, based on the agent task, the sample background attention and the sample 2D motion vector are transformed into sample pairs (refer to Equations 1 and 2). These sample pairs are then input into the encoding network for feature extraction. The resulting sample pair features are then mapped to a new feature space using a multi-layer MLP to obtain the query sample y of the 2D motion vector. query : (3); Among them, LP and ResNet are typical deep learning network modules, consisting of convolutional modules, activation functions, etc. query It is the output vector of the online network, also known as the query vector, yˆ i It is the 2D motion vector calculated by formula (2) and processed by the proxy task.

[0042] Unlike the proxy task in the online mode, the proxy task in the target mode first introduces an occlusion map overlaid on the background, and then performs operations such as motion aggregation, resulting in a positive sample vector ypos. For the predicted sample yquery, ypos is its corresponding positive sample. After this training, the coordinates of expired positive sample ypos are added to the negative sample queue, and the oldest element in the queue is deleted.

[0043] The optical flow increment can be obtained by comparing the query sample of the 2D motion vector with the positive sample vector.

[0044] S203 adjusts the network parameters of the online network based on optical flow increment.

[0045] Specifically, if the calculated optical flow increment cannot accurately reflect the optical flow between the original sample image and the target sample image, then the network parameters of the online network and the target network need to be adjusted.

[0046] Optionally, the preset occlusion map is adjusted based on the optical flow increment; the online network is iteratively trained based on the adjusted preset occlusion map.

[0047] Specifically, the preset occlusion map can be adjusted based on the optical flow increment, and the online network can be iteratively trained based on the adjusted preset occlusion map. At the same time, the contrastive loss needs to be calculated based on the query samples and positive sample vectors of 2D motion vectors. (4); Among them, y query It is a query sample generated by the online network, ypos is the positive sample generated by the target network, ki is the data in the queue of negative samples, T is the hyperparameter, and Lq is the contrastive loss, which is often used as an unsupervised loss function to assist in training.

[0048] The network parameters of the online network are adjusted based on contrastive loss, and the network parameters of the target network are updated according to the momentum update principle. The momentum update formula is as follows: (5); in, These are the network parameters for the online network and the target network, respectively, and m is the momentum parameter.

[0049] The training method of the above-mentioned online network uses the target network to update parameters. During the training process, the preset occlusion map is updated according to the optical flow increment, and the network parameters of the target network are updated according to the momentum update principle, thereby increasing the accuracy of the online network in predicting the optical flow calculation results.

[0050] To more comprehensively demonstrate this solution, this embodiment presents an optional approach to optical flow calculation based on contrastive learning, such as... Figure 4 As shown: S301, Obtain the preset occlusion map.

[0051] S302, input the sample background attention and sample 2D motion vector into the online network to obtain the query sample of 2D motion vector.

[0052] Among them, the sample background attention and sample 2D motion vector are obtained based on the original sample image and the sample target image.

[0053] S303, input the sample 2D motion vector and the preset occlusion map into the target network to obtain the positive sample vector.

[0054] S304, the optical flow increment is calculated based on the query sample and positive sample vector of the 2D motion vector.

[0055] S305, adjusts the preset occlusion map based on optical flow increment.

[0056] S306, calculate the contrastive loss based on the query sample and positive sample vectors using 2D motion vectors.

[0057] S307 adjusts the network parameters of the online network based on contrastive loss and updates the network parameters of the target network according to the momentum update principle.

[0058] S308 iteratively trains the online network based on the adjusted preset occlusion map.

[0059] S309 uses a convolutional network to extract features from the original image and the target image, obtaining the original feature map of the original image, the background features of the original image, and the target feature map of the target image.

[0060] S310, constructing a correlation body based on the original feature map and the target feature map.

[0061] Among them, the correlation volume is used to record the degree of pixel matching.

[0062] S311, perform attention calculation on background features to obtain background attention, and extract 2D motion vectors based on relevant volumes.

[0063] S312 transforms background attention and 2D motion vectors into sample pairs based on a proxy task.

[0064] S313, Input the sample pair into the encoding network for feature extraction to obtain the sample pair features.

[0065] S314 maps the features of sample pairs to a new feature space through a multi-layer MLP and performs predictions to obtain optical flow calculation results.

[0066] The specific processes of S301-S314 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0067] Based on the same inventive concept, this application also provides a contrastive learning-based optical flow computing device for implementing the contrastive learning-based optical flow computing method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of the one or more contrastive learning-based optical flow computing device embodiments provided below can be found in the limitations of the contrastive learning-based optical flow computing method described above, and will not be repeated here.

[0068] In one embodiment, such as Figure 5 As shown, an optical flow computing device based on contrastive learning is provided, the device comprising: The feature extraction module 40 is used to extract features from the original image and the target image through a convolutional network to obtain the original feature map of the original image, the background features of the original image, and the target feature map of the target image. The relevant construction module 41 is used to construct a correlation volume based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching. Feature processing module 42 is used to perform attention calculation on background features to obtain background attention and extract 2D motion vectors based on the relevant volume; The optical flow calculation module 43 is used to input background attention and 2D motion vectors into the online network to obtain optical flow calculation results.

[0069] In another embodiment, the above Figure 5 The optical flow calculation module 43 is specifically used for: converting background attention and 2D motion vectors into sample pairs based on the agent task; inputting the sample pairs into the encoding network for feature extraction to obtain sample pair features; mapping the sample pair features to a new feature space through a multi-layer MLP and performing prediction to obtain optical flow calculation results.

[0070] In another embodiment, the above Figure 5 The contrastive learning-based optical flow computing device also includes: The image acquisition module is used to acquire a preset occlusion image; The incremental calculation module is used to perform momentum comparison based on sample background attention, sample 2D motion vector and preset occlusion map to obtain optical flow increment; sample background attention and sample 2D motion vector are obtained based on sample original image and sample target image; The parameter adjustment module is used to adjust the network parameters of the online network based on the optical flow increment.

[0071] In another embodiment, the incremental calculation module in the above embodiment is specifically used to: input the sample background attention and the sample 2D motion vector into the online network to obtain the query sample of the 2D motion vector; input the sample 2D motion vector and the preset occlusion map into the target network to obtain the positive sample vector; and calculate the optical flow increment based on the query sample of the 2D motion vector and the positive sample vector.

[0072] The process of adjusting the network parameters of the online network based on optical flow increments includes: adjusting the preset occlusion map based on optical flow increments; and iteratively training the online network based on the adjusted preset occlusion map.

[0073] In another embodiment, the above Figure 5 The optical flow computing device based on contrastive learning is also used for: calculating contrastive loss based on query samples and positive sample vectors of 2D motion vectors; adjusting the network parameters of the online network based on the contrastive loss; and updating the network parameters of the target network according to the momentum update principle.

[0074] This application also provides an electronic device, in some embodiments, referring to... Figure 6As shown, the electronic device 700 includes an input unit 710, a memory 720, a processor 730, and an output unit 740. The memory 720 stores program instructions that can be executed on the processor 730. The processor 730 can execute the optical flow calculation method and / or technical solution based on contrastive learning in the foregoing embodiments by calling the program instructions. The electronic device 700 can be a mobile terminal device such as a mobile phone or a computer.

[0075] Furthermore, embodiments of this application also provide a computer-readable storage medium for storing a computer program that executes an optical flow calculation method based on contrastive learning. For example, computer program instructions, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. The program instructions that invoke the methods of this application may be stored in a fixed or removable storage medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in a storage medium that operates according to the program instructions.

[0076] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0077] The technical features of the above embodiments can be arbitrarily integrated. For the sake of brevity, not all possible integrations of the technical features in the above embodiments are described. However, as long as the integration of these technical features does not contradict each other, they should be considered to be within the scope of this specification.

[0078] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for optical flow calculation based on contrastive learning, characterized in that, The method includes: The convolutional network is used to extract features from the original image and the target image, resulting in the original feature map, the background features of the original image, and the target feature map of the target image. A correlation volume is constructed based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching. Attention is calculated on the background features to obtain background attention, and 2D motion vectors are extracted based on the relevant volumes; the formula for calculating the background attention is: ; Among them, Q i These are query samples generated by the online network, K j These are the positive samples generated by the target network, where N represents the total number of negative samples in the storage system, D is the dimension of the sample vector, and A... t '(i,j) is the background attention matrix; The training method of the online network includes: acquiring a preset occlusion map; performing momentum comparison based on sample background attention, sample 2D motion vectors, and the preset occlusion map to obtain optical flow increments; the sample background attention and the sample 2D motion vectors are obtained based on the original sample image and the sample target image; and adjusting the network parameters of the online network based on the optical flow increments. The optical flow increment is obtained by comparing the momentum of the sample background attention, the sample 2D motion vector, and the preset occlusion map. The process includes: inputting the sample background attention and the sample 2D motion vector into an online network to obtain a query sample of the 2D motion vector; inputting the sample 2D motion vector and the preset occlusion map into a target network to obtain a positive sample vector; and calculating the optical flow increment based on the query sample of the 2D motion vector and the positive sample vector. Adjusting the network parameters of the online network based on the optical flow increment includes: adjusting the preset occlusion map based on the optical flow increment; and iteratively training the online network based on the adjusted preset occlusion map. The background attention and 2D motion vectors are input into an online network to obtain optical flow calculation results, including: converting the background attention and 2D motion vectors into sample pairs based on a proxy task; inputting the sample pairs into an encoding network for feature extraction to obtain sample pair features; mapping the sample pair features to a new feature space through a multi-layer MLP and performing prediction to obtain optical flow calculation results.

2. The optical flow calculation method based on contrastive learning as described in claim 1, characterized in that, The method further includes: The contrast loss is calculated based on the query sample and the positive sample vector using the 2D motion vector; the formula for calculating the contrast loss is: ; Among them, y query It is a query sample generated by the online network, y pos The target network generates positive samples, and ki represents the data in the queue of negative samples. It's a hyperparameter. To compare the losses, K is the number of negative samples, and ki is a positive sample when i=0; The network parameters of the online network are adjusted based on the contrast loss, and the network parameters of the target network are updated according to the momentum update principle.

3. An optical flow computing device based on contrastive learning, characterized in that, The device includes: The feature extraction module is used to extract features from the original image and the target image through a convolutional network, resulting in the original feature map of the original image, the background features of the original image, and the target feature map of the target image. The correlation construction module is used to construct a correlation body based on the original feature map and the target feature map; the correlation body is used to record the degree of pixel matching. The feature processing module is used to calculate attention on the background features to obtain background attention, and extract 2D motion vectors based on the relevant volume; the calculation formula for the background attention is: ; Among them, Q i These are query samples generated by the online network, K j These are the positive samples generated by the target network, where N represents the total number of negative samples in the storage system, D is the dimension of the sample vector, and A... t '(i,j) is the background attention matrix; The training method of the online network includes: acquiring a preset occlusion map; performing momentum comparison based on sample background attention, sample 2D motion vectors, and the preset occlusion map to obtain optical flow increments; the sample background attention and the sample 2D motion vectors are obtained based on the original sample image and the sample target image; and adjusting the network parameters of the online network based on the optical flow increments. The optical flow increment is obtained by comparing the momentum of the sample background attention, the sample 2D motion vector, and the preset occlusion map. The process includes: inputting the sample background attention and the sample 2D motion vector into an online network to obtain a query sample of the 2D motion vector; inputting the sample 2D motion vector and the preset occlusion map into a target network to obtain a positive sample vector; and calculating the optical flow increment based on the query sample of the 2D motion vector and the positive sample vector. Adjusting the network parameters of the online network based on the optical flow increment includes: adjusting the preset occlusion map based on the optical flow increment; and iteratively training the online network based on the adjusted preset occlusion map. The optical flow calculation module is used to input the background attention and 2D motion vector into an online network to obtain optical flow calculation results. The module includes: converting the background attention and 2D motion vector into sample pairs based on a proxy task; inputting the sample pairs into an encoding network for feature extraction to obtain sample pair features; mapping the sample pair features to a new feature space through a multi-layer MLP and performing prediction to obtain optical flow calculation results.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the optical flow calculation method based on contrastive learning as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the optical flow calculation method based on contrastive learning as described in any one of claims 1 to 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the optical flow calculation method based on contrastive learning as described in any one of claims 1 to 2.