Optical flow calculation method, device and equipment based on comparative learning
By employing a contrastive learning-based optical flow computation method, which utilizes convolutional networks and momentum contrastive training, the problems of dataset dependency and related volume redundancy in existing optical flow computations are solved, achieving efficient optical flow computation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-25
- Publication Date
- 2026-04-10
AI Technical Summary
Existing optical flow calculation methods rely on small-scale artificially synthesized datasets, leading to overfitting problems. Furthermore, it is difficult to obtain optical flow label data in real-world scenarios, resulting in a lot of redundant information and a lack of effective filtering and purification network structures.
A contrastive learning-based approach is adopted, which extracts image features through a convolutional network, constructs a correlation volume, performs background attention calculation and 2D motion vector extraction, and uses momentum contrast training of the online network and the target network to overcome overfitting and realize optical flow calculation.
Optical flow calculations can be performed without artificially synthesized datasets, reducing overfitting, improving the accuracy and efficiency of optical flow calculations, and reducing reliance on real-world scene datasets.
Smart Images
Figure CN121837318A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of optical flow calculation, in particular to an optical flow calculation method and device based on contrast learning, equipment and readable storage medium. BACKGROUND
[0002] Optical flow calculation aims to find the corresponding motion relationship of pixels between images, and is widely used in automatic driving, visual odometry, video restoration and scene reconstruction tasks. The classical optical flow calculation is considered as an optimization problem, and since 2015, many end-to-end deep learning networks have emerged to directly calculate the optical flow value. In recent years, the optical flow calculation method based on visual attention has occupied a dominant position.
[0003] Optical flow calculation faces two problems: dataset dependence and correlation body redundancy. The existing optical flow calculation dataset is small in size and mostly artificially synthesized, for example, Sintel is a collection of images cut from a video, and FlyingChairs is a 3D chair model rendered on a picture. This causes most of the existing optical flow calculation methods to have overfitting phenomenon, and there is a decline in the test sequence of KITTI dataset. In order to make up for the huge gap between synthetic data and real data, more real scene datasets are needed, but it is very difficult to obtain optical flow label data in real scenes, and it is necessary to use a motion capture system to track the motion of each pixel. The correlation body is a key node in optical flow calculation, which records the similarity relationship between pixel pairs, but it also contains a large amount of redundant and erroneous information. The existing optical flow calculation methods focus on image feature extraction and correlation body calculation, and there are few network structures that specifically filter and purify the correlation body.
[0004] Therefore, there is an urgent need for an optical flow calculation method based on contrast learning to solve the above problems. SUMMARY
[0005] The purpose of the present application is to provide an optical flow calculation method, device and equipment based on contrast learning, which realizes the calculation of optical flow through a neural network. In this process, there is no need for artificial synthesis of data sets, overcoming the problem of overfitting, and only one target image and one original image are needed to realize optical flow calculation, without the need for a large number of real scene data sets, realizing optical flow calculation on the basis of overcoming the prior art.
[0006] To achieve the above purpose, the present application provides the following technical scheme: In a first aspect, the present application provides an optical flow calculation method based on contrast learning, which comprises: extracting features of the original image and the target image through a convolutional network to obtain an original feature map of the original image, a background feature of the original image and a target feature map of the target image; constructing a correlation body based on the original feature map and the target feature map; the correlation body is used to record the pixel matching degree; performing attention calculation on the background feature to obtain background attention, and extracting a 2D motion vector based on the correlation body; inputting the background attention and the 2D motion vector into an online network to obtain an optical flow calculation result.
[0007] In some embodiments, inputting the background attention and the 2D motion vector into the online network to obtain the optical flow calculation result comprises: translating the background attention and the 2D motion vector into a sample pair based on a proxy task; inputting the sample pair into an encoding network to extract a sample pair feature; mapping the sample pair feature to a new feature space through a multi-layer MLP and performing prediction to obtain the optical flow calculation result.
[0008] In some embodiments, the training method of the online network comprises: obtaining a preset occlusion map; performing momentum comparison based on a sample background attention, a sample 2D motion vector and the preset occlusion map to obtain an optical flow increment; the sample background attention and the sample 2D motion vector are obtained based on a sample original image and a sample target image; adjusting network parameters of the online network based on the optical flow increment.
[0009] In some embodiments, performing momentum comparison based on the sample background attention, the sample 2D motion vector and the preset occlusion map to obtain the optical flow increment comprises: inputting the sample background attention and the sample 2D motion vector into the online network to obtain a query sample of the 2D motion vector; inputting the sample 2D motion vector and the preset occlusion map into a target network to obtain a positive sample vector; calculating the optical flow increment based on the query sample of the 2D motion vector and the positive sample vector.
[0010] In some embodiments, adjusting the network parameters of the online network based on the optical flow increment comprises: adjusting the preset occlusion map based on the optical flow increment; iteratively training the online network based on the adjusted preset occlusion map.
[0011] In some embodiments, the method further comprises: calculating a comparison loss based on the query sample of the 2D motion vector and the positive sample vector; The network parameters of the online network are adjusted based on the contrast loss, and the network parameters of the target network are updated according to a momentum update principle.
[0012] In a second aspect, the present application further provides a light flow calculation device based on contrast learning, comprising: The feature extraction module is configured to extract features of the original image and the target image through a convolution network to obtain an original feature map of the original image, background features of the original image, and a target feature map of the target image. The correlation construction module is configured to construct a correlation volume based on the original feature map and the target feature map, and the correlation volume is configured to record a pixel matching degree. The feature processing module is configured to perform attention calculation on the background features to obtain background attention and extract 2D motion vectors based on the correlation volume. The light flow calculation module is configured to input the background attention and the 2D motion vectors into the online network to obtain a light flow calculation result.
[0013] In a third aspect, the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the light flow calculation method based on contrast learning provided in the first aspect when executing the computer program.
[0014] In a fourth aspect, the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executable on the processor to implement the light flow calculation method based on contrast learning provided in the first aspect.
[0015] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, and the computer program is executable on the processor to implement the light flow calculation method based on contrast learning provided in the first aspect.
[0016] The light flow calculation method based on contrast learning in the present application first extracts features of an original image and a target image through a convolution network to obtain an original feature map of the original image, background features of the original image, and a target feature map of the target image, then constructs a correlation volume based on the original feature map and the target feature map, and the correlation volume is configured to record a pixel matching degree, then performs attention calculation on the background features to obtain background attention and extracts 2D motion vectors based on the correlation volume, and finally inputs the background attention and the 2D motion vectors into an online network to obtain a light flow calculation result. The above method realizes the calculation of light flow through a neural network, and in this process, artificial synthetic data sets are not required, the problem of overfitting is overcome, and the light flow calculation can be realized only based on one target image and one original image, without the need to obtain a large number of real scene data sets, thereby realizing the light flow calculation on the basis of overcoming the prior art.
[0017] The above description is only a summary of the technical solutions of the present application. In order to make the technical means of the present application clearer and can be implemented according to the content of the specification, the following will be described in detail with the preferred embodiments of the present application and with the help of the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 A flowchart of a light flow calculation method based on contrast learning according to an embodiment of the present application is shown. Figure 2 A flowchart of a proxy task according to an embodiment of the present application is shown. Figure 3 A flowchart of a training method of an online network according to an embodiment of the present application is shown. Figure 4 A flowchart of another light flow calculation method based on contrast learning according to an embodiment of the present application is shown. Figure 5 A structural diagram of a light flow calculation device based on contrast learning according to an embodiment of the present application is shown. Figure 6 An electronic device structure according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0019] The technical solutions of the present application will be described in detail below with the help of the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application. It should be noted that the description of "one embodiment", "embodiment", "example embodiment" and the like in the specification means that the described embodiment can include a specific feature, structure or property, but not every embodiment must include these specific features, structures or properties. In addition, such expressions do not mean the same embodiment. Further, when a specific feature, structure or property is described in combination with an embodiment, it is indicated that such a feature, structure or property is combined with other embodiments within the knowledge of those skilled in the art, whether or not it is explicitly described.
[0020] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as there is no conflict.
[0021] In some embodiments, as shown in Figure 1 a light flow calculation method based on contrast learning is provided, which comprises: S101, feature extraction is performed on the original image and the target image through a convolutional network to obtain an original feature map of the original image, background features of the original image, and a target feature map of the target image.
[0022] Specifically, the optical flow calculation refers to the optical flow change of two images at different times, and thus requires an original image and a target image. The target image corresponds to a time later than the original image. The original image and the target image are input into the convolutional network to extract the original feature map of the original image, the background features of the original image, and the target feature map of the target image.
[0023] S102, a correlation body is constructed based on the original feature map and the target feature map.
[0024] The correlation body is used to record the pixel matching degree.
[0025] The product calculation is performed on each position in the original feature map and the target feature map to obtain the correlation body, which retains the similarity information of each point in the two feature maps.
[0026] S103, attention calculation is performed on the background features to obtain background attention, and a 2D motion vector is extracted based on the correlation body.
[0027] The 2D motion vector reflects the motion information of each point in the two feature maps.
[0028] Specifically, the attention calculation is performed on the background features to obtain the background attention, and the 2D motion vector is extracted from the correlation body.
[0029] S104, the background attention and the 2D motion vector are input into an online network to obtain an optical flow calculation result.
[0030] Specifically, the background attention and the 2D motion vector are input into the online network, and the online network can predict the optical flow calculation result.
[0031] Optionally, the background attention and the 2D motion vector can be first converted into a sample pair based on a proxy task; then the sample pair is input into an encoding network for feature extraction to obtain sample pair features; finally, the sample pair features are mapped to a new feature space through a multi-layer MLP, and prediction is performed to obtain the optical flow calculation result.
[0032] Specifically, the 2D motion vectors extracted from the correlation body are not completely accurate, so further purification of the 2D motion vectors is needed based on the online network, so as to obtain accurate optical flow calculation results. The background attention and 2D motion vector can be converted into a sample pair based on the proxy task. The proxy task can be motion aggregation, that is, the motion characteristics of the occluded pixels are calculated based on the correlation of the background features, so as to obtain the sample pair, as shown in Figure 2 First, Q and K are extracted from the background features, and then the background attention matrix A t is calculated using formula (1), which records the similarity between feature points at different positions: (1); Where Q i is the query sample generated by the online network, K j is the positive sample generated by the target network, N represents the number of all negative sample sets in the Memory Bank, D is the dimension of the sample vector, and A t '(i, j) is the background attention matrix.
[0033] According to the principle that similar pixels have similar motion, the motion of the occluded pixels can be derived from the motion vectors of the unoccluded pixels. As shown in formula (2), vj is the 2D motion vector corresponding to other pixels, and y i and y i are the 2D motion vectors before and after the proxy task: (2); Where y i and y i are the 2D motion vectors before and after the proxy task, and v j is the 2D motion vector corresponding to other pixels except the current pixel.
[0034] After feature extraction (RestNet), motion mapping (MLP) and motion prediction (MLP), the online network outputs the optical flow calculation result.
[0035] The light flow calculation method based on contrast learning in the above embodiment first extracts features of the original image and the target image through a convolutional network to obtain an original feature map of the original image, a background feature of the original image, and a target feature map of the target image; then, a correlation body is constructed based on the original feature map and the target feature map; the correlation body is used to record the pixel matching degree; then, attention calculation is performed on the background feature to obtain background attention, and a 2D motion vector is extracted based on the correlation body; finally, the background attention and the 2D motion vector are input into an online network to obtain a light flow calculation result. The above method realizes the calculation of light flow through a neural network, and in this process, manual synthetic data sets are not needed, the problem of overfitting is overcome, and light flow calculation can be realized only according to one target image and one original image, without the need to obtain a large number of real scene data sets, thereby realizing light flow calculation on the basis of overcoming the prior art.
[0036] In another embodiment, as shown in Figure 3 a training method of an online network is provided, and the method comprises the following steps: S201, obtaining a preset occlusion map.
[0037] The occlusion information contained in the occlusion map represents the credibility of pixel matching (the unoccluded area has high credibility of matching), so that in the training process, the prediction result can shield the aggregation of the occluded area as much as possible, and better 2D motion vector feature expression is extracted.
[0038] S202, performing momentum comparison based on the sample background attention, the sample 2D motion vector, and the preset occlusion map to obtain a light flow increment.
[0039] The sample background attention and the sample 2D motion vector are obtained based on a sample original image and a sample target image. The sample background attention and the sample 2D motion vector are obtained in the same manner as S101-S103, which will not be described here.
[0040] Specifically, the sample background attention and the sample 2D motion vector can be input into the online network to obtain a query sample of the 2D motion vector; then, the sample 2D motion vector and the preset occlusion map are input into the target network to obtain a positive sample vector; finally, the light flow increment is calculated based on the query sample of the 2D motion vector and the positive sample vector.
[0041] In the training process of the online network, the assistance of the target network is needed, but in the use process of the online network, the target network is not needed. For the online network, the sample context attention and the sample 2D motion vector are converted into a sample pair based on the proxy task (see formula 1 and formula 2); the sample pair is input into an encoding network for feature extraction to obtain a sample pair feature; the sample pair feature is mapped to a new feature space through a multi-layer MLP to obtain a 2D motion vector query sample y query : (3); Wherein, LP, ResNet are typical deep learning network modules respectively, including convolution modules, activation functions and the like, y query is the output vector of the online network, also known as the query vector, yˆ i is the 2D motion vector processed by the proxy task calculated by formula (2).
[0042] The difference between the proxy task in the online and the proxy task in the target is that the proxy task in the target will first introduce the occlusion map superimposed on the background, and then perform motion aggregation and the like, and the obtained is called a positive sample vector ypos. For the prediction sample yquery, ypos is the corresponding positive sample, and after the end of this training, the coordinates of the expired positive sample ypos are added to the negative sample queue, and the oldest element in the queue is deleted.
[0043] Comparing the query sample and the positive sample vector of the 2D motion vector, the optical flow increment can be obtained.
[0044] S203, adjusting the network parameters of the online network based on the optical flow increment.
[0045] Specifically, if the optical flow increment calculated at this time cannot accurately reflect the optical flow between the sample original image and the sample target image, the network parameters of the online network and the target network need to be adjusted.
[0046] Optionally, the preset occlusion map is adjusted based on the optical flow increment; and the online network is iteratively trained based on the adjusted preset occlusion map.
[0047] Specifically, the preset occlusion map can be adjusted based on the optical flow increment, and the online network is iteratively trained based on the adjusted preset occlusion map, and at the same time, a comparison loss is calculated based on the query sample and the positive sample vector of the 2D motion vector: (4); Wherein, y query is the query sample generated by the online network, ypos is a positive sample generated by the target network, ki is data in the set of negative samples of the queue, T is a hyperparameter, and Lq is a contrastive loss, which is often used as an unsupervised loss function to assist training.
[0048] The network parameters of the online network are adjusted based on the contrastive loss, and the network parameters of the target network are updated according to the momentum update principle. The formula of the momentum update is as follows: (5); wherein, are the network parameters of the online network and the target network respectively, and m is a momentum parameter.
[0049] The training method of the online network updates the parameters with the help of the target network, and in the training process, the preset occlusion map is updated according to the optical flow increment, and the network parameters of the target network are updated according to the momentum update principle, thereby increasing the accuracy of the online network in predicting the optical flow calculation result.
[0050] In order to more comprehensively show the present scheme, an optional way of the optical flow calculation method based on contrastive learning is given in the embodiment, as shown in Figure 4 . S301, obtaining a preset occlusion map.
[0051] S302, inputting a sample background attention and a sample 2D motion vector into the online network to obtain a query sample of the 2D motion vector.
[0052] Wherein, the sample background attention and the sample 2D motion vector are obtained based on a sample original image and a sample target image.
[0053] S303, inputting the sample 2D motion vector and the preset occlusion map into the target network to obtain a positive sample vector.
[0054] S304, calculating an optical flow increment based on the query sample of the 2D motion vector and the positive sample vector.
[0055] S305, adjusting the preset occlusion map based on the optical flow increment.
[0056] S306, calculating a contrastive loss based on the query sample of the 2D motion vector and the positive sample vector.
[0057] S307, adjusting the network parameters of the online network based on the contrastive loss, and updating the network parameters of the target network according to the momentum update principle.
[0058] S308, iteratively train the online network based on the adjusted preset occlusion map.
[0059] S309, feature extraction of the original image and the target image is performed through a convolutional network to obtain an original feature map of the original image, a background feature of the original image, and a target feature map of the target image.
[0060] S310, a correlation body is constructed based on the original feature map and the target feature map.
[0061] The correlation body is used to record the pixel matching degree.
[0062] S311, attention calculation is performed on the background feature to obtain background attention, and a 2D motion vector is extracted based on the correlation body.
[0063] S312, the background attention and the 2D motion vector are converted into a sample pair based on a proxy task.
[0064] S313, the sample pair is input into an encoding network for feature extraction to obtain a sample pair feature.
[0065] S314, the sample pair feature is mapped to a new feature space through a multi-layer MLP, and prediction is performed to obtain an optical flow calculation result.
[0066] The specific process of S301-S314 can be referred to the description of the above method embodiments, which has similar implementation principles and technical effects, and will not be repeated here.
[0067] Based on the same inventive concept, the embodiments of the present application also provide a contrast learning-based optical flow calculation device for implementing the contrast learning-based optical flow calculation method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more contrast learning-based optical flow calculation device embodiments provided below can be referred to the limitations of the contrast learning-based optical flow calculation method described above, and will not be repeated here.
[0068] In one embodiment, as shown in Figure 5 a contrast learning-based optical flow calculation device is provided, which comprises: A feature extraction module 40 is configured to perform feature extraction of the original image and the target image through a convolutional network to obtain an original feature map of the original image, a background feature of the original image, and a target feature map of the target image. A correlation construction module 41 is configured to construct a correlation body based on the original feature map and the target feature map. The correlation body is used to record the pixel matching degree. A feature processing module 42 is configured to perform attention calculation on the background feature to obtain background attention, and extract a 2D motion vector based on the correlation body. The optical flow calculation module 43 is configured to input the background attention and the 2D motion vector into an online network to obtain an optical flow calculation result.
[0069] In another embodiment, the optical flow calculation module 43 in the above Figure 5 is specifically configured to: convert the background attention and the 2D motion vector into a sample pair based on a proxy task; input the sample pair into an encoding network to extract a sample pair feature; map the sample pair feature to a new feature space through a multi-layer MLP, and perform prediction to obtain the optical flow calculation result.
[0070] In another embodiment, the optical flow calculation module 43 in the above Figure 5 is specifically configured to: convert the background attention and the 2D motion vector into a sample pair based on a proxy task; input the sample pair into an encoding network to extract a sample pair feature; map the sample pair feature to a new feature space through a multi-layer MLP, and perform prediction to obtain the optical flow calculation result. The image acquisition module is configured to acquire a preset occlusion map. The incremental calculation module is configured to perform momentum contrast based on the sample background attention, the sample 2D motion vector, and the preset occlusion map to obtain an optical flow increment; the sample background attention and the sample 2D motion vector are obtained based on a sample original image and a sample target image. The parameter adjustment module is configured to adjust network parameters of the online network based on the optical flow increment.
[0071] In another embodiment, the incremental calculation module in the above embodiment is specifically configured to: input the sample background attention and the sample 2D motion vector into the online network to obtain a query sample of the 2D motion vector; input the sample 2D motion vector and the preset occlusion map into the target network to obtain a positive sample vector; and calculate the optical flow increment based on the query sample of the 2D motion vector and the positive sample vector.
[0072] The parameter adjustment module is configured to adjust network parameters of the online network based on the optical flow increment.
[0073] In another embodiment, the optical flow calculation module 43 in the above Figure 5 is specifically configured to: convert the background attention and the 2D motion vector into a sample pair based on a proxy task; input the sample pair into an encoding network to extract a sample pair feature; map the sample pair feature to a new feature space through a multi-layer MLP, and perform prediction to obtain the optical flow calculation result.
[0074] The present application also provides an electronic device, in some embodiments, referring to Figure 6As shown, the electronic device 700 includes an input unit 710, a memory 720, a processor 730, and an output unit 740. The memory 720 stores program instructions executable on the processor 730, and the processor 730 invokes the program instructions to perform the contrast learning based optical flow calculation method and / or technical solutions in the foregoing embodiments. The electronic device 700 can be a mobile terminal device such as a mobile phone or a computer.
[0075] In addition, the embodiments of the present application also provide a computer readable storage medium for storing a computer program for performing the contrast learning based optical flow calculation method. For example, computer program instructions, when executed by a computer, can invoke or provide the method and / or technical solutions according to the present application through the operation of the computer. The program instructions for invoking the method of the present application can be stored in a fixed or removable storage medium, and / or transmitted and / or stored in a storage medium running according to the program instructions through a data stream in a broadcast or other signal bearing medium.
[0076] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Alternatively, they can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module to realize. Therefore, the present application is not limited to any specific combination of hardware and software.
[0077] The technical features of the above embodiments can be integrated in any manner. In order to make the description simple, not all possible integrations of the technical features in the above embodiments are described, however, as long as the integrations of the technical features do not exist contradictions, they should be considered as the scope of the present application.
[0078] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.
Claims
1. A method for optical flow calculation based on contrastive learning, characterized in that, The method includes: The convolutional network is used to extract features from the original image and the target image, resulting in the original feature map, the background features of the original image, and the target feature map of the target image. A correlation volume is constructed based on the original feature map and the target feature map; the correlation volume is used to record the degree of pixel matching. Attention is calculated on the background features to obtain background attention, and 2D motion vectors are extracted based on the relevant volumes; the formula for calculating the background attention is: ； Among them, Q i These are query samples generated by the online network, K j These are the positive samples generated by the target network, N represents the total number of negative samples in the storage system, D is the dimension of the sample vector, and A... t '(i,j) is the background attention matrix; The background attention and 2D motion vectors are input into the online network to obtain the optical flow calculation results.
2. The optical flow calculation method based on contrastive learning as described in claim 1, characterized in that, The background attention and 2D motion vectors are input into the online network to obtain optical flow calculation results, including: Based on the agent task, the background attention and the 2D motion vector are converted into sample pairs; The sample pairs are input into an encoding network for feature extraction to obtain sample pair features; The sample pairs are mapped to a new feature space using a multi-layer MLP, and predictions are performed to obtain optical flow calculation results.
3. The optical flow calculation method based on contrastive learning as described in claim 1, characterized in that, The training method for the online network includes: Obtain the preset occlusion image; Momentum comparison is performed based on sample background attention, sample 2D motion vectors, and a preset occlusion map to obtain optical flow increments; the sample background attention and the sample 2D motion vectors are obtained based on the original sample image and the sample target image. The network parameters of the online network are adjusted based on the optical flow increment.
4. The optical flow calculation method based on contrastive learning as described in claim 3, characterized in that, Momentum comparison is performed based on sample background attention, sample 2D motion vectors, and a preset occlusion map to obtain optical flow increments, including: The sample background attention and sample 2D motion vectors are input into the online network to obtain query samples of 2D motion vectors; The sample 2D motion vector and the preset occlusion map are input into the target network to obtain the positive sample vector; The optical flow increment is calculated based on the query sample and the positive sample vector of the 2D motion vector.
5. The optical flow calculation method based on contrastive learning as described in claim 4, characterized in that, Adjusting the network parameters of the online network based on the optical flow increment includes: The preset occlusion map is adjusted based on the optical flow increment; The online network is iteratively trained based on the adjusted preset occlusion map.
6. The optical flow calculation method based on contrastive learning as described in claim 4, characterized in that, The method further includes: The contrast loss is calculated based on the query sample and the positive sample vector using the 2D motion vector; the formula for calculating the contrast loss is: ； Among them, y query It is a query sample generated by the online network, y pos is the positive sample generated by the target network, ki is the data in the queue of negative samples, T is the hyperparameter, and Lq is the contrastive loss; The network parameters of the online network are adjusted based on the contrast loss, and the network parameters of the target network are updated according to the momentum update principle.
7. An optical flow computing device based on contrastive learning, characterized in that, The device includes: The feature extraction module is used to extract features from the original image and the target image through a convolutional network, resulting in the original feature map of the original image, the background features of the original image, and the target feature map of the target image. The correlation construction module is used to construct a correlation body based on the original feature map and the target feature map; the correlation body is used to record the degree of pixel matching. The feature processing module is used to perform attention calculation on the background features to obtain background attention, and extract 2D motion vectors based on the relevant volume; The optical flow calculation module is used to input the background attention and 2D motion vectors into the online network to obtain the optical flow calculation results.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the optical flow calculation method based on contrastive learning as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the optical flow calculation method based on contrastive learning as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the optical flow calculation method based on contrastive learning as described in any one of claims 1 to 6.