Optical flow estimation method, system and electronic device combining convolution and axial attention

Abstract

The application relates to an optical flow estimation method, system and electronic equipment combining convolution and shaft attention, the method comprising: acquiring and extracting first matching features and second matching features of the first frame image and the second frame image, and extracting context network features of the first frame image; extracting peripheral relationship information of each feature point in the first matching features, the second matching features and the context network features respectively to obtain first LC-LD matching features, second LC-LD matching features and LC-LD context features; calculating correlation cost of the first LC-LD matching features and the second LC-LD matching features, and inputting the correlation cost and the LC-LD context features into a GRU network for iterative optimization to obtain low-resolution optical flow; and up-sampling the low-resolution optical flow to obtain high-resolution optical flow. The application can still obtain good precision and generalization performance when a small number of iterations is used in optical flow estimation.

Description

Technical Field

[0001] The present invention relates to the technical field of optical flow estimation, and in particular to an optical flow estimation method, system and electronic device combining convolution and axis attention. Background Art

[0002] Optical flow estimation is a fundamental task in computer vision and is widely used in numerous fields, such as autonomous driving, target tracking, obstacle avoidance, and 3D reconstruction. Early optical flow estimation methods, based on the assumptions of constant brightness and spatial consistency, demonstrated their advantages. However, these methods inherently lack the ability to address large displacements and non-constant brightness, forcing researchers to seek new methods for more accurate optical flow estimation. Recently, with the development of related equipment and the rise of deep learning, an increasing number of researchers have achieved significant improvements in the accuracy and speed of optical flow estimation through deep learning. However, to achieve superior accuracy, many models rely on numerous iterations of optimization, further limiting their deployment performance on edge computing platforms. Summary of the Invention

[0003] To this end, the technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology that the model has a large number of iterations and is not easy to deploy on the edge computing platform.

[0004] To solve the above technical problems, the present invention provides an optical flow estimation method combining convolution and axis attention, comprising:

[0005] Step S1: Acquire a first frame image and a second frame image;

[0006] Step S2: extracting first matching features and second matching features of the first frame image and the second frame image respectively, and extracting context network features of the first frame image;

[0007] Step S3: extracting peripheral relationship information of each feature point in the first matching feature, the second matching feature, and the context network feature respectively to obtain a first LC-LD matching feature, a second LC-LD matching feature, and an LC-LD context feature;

[0008] Step S4: calculating the correlation cost of the first LC-LD matching feature and the second LC-LD matching feature, and inputting the correlation cost and the LC-LD context feature into the GRU network for iterative optimization to obtain a low-resolution optical flow;

[0009] Step S5: upsampling the low-resolution optical flow to obtain a high-resolution optical flow, wherein the high-resolution optical flow is an optical flow direction generated by the moving distance and moving direction of the same target in the first frame image and the second frame image.

[0010] In one embodiment of the present invention, the method of extracting the surrounding relationship information of each feature point in the first matching feature, the second matching feature, and the context network feature in step S3 includes:

[0011] Step S31: sending the initial feature to a local constraint module composed of a layer of 3×3 convolution to obtain local information of the initial feature and generate LC features, wherein the initial feature is the first matching feature, the second matching feature or the context network feature;

[0012] Step S32: Input the LC feature into a multi-axis attention module composed of horizontal axis attention and vertical axis attention to obtain axis attention features about the LC feature, wherein the horizontal axis attention is based on the current feature point, and self-attention is performed on the row where the current feature point is located, to obtain the horizontal axis attention of the current feature point and its row; the vertical axis attention is based on the current feature point, and self-attention is performed on the column where the current feature point is located, to obtain the vertical axis attention of the current feature point and its column;

[0013] Step S33: Use a local expansion module composed of a layer of 3×3 convolution to interact the axis attention feature with the feature points around it to generate LD features;

[0014] Step S34: Input the LD into the feedforward neural network to obtain the feedforward neural network features, then use convolution to fuse the feedforward neural network features with the initial features, and finally perform dimension enhancement on the fused features to obtain the local constraint and local expansion features F LC-LD .

[0015] In one embodiment of the present invention, the multi-axis attention module formula in step S32 is: A =col(row(F C ), where col(·) represents vertical axis attention, row(·) represents horizontal axis attention, and F C Indicates LC characteristics, F A Represents the axis attention feature.

[0016] In one embodiment of the present invention, step S33 is specifically: using a local expansion module composed of a layer of 3×3 convolution to obtain the horizontal axis attention of the row and the vertical axis attention of the column where the feature points around the current feature point are located to generate LD features.

[0017] In one embodiment of the present invention, the surrounding relationship information of each feature point in the first matching feature, the second matching feature and the context network feature is extracted respectively, and the formula is: LC-LD =Conv1×1 (Conv 3×3 ([FFN(F D ),F])), where F D represents LD features, FFN(·) represents feedforward neural network, F represents initial features, [·] represents feature concatenation, Conv 3×3 Represents 3×3 convolution, Conv 1×1 represents 1×1 convolution, F LC-LD Represents the obtained local constraints and local expansion features.

[0018] In one embodiment of the present invention, the correlation cost between the first LC-LD matching feature and the second LC-LD matching feature is calculated in step S4 using the formula: corr(i,j,p,q)=G1(i,j) T G2(p,q), where G1(i,j) represents any pixel in the first LC-LD matching feature G1, G2(p,q) represents any pixel in the second LC-LD matching feature G2, and corr(i,j,p,q) represents the four-dimensional LC-LD correlation cost.

[0019] In one embodiment of the present invention, in step S5, the low-resolution optical flow is upsampled to obtain a high-resolution optical flow, specifically by performing group convolution, channel shuffling and convex optimization-based upsampling on the low-resolution optical flow in sequence to obtain a high-resolution optical flow.

[0020] To solve the above technical problems, the present invention provides an optical flow estimation system combining convolution and axis attention, comprising:

[0021] Acquisition module: used to acquire the first frame image and the second frame image;

[0022] A first extraction module is used to extract first matching features and second matching features of the first frame image and the second frame image respectively, and extract context network features of the first frame image;

[0023] A second extraction module is used to extract the surrounding relationship information of each feature point in the first matching feature, the second matching feature and the context network feature, respectively, to obtain a first LC-LD matching feature, a second LC-LD matching feature and an LC-LD context feature;

[0024] Iterative optimization module: used to calculate the correlation cost of the first LC-LD matching feature and the second LC-LD matching feature, and input the correlation cost and the LC-LD context feature into the GRU network for iterative optimization to obtain low-resolution optical flow;

[0025] Upsampling module: used to upsample the low-resolution optical flow to obtain a high-resolution optical flow, wherein the high-resolution optical flow is the optical flow direction generated by the moving distance and moving direction of the same target in the first frame image and the second frame image.

[0026] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned optical flow estimation method combining convolution and axis attention are implemented.

[0027] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned optical flow estimation method combining convolution and axis attention are implemented.

[0028] The above technical solution of the present invention has the following advantages over the prior art:

[0029] The present invention combines convolution and axial attention to construct a local constraint and local expansion module (LC-LD module), which effectively improves the performance of visual features, improves the deficiency of traditional axial attention that requires two iterations to obtain a larger receptive field, and alleviates the problem of poor generalization caused by using only axial attention. Since the LC-LD module used in the present invention pays attention to the different levels of importance in the surrounding areas of each feature point, it generates a more accurate correlation cost, thereby reducing the number of iterations of the network model.

[0030] The present invention combines group convolution, channel shuffling and convex optimization upsampling to further reduce the number of parameters.

[0031] By adopting the idea of ​​"one enhancement, multiple iterations of benefit", the present invention focuses on generating more robust correlation cost quantities, so that each query can obtain more accurate results and relatively high accuracy with fewer iterations, better achieving the balance between time and accuracy, improving the computing performance deployed on the edge computer platform, and providing a new solution for lightweight models. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0033] Figure 1 is a flow chart of the method of the present invention;

[0034] Figure 2 This is a schematic diagram of the overall framework of the network model of the present invention;

[0035] Figure 3Schematic diagram of the LC-LD module framework of the present invention;

[0036] Figure 4 It is a comparison chart of the experimental results of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0038] Example 1

[0039] Reference Figure 1 and Figure 2 As shown, the optical flow estimation method combining convolution and axis attention of the present invention includes:

[0040] Step S1: Acquire a first frame image and a second frame image;

[0041] Step S2: extracting first matching features and second matching features of the first frame image and the second frame image respectively, and extracting context network features of the first frame image;

[0042] Step S3: extracting peripheral relationship information of each feature point in the first matching feature, the second matching feature, and the context network feature respectively to obtain a first LC-LD matching feature, a second LC-LD matching feature, and an LC-LD context feature;

[0043] Step S4: calculating the correlation cost of the first LC-LD matching feature and the second LC-LD matching feature, and inputting the correlation cost and the LC-LD context feature into the GRU network for iterative optimization to obtain a low-resolution optical flow;

[0044] Step S5: upsampling the low-resolution optical flow to obtain a high-resolution optical flow, wherein the high-resolution optical flow is an optical flow direction generated by the moving distance and moving direction of the same target in the first frame image and the second frame image.

[0045] This embodiment combines convolution and axial attention to construct a local constraint and local expansion module (LC-LDModule), focusing on the different levels of importance in the surrounding areas of each feature point to generate a more accurate correlation cost, thereby reducing the number of iterations of the network model. At the same time, combining group convolution, channel shuffling and convex optimization upsampling can further reduce the number of parameters. This embodiment can effectively reduce the number of iterative optimizations of the network model and improve the operating efficiency of the model on the edge computing platform.

[0046] The following is a detailed introduction to this embodiment:

[0047] Step S2 specifically includes: inputting the first frame image and the second frame image into the RESNET network to extract first matching features (Matching Features) and second matching features (Matching Features), and inputting the first frame image into the RESNET network without regularization to extract context network features (Context Features).

[0048] See also Figure 3 The method of extracting the surrounding relationship information of each feature point in the first matching feature, the second matching feature, and the context network feature in step S3 includes:

[0049] Step S31: Send the initial feature F to a local constraint module composed of a layer of 3×3 convolution to obtain local information of the initial feature and generate LC features, wherein the initial feature F is the first matching feature, the second matching feature or the context network feature in step S2;

[0050] Step S32: Input the LC feature into a multi-axis attention module consisting of horizontal axis attention and vertical axis attention to obtain axis attention features about the LC feature, wherein the horizontal axis attention is based on the current feature point (i.e., the current pixel point), and self-attention is performed on the row where the current feature point is located to obtain the horizontal axis attention of the current feature point and its row; the vertical axis attention is based on the current feature point, and self-attention is performed on the column where the current feature point is located to obtain the vertical axis attention of the current feature point and its column;

[0051] The formula of the multi-axis attention module in step S32 is:

[0052] F A =col(row(F C ))

[0053] Among them, col(·) represents the vertical axis attention, row(·) represents the horizontal axis attention, F C Indicates LC characteristics, F A Represents the axis attention feature;

[0054] Step S33: Use a local expansion module composed of a layer of 3×3 convolution to interact the axis attention feature with the feature points around it to generate LD features. Specifically, use a local expansion module composed of a layer of 3×3 convolution to obtain the horizontal axis attention of the rows and vertical axis attention of the columns of the feature points around the current feature point, thereby obtaining a larger receptive field and generating LD features.

[0055] Step S34: Use a feedforward neural network (FFN) to further represent the LD feature to obtain a feedforward neural network feature, then use a 3×3 convolution to fuse the feedforward neural network feature with the initial feature F, and finally use a 1×1 convolution to increase the dimension of the fused feature to obtain a local constraint and local expansion feature F. LC-LD .

[0056] In summary, the overall steps of step S3 can be expressed by the following formula:

[0057] F LC-LD =Conv 1×1 (Conv 3×3 ([FFN(F D ),F]))

[0058] Among them, F D represents LD features, FFN(·) represents feedforward neural network, F represents initial features, [·] represents feature concatenation, Conv 3×3 Represents 3×3 convolution, Conv 1×1 represents 1×1 convolution, F LC-LD Represents the obtained local constraints and local expansion features.

[0059] In step S4, the correlation cost between the first LC-LD matching feature and the second LC-LD matching feature is calculated using the formula:

[0060] corr(i,j,p,q)=G1(i,j) T G2(p,q)

[0061] Among them, G1(i,j) represents any pixel in the first LC-LD matching feature G1, G2(p,q) represents any pixel in the second LC-LD matching feature G2, and corr(i,j,p,q) represents the four-dimensional LC-LD correlation cost.

[0062] In step S5, the low-resolution optical flow is upsampled to obtain a high-resolution optical flow, specifically: the low-resolution optical flow is sequentially subjected to group convolution, channel shuffling, and upsampling based on convex optimization to obtain a high-resolution optical flow.

[0063] The following experiments verify the high performance of this embodiment:

[0064] The model of this embodiment is built under the PyTorch framework, trained using a 2080Ti GPU, and the AdamW optimizer is selected to optimize the network. In order to evaluate the effectiveness of the model, four datasets, FlyingChairs, FlyingThings3D, MPI-Sintel, and KITTI 2015, are used for training and evaluation. First, pre-training is performed on FlyingChairs and FlyingThings3D, and then fine-tuning is performed on MPI-Sintel. When training on FlyingChairs, the batch size is set to 10, the number of iterations is set to 100,000 times, the learning rate is 0.0004, and the image crop size is 352×480. When training on FlyingThings3D, the batch size is set to 6, the number of iterations is set to 120,000 times, the learning rate is 0.000125, and the image crop size is 400×720. Finally, fine-tuning is performed on the MPI-Sintel training set with the batch size set to 6, the learning rate to 0.0004, and the image cropping size to 368×768.

[0065] This example uses two performance indicators to evaluate the model:

[0066] (1) Time metric: This example uses the time required for the model to forward infer a pair of MPI-Sintel images (436×1024) on the Jetson AGX Xavier edge computing platform as the time metric. In addition, to avoid errors in time statistics, this example uses the method of forward inferring 1000 groups of images and taking the average to calculate the time statistics.

[0067] (2) Accuracy: For the MPI-Sintel dataset, this embodiment uses the end point error (EPE) as the accuracy metric. The EPE formula is shown below, and represents the distance between the predicted optical flow vector and the true optical flow vector for the same pixel. For the KITTI 2015 dataset, this embodiment uses both the end point error and F1-all as accuracy metrics. F1-all represents the percentage of pixels with incorrect optical flow estimation.

[0068]

[0069] Among them, (u0,v0) represents the predicted optical flow vector, and (u1,v1) represents the real optical flow vector.

[0070] The specific experimental results are shown in Table 1. Compared with the baseline model RAFT(small), the model of this embodiment only needs 12 iterations to surpass the accuracy of the baseline model with 32 iterations on the MPI-Sintel test set, and the forward inference time on the edge computing platform is reduced by 46%.

[0071] Table 1 Experimental comparison results

[0072]

[0073] Compared with other lightweight methods, the method of this embodiment surpasses FlowNet2, SpyNet and LiteFlowNet in terms of time and accuracy on the MPI-Sintel test set. Compared with LiteFlowNet2, the model iterated 6 times using this embodiment (i.e., LC-LDs in Table 1) can achieve a time similar to that of LiteFlowNet2, and can achieve better accuracy on the MPI-Sintel and KITTI2015 training sets, and has better generalization performance than that of LiteFlowNet2. In addition, compared with FlowNetC and PWC-Net, LC-LDs achieves an overall improvement in accuracy at a lower time cost. Compared with FastFlowNet, since it is impossible to completely avoid the use of GRU for iterative optimization, a certain number of iterations are still required to obtain a certain accuracy, which will consume relatively more time. Based on the above experiments, this embodiment can better achieve a balance between accuracy and time. Figure 4 A visual comparison of the method of this embodiment (i.e., LC-LD in Table 1) and RAFT(small) on the MPI-Sintel(Final) test set is shown.

[0074] In addition, this embodiment uses the KITTI 2015 training set as the test set to verify the generalization performance of the proposed model on a real-world dataset. As shown in Table 1, compared with RAFT(small), the proposed model can achieve accuracy on the KITTI 2015 training set with only 12 iterations, which is similar to the accuracy achieved with RAFT(small) after 32 iterations. Compared with other optical flow estimation methods, the proposed model achieves good accuracy. Testing on a challenging real-world dataset demonstrates that the proposed model can still achieve good generalization performance even with fewer iterations.

[0075] Example 2

[0076] This embodiment provides an optical flow estimation system combining convolution and axis attention, including:

[0077] Acquisition module: used to acquire the first frame image and the second frame image;

[0078] A first extraction module is used to extract first matching features and second matching features of the first frame image and the second frame image respectively, and extract context network features of the first frame image;

[0079] A second extraction module is used to extract the surrounding relationship information of each feature point in the first matching feature, the second matching feature and the context network feature, respectively, to obtain a first LC-LD matching feature, a second LC-LD matching feature and an LC-LD context feature;

[0080] Iterative optimization module: used to calculate the correlation cost of the first LC-LD matching feature and the second LC-LD matching feature, and input the correlation cost and the LC-LD context feature into the GRU network for iterative optimization to obtain low-resolution optical flow;

[0081] Upsampling module: used to upsample the low-resolution optical flow to obtain a high-resolution optical flow, wherein the high-resolution optical flow is the optical flow direction generated by the moving distance and moving direction of the same target in the first frame image and the second frame image.

[0082] Example 3

[0083] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the optical flow estimation method combining convolution and axis attention described in the first embodiment are implemented.

[0084] Example 4

[0085] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the optical flow estimation method combining convolution and axis attention described in the first embodiment are implemented.

[0086] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0087] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0090] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0091] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. An optical flow estimation method combining convolution and axis attention, characterized by: include: Step S1: Acquire a first frame image and a second frame image; Step S2: extracting first matching features and second matching features of the first frame image and the second frame image respectively, and extracting context network features of the first frame image; Step S3: extracting peripheral relationship information of each feature point in the first matching feature, the second matching feature, and the context network feature respectively to obtain a first LC-LD matching feature, a second LC-LD matching feature, and an LC-LD context feature, the method comprising: Step S31: sending the initial feature to a local constraint module composed of a layer of 3×3 convolution to obtain local information of the initial feature and generate an LC feature, wherein the LC feature is a local constraint feature, and the initial feature is a first matching feature, a second matching feature, or a context network feature; Step S32: Input the LC feature into a multi-axis attention module consisting of horizontal axis attention and vertical axis attention to obtain axis attention features about the LC feature, wherein the horizontal axis attention is based on the current feature point, and self-attention is performed on the row where the current feature point is located, and the horizontal axis attention between the current feature point and its row is obtained; the vertical axis attention is based on the current feature point, and self-attention is performed on the column where the current feature point is located, and the vertical axis attention between the current feature point and its column is obtained; Step S33: using a local expansion module consisting of a layer of 3×3 convolution to interact the axis attention feature with the feature points around it to generate an LD feature, wherein the LD feature is a local expansion feature; Step S34: Input the LD feature into the feedforward neural network to obtain the feedforward neural network feature, then use convolution to fuse the feedforward neural network feature with the initial feature, and finally perform dimension enhancement on the fused feature to obtain the local constraint and local expansion feature F LC-LD ; Step S4: calculating the correlation cost of the first LC-LD matching feature and the second LC-LD matching feature, and inputting the correlation cost and the LC-LD context feature into the GRU network for iterative optimization to obtain a low-resolution optical flow; Step S5: upsampling the low-resolution optical flow to obtain a high-resolution optical flow, wherein the high-resolution optical flow is an optical flow direction generated by the moving distance and moving direction of the same target in the first frame image and the second frame image.

2. The optical flow estimation method combining convolution and axis attention according to claim 1, characterized in that: The multi-axis attention module formula in step S32 is: A =col(row(F C ), where col(·) represents vertical axis attention, row(·) represents horizontal axis attention, and F C Indicates LC characteristics, F A Represents the axis attention feature.

3. The optical flow estimation method combining convolution and axis attention according to claim 1, characterized in that: The step S33 is specifically as follows: using a local expansion module composed of a layer of 3×3 convolution to obtain the horizontal axis attention of the row and the vertical axis attention of the column where the feature points around the current feature point are located, and generate LD features.

4. The optical flow estimation method combining convolution and axis attention according to claim 1, characterized in that: The formula for extracting the surrounding relationship information of each feature point in the first matching feature, the second matching feature and the context network feature is: LC-LD =Conv 1×1 (Conv 3×3 ([FFN(F D ),F])), where F D represents LD features, FFN(·) represents feedforward neural network, F represents initial features, [·] represents feature concatenation, Conv 3×3 Represents 3×3 convolution, Conv 1×1 represents 1×1 convolution, F LC-LD Represents the obtained local constraints and local expansion features.

5. The optical flow estimation method combining convolution and axis attention according to claim 1, characterized in that: In step S4, the correlation cost between the first LC-LD matching feature and the second LC-LD matching feature is calculated using the formula: corr(i,j,p,q)=G1(i,j) T G2(p,q), where G1(i,j) represents any pixel in the first LC-LD matching feature G1, G2(p,q) represents any pixel in the second LC-LD matching feature G2, and corr(i,j,p,q) represents the four-dimensional LC-LD correlation cost.

6. The optical flow estimation method combining convolution and axis attention according to claim 1, characterized in that: In step S5, the low-resolution optical flow is upsampled to obtain a high-resolution optical flow, specifically: the low-resolution optical flow is sequentially subjected to group convolution, channel shuffling, and upsampling based on convex optimization to obtain a high-resolution optical flow.

7. An optical flow estimation system combining convolution and axis attention, for implementing the optical flow estimation method combining convolution and axis attention described in any one of 1 to 6, characterized in that: include: Acquisition module: used to acquire the first frame image and the second frame image; A first extraction module is used to extract first matching features and second matching features of the first frame image and the second frame image respectively, and extract context network features of the first frame image; A second extraction module is used to extract the surrounding relationship information of each feature point in the first matching feature, the second matching feature and the context network feature, respectively, to obtain a first LC-LD matching feature, a second LC-LD matching feature and an LC-LD context feature; Iterative optimization module: used to calculate the correlation cost of the first LC-LD matching feature and the second LC-LD matching feature, and input the correlation cost and the LC-LD context feature into the GRU network for iterative optimization to obtain low-resolution optical flow; Upsampling module: used to upsample the low-resolution optical flow to obtain a high-resolution optical flow, wherein the high-resolution optical flow is the optical flow direction generated by the moving distance and moving direction of the same target in the first frame image and the second frame image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the processor implements the steps of the optical flow estimation method combining convolution and axis attention according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the optical flow estimation method combining convolution and axis attention as claimed in any one of claims 1 to 6 are implemented.

Images