Local optical flow activation method of multi-aperture array imaging system under sparse topological framework

Abstract

The invention relates to a local optical flow activation method of a multi-aperture imaging system under a sparse topological framework, which is used for self-adaptive motion vector estimation of the multi-aperture imaging system in a complex motion scene, and compared with the traditional variational method, deep learning and other optical flow estimation methods, the local optical flow activation method can process visual motion data without complicated label labeling, and has the advantages of simple operation and high efficiency. And a high-precision array flow field is ensured, so that the flow field estimation error caused by pixel-level processing shielding and visual angle change is realized. Meanwhile, correlation motion characteristics among apertures are fully considered, self-adaptive estimation of the regional apertures is achieved, and the burden of computing resources is greatly reduced. The lightweight neural network is utilized to accelerate SIFT feature search, the accuracy of key point descriptors can be improved through a self-enhancement mechanism, the key point descriptors can be embedded into a multi-aperture array imaging system by occupying lower computing resources, morphological feature key point screening is achieved, and finally the purpose of flow field local robustness enhancement is achieved.

Description

Technical Field

[0001] The invention relates to a local optical flow activation method, in particular to a local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework. Background Art

[0002] Monocular imaging systems are no longer able to meet the needs of autonomous driving, computer vision, and autonomous navigation applications. Therefore, multi-camera imaging has been widely used in accurate and fast positioning measurement to achieve target motion perception, thereby achieving visual control, directional navigation, high-dynamic target tracking, and obstacle avoidance for unmanned intelligent vehicles. Visual motion perception information is necessary for the realization of intelligent perception in IoT applications such as unmanned motion platforms. Multi-aperture optical imaging systems integrate preparation technology, optical design, and machine vision algorithms. Compared with traditional single-aperture imaging methods, they can provide multi-channel perception and wide-field high-speed processing. A large number of studies have been conducted on multi-aperture optical imaging systems for different tasks, and a variety of array imaging motion perception methods have been applied. Among them, the establishment of a mathematical model for aperture light field information processing plays a vital role in narrowing the gap between structural design and application in the research process.

[0003] A single global optical flow estimation has been proven to be unable to better meet the needs of multi-aperture imaging systems in motion detection. In practical applications, optical flow information interfered by static information of the scene is a major source of error, making it difficult to achieve a balance between computing resources and estimation accuracy. Introducing feature descriptors to assist the optical flow estimation task has been proven to be a feasible approach, which can significantly improve the accuracy of optical flow estimation and adaptively screen the optical flow. Such as ORB, SIFT, ALIKE, SURF. Both handcrafted and learning-based descriptors perform well in motion estimation practice, and some of them have become the default descriptors for certain motion estimation algorithms. For example, the binary descriptor ORB is widely used in binocular SLAM systems, and SURF is used for motion structure system recovery.

[0004] However, such existing technologies may require current hardware to have more computing power, or sometimes require extensive modifications to the algorithm model due to changes in descriptor types, which poses higher challenges to upper-level visual perception tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework, comprising:

[0008] Step S1: Use a multi-aperture array imaging system to capture a series of images, select a source image and a target image, and calculate an initial optical flow vector motion field;

[0009] Step S2: Based on the initial optical flow vector motion field, combined with the number and arrangement of apertures of the multi-aperture array imaging system, construct an optical flow vector motion field under a sparse framework;

[0010] Step S3: Based on the optical flow vector motion field, the optical flow sparse matrix is ​​calculated using the GSP analysis process;

[0011] Step S4: Use a multi-layer perceptron to map the original descriptor into an adaptive description operator through scale-invariant feature transformation to capture the relationship between key points;

[0012] Step S5: local optical flow activation is achieved by combining the relationship between key points and the optical flow sparse matrix.

[0013] The mathematical expression of the optical flow vector motion field is:

[0014]

[0015] Where: E(v(x)) is the energy function, C(x,v(x)) is the basic energy function in the optical flow vector motion field, G(v(x)) is the regular energy function in the optical flow vector motion field, x is the gray value of the pixel, and v(x) is the local motion activation flow field.

[0016] The process of acquiring the optical flow vector motion field under the sparse framework in step S2 includes:

[0017] Step S2-1: Update the local motion activation flow field based on the number and arrangement of apertures and the current optical flow vector motion field:

[0018] Step S2-2: updating the optical flow vector motion field according to the updated local motion activation flow field;

[0019] Step S2-3: Repeat step S2-1 and step S2-2 until convergence, and obtain the optical flow vector motion field under the sparse framework.

[0020] Update the local motion activation flow field as:

[0021]

[0022] Where: v n (x) is the optical flow motion field of a single aperture, and N is the number of apertures in the multi-aperture array imaging system.

[0023] The step S3 comprises:

[0024] Step S3-1: Represent the optical flow vector motion field as a flow field matrix, and combine it with the set dictionary matrix to obtain an estimated value of the coefficient matrix:

[0025]

[0026] in: is the estimated value of the coefficient matrix, X is the flow field matrix, D is the dictionary matrix, A is the coefficient matrix, λ is the sparse constraint for regularization constraint, is a regularized calculation method, ||·|| 1 is another regularization calculation method, where a is an element in the coefficient matrix.

[0027] Step S3-2: Introduce the graph Laplace regularization term, combine the energy function of the optical flow estimation, and combine the estimated value of the coefficient matrix to obtain the estimated value of the reconstructed flow field;

[0028] Step S3-3: Based on the gradient adaptive strategy, a gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap area, and the flow field is reconstructed according to the array arrangement mode for the reconstructed flow field V.

[0029] The estimated value of the reconstructed flow field is:

[0030]

[0031] in: is the estimated value of the reconstructed flow field, M is the set of aperture numbers, H is the flow field degradation strategy, V is the reconstructed flow field, γ is the graph Laplace regularization parameter, Tr is the matrix trace, U is the Laplace transform matrix, and L is the matrix after Laplace transform decomposition.

[0032] The flow field reconstructed in step S3-3 is:

[0033]

[0034] in: is the reconstructed optical flow motion vector under the pixel set i that matches the motion vector j. The purpose is to suppress the basis vectors at the same position through gradient operation. V ikm is the reconstructed optical flow motion vector of the pixel set i that matches the motion vector k under the graph constraint, V ijn is the reconstructed optical flow motion suppression vector under the pixel set i that matches the motion vector j, V ikn is the optical flow motion suppression vector under the pixel set i that matches the motion vector k under the graph constraint, V ijm is the reconstructed optical flow motion vector of the pixel set i that matches the motion vector j.

[0035] The step S4 specifically includes:

[0036] The Transformer network is used to embed the key points of scale-invariant feature transformation to capture the relationship between key points and enhance the expressiveness of features. A multi-layer perceptron is used to approximate the descriptor space to achieve fast extraction of scale-invariant feature transformation.

[0037] The input of the Transformer network is N local features in the same image, and the output is an enhanced feature descriptor.

[0038] In the attention calculation of the Transformer network, the connection mode is only used in the calculation of keys or values, and element multiplication is used instead of matrix multiplication. The mathematical expression is

[0039]

[0040] Where: f i (X) is the improved Transformer attention function, σ(Q i ) is the sigmoid function operation, Q i is the key point of the i-th row of Q, K j is the key point of the jth row of Q, V j is the key point of the j-th row of V, softmax is the activation function, K is the key, and ⊙ is the element-by-element multiplication operation.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. Introduce a multi-aperture topological sparse representation framework, introduce sparse coding technology into array aperture optical flow motion estimation, give a generalized modulation framework for array optical flow, establish a dictionary mapping relationship and learn important motion features in the field of view, so as to achieve pixel-level processing of flow field estimation errors caused by occlusion and perspective changes. Compared with traditional optical flow estimation methods such as variational methods and deep learning, it can process visual motion data without complicated label annotation and ensure high-precision array flow field.

[0043] 2. Aperture-dependent robust optical flow estimation is adopted. In view of the characteristics of overlapping imaging of multi-aperture imaging system arrays, GSP (graph signal processing) technology is introduced to further characterize the optical flow motion vector, and sparse coding technology is used to realize joint sparse representation of the flow field. The correlated motion characteristics between apertures are fully considered to realize regional aperture adaptive estimation, which greatly reduces the burden of computing resources.

[0044] 3. Introduce a fast SIFT feature extraction network and use a lightweight neural network to accelerate SIFT feature search. Compared with traditional feature representation operators, it can not only improve the accuracy of key point descriptors through self-enhancement mechanisms, but also occupy lower computing resources and be embedded in multi-aperture array imaging systems to achieve adaptive array optical flow extraction tasks. By screening morphological feature key points, local robust enhancement of the flow field can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the technical principle of the present invention;

[0046] Figure 2 It is an aperture constrained optimization method based on graph signal processing;

[0047] Figure 3 This is a visual comparison diagram of the optical flow field estimation effect of the multi-aperture array image of the present invention and other methods;

[0048] Figure 4 It is a schematic diagram of the main steps of the present invention. DETAILED DESCRIPTION

[0049] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0050] A local optical flow activation method for multi-aperture array imaging system under a sparse topology framework, such as Figure 1 and Figure 4 As shown, including:

[0051] Step S1: Use a multi-aperture array imaging system to capture a series of images, select a source image and a target image, and calculate an initial optical flow vector motion field;

[0052] Use a multi-aperture array imaging system to shoot a series of images, select the source image and the target image, and the optical flow vector motion field needs to process the multi-aperture images frame by frame to obtain multiple sets of multi-aperture image sequences at time t and time t-1; calculate the vertical and horizontal increments of the compound eye image sequence at time t and time t-1, and the calculation formula is I t (x+v)-I t-1 (x). The initial velocity field is calculated according to the calculation formula, including the initial motion increment in the vertical direction and the initial motion increment in the horizontal direction.

[0053] The mathematical expression of the optical flow vector motion field is:

[0054]

[0055] Where: E(v(x)) is the energy function, C(x,v(x)) is the basic energy function in the optical flow vector motion field, G(v(x)) is the regular energy function in the optical flow vector motion field, x is the gray value of the pixel, v(x) is the local motion activation flow field, and image I t-1 The global pixel of the scene and the image I t The regularization term of the motion vector between , through minimization solution to obtain the dense motion vector field v D (x).

[0056] The basic energy function in the optical flow vector motion field can be expressed as:

[0057] C(x,v(x))=|I t (x+v)-I t-1 (x)|+β(x)|▽I t (x+v)-▽I t-1 (x)|

[0058] Where: β(x) = 1-ρ(|▽I t (x)|), β(x) is a robust adjustment term used to increase the matching robustness of the texture-free area of ​​array imaging, and ρ is an adjustment factor to control the weight, I t (x+v) is the brightness value of pixel x at displacement v at time t, I t-1 (x) is the brightness value of pixel x at time t-1, ▽I t (x+v) is the gradient value of pixel x at displacement v at time t, ▽I t-1 (x) is the gradient value of pixel x at v at time t.

[0059] The calculation method of the regular energy function in the optical flow vector motion field can be expressed as:

[0060]

[0061] Among them: ω(x) is used to penalize discontinuity, D represents the flow field set of the entire image, v(x d )-v(x d-1 ) represents the difference between adjacent flow fields.

[0062] Step S2: Based on the initial optical flow vector motion field, combined with the number and arrangement of apertures of the multi-aperture array imaging system, construct an optical flow vector motion field under a sparse framework;

[0063] Among them, the process of obtaining the optical flow vector motion field under the sparse framework includes:

[0064] Step S2-1: Update the local motion activation flow field based on the number and arrangement of apertures and the current optical flow vector motion field:

[0065]

[0066] Where: v n (x) is the optical flow motion field of a single aperture, and N is the number of apertures in the multi-aperture array imaging system.

[0067] Step S2-2: updating the optical flow vector motion field according to the updated local motion activation flow field;

[0068] Step S2-3: Repeat step S2-1 and step S2-2 until convergence, and obtain the optical flow vector motion field under the sparse framework.

[0069] Step S3: Based on the optical flow vector motion field, the optical flow sparse matrix is ​​calculated using the GSP analysis process, including:

[0070] Step S3-1: According to the arrangement characteristics of the aperture array of the multi-aperture imaging system, the image information can be represented by N is the data dimension, M is the number of apertures. The image is regarded as a set of nodes, each node represents a pixel, and the edge represents the spatial proximity and similarity between nodes. GSP (graph signal processing) is used to operate on the graph structure composed of the pixels of the optical flow. Define the representation method of the array aperture optical flow map G OF (V, Y, W), V represents a set of N aperture flow field nodes, Y represents the boundary of the aperture set, and W represents the neighbor weight matrix. For an image containing optical flow information, the weight w of the edge connecting nodes i and j is i,j It can be calculated using the Gaussian kernel as shown below:

[0071]

[0072] Among them, δ 1 ,δ 2 is the weight adjustment factor, and x represents the pixel set in the current image.

[0073] Calculate the sparse matrix characteristics of optical flow based on GSP and define the angle matrix D, where D = d i,j =Σ j w i,j The combined graph Laplacian matrix L is:

[0074] L=DW=(∑ j w i,j )-w i,j

[0075] According to the matrix characteristics, L is symmetric, so the spectral theorem can be used to prove that it can be characteristically decomposed into L = UΛU T, the eigenvector U can be interpreted as the corresponding graph frequency component. The set of eigenvectors U of L together form the Graph Fourier Transform (GFT).

[0076] The optical flow vector motion field is represented as a flow field matrix, and combined with the set dictionary matrix, the estimated value of the coefficient matrix is ​​obtained:

[0077]

[0078] in: is the estimated value of the coefficient matrix, X is the flow field matrix, D is the dictionary matrix, A is the coefficient matrix, λ is the sparse constraint for regularization constraint, is a regularized calculation method, ||·|| 1 is another regularization calculation method, where a is an element in the coefficient matrix.

[0079] Step S3-2: Introduce the graph Laplace regularization term, combine the energy function of the optical flow estimation, and combine the estimated value of the coefficient matrix to obtain the estimated value of the reconstructed flow field;

[0080] Specifically, a graph signal X is defined on the graph to represent the set of flow field vectors in the image frame. Aperture position distribution constraints are implemented. Based on the sparse coding structure, the graph Laplace regularization term is introduced to implement graph structure constraints, thereby solving the aperture association problem, and replacing E with the energy function of optical flow estimation in the coefficient matrix A. At the same time, a suitable degradation strategy is selected. Therefore, the sparse representation optical flow field minimization solution expression based on GSP analysis can be re-expressed as:

[0081]

[0082] in: is the estimated value of the reconstructed flow field, M is the set of aperture numbers, H is the flow field degradation strategy, V is the reconstructed flow field, γ is the graph Laplace regularization parameter, Tr is the matrix trace, U is the Laplace transform matrix, and L is the matrix after Laplace transform decomposition.

[0083] Step S3-3: Based on the gradient adaptive strategy, a gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap area, and the flow field is reconstructed according to the array arrangement mode for the reconstructed flow field V.

[0084] Specifically, a gradient adaptive strategy is designed to avoid gradient explosion and increase the topological sparsity of the framework. A gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap area. For the reconstructed flow field V, the flow field is reconstructed according to the array arrangement, as shown in the formula:

[0085]

[0086] in: is the reconstructed optical flow motion vector under the pixel set i that matches the motion vector j. The purpose is to suppress the basis vectors at the same position through gradient operation. V ikm is the reconstructed optical flow motion vector of the pixel set i that matches the motion vector k under the graph constraint, V ijn is the reconstructed optical flow motion suppression vector under the pixel set i that matches the motion vector j, V ikn is the optical flow motion suppression vector under the pixel set i that matches the motion vector k under the graph constraint, V ijm is the reconstructed optical flow motion vector of the pixel set i that matches the motion vector j.

[0087] The gradient error can be calculated by:

[0088]

[0089] in: is the gradient value of pixel set i that matches motion vector j, corr 2 To calculate the correlation coefficient operation.

[0090] Not only can the position of discontinuous motion be given greater weight support, but the edge of the aperture can also be constrained, thereby obtaining a robust global optical flow field.

[0091] Step S4: Use a multi-layer perceptron to map the original descriptor into an adaptive description operator through scale-invariant feature transformation to capture the relationship between key points;

[0092] In order to realize adaptive aperture optical flow extraction, SIFT operator is used for feature matching. In order to improve the accuracy of outlier screening, a secondary screening mechanism based on symmetric optical flow is added in the detected dynamic area, that is, the optical flow vector of the feature point at time t is calculated with reference to the previous moment, and the optical flow vector of the feature point at time t is calculated with reference to the moment t. The calculation results must satisfy the characteristics of equal modulus and opposite direction. If the above characteristics are not met, it can be regarded as an outlier and eliminated.

[0093] Use MLP (Multi-layer Perceptron) to map the original descriptor to a new descriptor. Use the Transformer network to embed SIFT key points to capture the complex relationship between key points and enhance the expressiveness of features. Use MLP to approximate the descriptor space and achieve fast SIFT extraction. Use Transformer to capture the spatial context clues of sparse local features extracted from the same image. The entire array aperture flow field calculation process can be expressed as:

[0094]

[0095] The input of the Transformer is N local features in the same image, and the output is an enhanced feature descriptor. With the help of the attention mechanism in the Transformer, all local feature information can be aggregated to form a global context. By integrating this global context information, the local feature descriptor can have a larger receptive field. The sparse attention mode is improved in the following ways:

[0096] By using an efficient sparse improved Transformer to replace the matmul matrix operation in the ordinary Transformer, the connection mode is only used in the calculation of the key (K) or value (V) in the attention calculation, and the element multiplication is used instead of the matrix multiplication, as shown in the formula:

[0097]

[0098] Where: f i (X) is the improved Transformer attention function, σ(Q i ) is the sigmoid function operation, Q i is the key point of the i-th row of Q, K j is the key point of the jth row of Q, V j is the key point of the j-th row of V, softmax is the activation function, and ⊙ is the element-by-element multiplication operation.

[0099] Map the attention query (Q), key (K), or value (V) to a lower dimensional space, thereby reducing the complexity of attention calculation. By introducing a structured sparse pattern, the complexity of attention calculation is reduced by leveraging the sparse structure model based on the graph signal. Use sparse encoders and decoders to transform the attention matrix to reduce the number of attention weights that need to be calculated.

[0100] Through the above steps, the topological sparsity of the optical flow framework is improved by combining sparse representation. Therefore, the lightweight transformer can be applied to the sieve operator to enhance the feature expression, which effectively solves the problems of abnormal valuation and dynamic feature selection in visual tracking tasks and improves the accuracy and robustness of tracking.

[0101] like Figure 2 As shown, the energy function of multi-aperture optical flow can be expressed as:

[0102] E=E R +λE H +γE G +δE T (Basic reconstruction + aperture distribution constraint + topological sparsity)

[0103] Among them, E R represents the basic aperture reconstruction energy function, E H 、EG represents the sparse representation optical flow field and aperture constraint based on GSP analysis, E T Represents the adaptive SIFT screening process, where λ, γ, and δ are weight factors of the previous steps.

[0104] Step S5: local optical flow activation is achieved by combining the relationship between key points and the optical flow sparse matrix.

[0105] Finally, the joint sparse local optical flow activation model of this embodiment is minimized and the iteration termination condition is set to the photometric consistency error change is less than the threshold λ θ =10 -6 .

[0106] After completing the iteration of the optical flow field solution, the final adaptive estimation result of the local optical flow field of the array aperture image can be obtained.

[0107] The following comparative experiments verify the effectiveness of this method. The experimental platform selected is an unmanned vehicle equipped with a multi-aperture visual sensor, a position reference, and a stable gimbal. The algorithm is compared with the non-adaptive SITF screening, LK, and LK+SIFT optical flow estimation methods. Figure 3 This is a visualization of the flow field effect of optical flow estimation. In the flow field estimation results of the first set of multi-aperture images, the method of this example can estimate the optical flow field of each aperture more accurately, while LK has a flow field overflow. Under the adaptive screening of the SIFT operator, LK+SIFT realizes adaptive estimation of motion details, but there is still a certain error, and LK has a large deviation from the true value. The second set of multi-aperture image data is adaptively predicted by the method of this example. It can be seen that the displacement size and direction of each moving target in the scene, and the aperture static motion information is well eliminated. In general, this example can estimate and screen the optical flow field more accurately when there are complex displacements and multiple objects in the multi-aperture image.

[0108] In general, compared with traditional optical flow estimation methods such as variational methods and deep learning, the present invention can process visual motion data without complicated label annotation, ensuring high-precision array flow field, thereby achieving pixel-level processing of flow field estimation errors caused by occlusion and perspective changes. At the same time, it also fully considers the associated motion characteristics between apertures, realizes regional aperture adaptive estimation, and greatly reduces the burden of computing resources. The use of lightweight neural networks to accelerate SIFT feature search can not only improve the accuracy of key point descriptors through a self-enhancement mechanism, but also occupy lower computing resources and be embedded in a multi-aperture array imaging system to achieve morphological feature key point screening, and ultimately achieve the purpose of local robust enhancement of the flow field.

[0109] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

Claims

1. A local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework, characterized in that: include: Step S1: Use a multi-aperture array imaging system to capture a series of images, select a source image and a target image, and calculate an initial optical flow vector motion field; Step S2: Based on the initial optical flow vector motion field, combined with the number and arrangement of apertures of the multi-aperture array imaging system, construct an optical flow vector motion field under a sparse framework; Step S3: Based on the optical flow vector motion field, the optical flow sparse matrix is ​​calculated using the GSP analysis process; Step S4: Use a multi-layer perceptron to map the original descriptor into an adaptive description operator through scale-invariant feature transformation to capture the relationship between key points; Step S5: local optical flow activation is achieved by combining the relationship between key points and the optical flow sparse matrix.

2. According to the method for local optical flow activation of a multi-aperture array imaging system in a sparse topology framework of claim 1, it is characterized in that: The mathematical expression of the optical flow vector motion field is: Where: E(v(x)) is the energy function, C(x,v(x)) is the basic energy function in the optical flow vector motion field, G(v(x)) is the regular energy function in the optical flow vector motion field, x is the gray value of the pixel, and v(x) is the local motion activation flow field.

3. According to the method for local optical flow activation of a multi-aperture array imaging system in a sparse topology framework in claim 1, it is characterized in that: The process of acquiring the optical flow vector motion field under the sparse framework in step S2 includes: Step S2-1: Update the local motion activation flow field based on the number and arrangement of apertures and the current optical flow vector motion field: Step S2-2: updating the optical flow vector motion field according to the updated local motion activation flow field; Step S2-3: Repeat step S2-1 and step S2-2 until convergence, and obtain the optical flow vector motion field under the sparse framework.

4. According to the method for local optical flow activation of a multi-aperture array imaging system in a sparse topology framework of claim 3, it is characterized in that: Update the local motion activation flow field as: Where: v n (x) is the optical flow motion field of a single aperture, and N is the number of apertures in the multi-aperture array imaging system.

5. According to the method for local optical flow activation of a multi-aperture array imaging system in a sparse topology framework in claim 1, it is characterized in that: The step S3 comprises: Step S3-1: Represent the optical flow vector motion field as a flow field matrix, and combine it with the set dictionary matrix to obtain an estimated value of the coefficient matrix: in: is the estimated value of the coefficient matrix, X is the flow field matrix, D is the dictionary matrix, A is the coefficient matrix, λ is the sparse constraint for regularization constraint, is a regularization calculation method, ||·||1 is another regularization calculation method, and a is an element in the coefficient matrix. Step S3-2: Introduce the graph Laplace regularization term, combine the energy function of the optical flow estimation, and combine the estimated value of the coefficient matrix to obtain the estimated value of the reconstructed flow field; Step S3-3: Based on the gradient adaptive strategy, a gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap area, and the flow field is reconstructed according to the array arrangement mode for the reconstructed flow field V.

6. The local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework according to claim 5, characterized in that: The estimated value of the reconstructed flow field is: in: is the estimated value of the reconstructed flow field, M is the set of aperture numbers, H is the flow field degradation strategy, V is the reconstructed flow field, γ is the graph Laplace regularization parameter, Tr is the matrix trace, U is the Laplace transform matrix, and L is the matrix after Laplace transform decomposition.

7. The local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework according to claim 6, characterized in that: The flow field reconstructed in step S3-3 is: in: is the reconstructed optical flow motion vector under the pixel set i that matches the motion vector j. The purpose is to suppress the basis vectors at the same position through gradient operation. V ikm is the reconstructed optical flow motion vector of the pixel set i that matches the motion vector k under the graph constraint, V ijn is the reconstructed optical flow motion suppression vector under the pixel set i that matches the motion vector j, V ikn is the optical flow motion suppression vector under the pixel set i that matches the motion vector k under the graph constraint, V ijm is the reconstructed optical flow motion vector of the pixel set i that matches the motion vector j.

8. The local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework according to claim 7, characterized in that: The step S4 specifically includes: The Transformer network is used to embed the key points of scale-invariant feature transformation to capture the relationship between key points and enhance the expressiveness of features. A multi-layer perceptron is used to approximate the descriptor space to achieve fast extraction of scale-invariant feature transformation.

9. The local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework according to claim 8, characterized in that: The input of the Transformer network is N local features in the same image, and the output is an enhanced feature descriptor.

10. The local optical flow activation method for a multi-aperture array imaging system in a sparse topology framework according to claim 9, characterized in that: In the attention calculation of the Transformer network, the connection mode is only used in the calculation of keys or values, and element multiplication is used instead of matrix multiplication. The mathematical expression is Where: f i (X) is the improved Transformer attention function, σ(Q i ) is the sigmoid function operation, Q i is the key point of the i-th row of Q, K j is the key point of the jth row of Q, V j is the key point of the j-th row of V, softmax is the activation function, K is the key, and ⊙ is the element-by-element multiplication operation.

Images