Local optical flow activation method for multi-aperture array imaging systems under sparse topology framework

Abstract

This invention relates to a local optical flow activation method for multi-aperture imaging systems within a sparse topology framework. This method is used for adaptive motion vector estimation in complex motion scenes within multi-aperture imaging systems. Compared to traditional variational methods and deep learning-based optical flow estimation methods, it eliminates the need for complex labeling to process visual motion data, ensuring high-precision array flow fields. This allows for pixel-level processing of flow field estimation errors caused by occlusion and viewpoint changes. Furthermore, it fully considers the inter-aperture motion characteristics to achieve adaptive estimation of regional apertures, significantly reducing computational resource burden. A lightweight neural network is used to accelerate SIFT feature search, which not only improves the accuracy of keypoint descriptors through a self-enhancing mechanism but also allows for more efficient embedding into multi-aperture array imaging systems with lower computational resources. This enables morphological feature keypoint selection, ultimately achieving robust local enhancement of the flow field.

Description

Technical Field

[0001] This invention relates to a local optical flow activation method, and more particularly to a local optical flow activation method for a multi-aperture array imaging system under a sparse topology framework. Background Technology

[0002] Monocular imaging systems are no longer sufficient for applications in autonomous driving, computer vision, and autonomous navigation. Therefore, multi-view imaging has been widely applied for precise and rapid positioning and measurement to achieve target motion perception, thereby enabling applications such as visual control, directional navigation, high-dynamic target tracking, and obstacle avoidance for unmanned intelligent vehicles. Visual motion perception information is essential for intelligent perception in IoT applications such as unmanned motion platforms. Multi-aperture optical imaging systems integrate fabrication technology, optical design, and machine vision algorithms, providing multi-channel perception and wide-field-of-view high-speed processing compared to traditional single-aperture imaging methods. Extensive research has been conducted on multi-aperture optical imaging systems for various missions, and various array imaging motion perception methods have been applied. The establishment of mathematical models for aperture light field information processing plays a crucial role in bridging the gap between structural design and application during research.

[0003] Single global optical flow estimation has proven inadequate for meeting the motion detection requirements of multi-aperture imaging systems. In practical applications, optical flow information, often affected by static scene information, is a significant source of error, making it difficult to balance computational resources with estimation accuracy. Introducing feature descriptors to assist in optical flow estimation has proven to be a viable approach, significantly improving accuracy and allowing for adaptive selection of optical flow parameters. Examples include ORB, SIFT, ALIKE, and SURF. Both handcrafted and learning-based descriptors have performed well in motion estimation practice, with some becoming default descriptors for certain algorithms. For instance, the binary descriptor ORB is widely used in stereo SLAM systems, and SURF is used for motion structure system reconstruction.

[0004] However, such existing technologies may require more computing power from current hardware, or sometimes require significant modifications to the algorithm model due to changes in the descriptor type, which poses a greater challenge to higher-level visual perception tasks. Summary of the Invention

[0005] The purpose of this invention is to provide a method for local optical flow activation in a multi-aperture array imaging system under a sparse topology framework.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A local optical flow activation method for a multi-aperture array imaging system within a sparse topological framework includes:

[0008] Step S1: Use a multi-aperture array imaging system to capture a continuous array of images, select the source image and the target image, and calculate the initial optical flow vector motion field;

[0009] Step S2: Based on the initial optical flow vector motion field, and combined with the number and arrangement of apertures in the multi-aperture array imaging system, construct the optical flow vector motion field under the sparse framework;

[0010] Step S3: Based on the optical flow vector motion field, calculate the optical flow sparse matrix using GSP analysis;

[0011] Step S4: Use a multilayer perceptron to map the original descriptor to an adaptive descriptor through scale-invariant feature transformation, capturing the relationships between key points;

[0012] Step S5: Combine the relationships between key points and the optical flow sparse matrix to achieve local optical flow activation.

[0013] The mathematical expression for the optical flow vector motion field is:

[0014]

[0015] Where: E(v(x)) is the energy function, C(x,v(x)) is the basic energy function in the optical flow vector motion field, G(v(x)) is the regular energy function in the optical flow vector motion field, x is the gray value of the pixel, and v(x) is the local motion activated flow field.

[0016] The process of obtaining the optical flow vector motion field under the sparse frame in step S2 includes:

[0017] Step S2-1: Based on the number and arrangement of apertures, and combined with the current optical flow vector motion field, update the local motion activation flow field:

[0018] Step S2-2: Activate the flow field based on the updated local motion and update the optical flow vector motion field;

[0019] Step S2-3: Repeat steps S2-1 and S2-2 until convergence, to obtain the optical flow vector motion field under the sparse frame.

[0020] The locally activated flow field is updated as follows:

[0021]

[0022] Where: v n (x) represents the optical flow field of a single aperture, and N represents the number of apertures in the multi-aperture array imaging system.

[0023] Step S3 includes:

[0024] Step S3-1: Represent the optical flow vector motion field as a flow field matrix, and combine it with the set dictionary matrix to obtain the estimated value of the coefficient matrix:

[0025]

[0026] in: Let X be the estimated value of the coefficient matrix, D be the dictionary matrix, A be the coefficient matrix, and λ be the sparse constraint for regularization. is one regularization calculation method, ||·||1 is another regularization calculation method, and a is an element in the coefficient matrix.

[0027] Step S3-2: Introduce the Graph Laplace regularization term, combine it with the energy function of the optical flow estimation, and combine it with the estimated value of the coefficient matrix to obtain the estimated value of the reconstructed flow field;

[0028] Step S3-3: Based on the gradient adaptive strategy, a gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap region. For the reconstructed flow field V, the flow field is reconstructed according to the array arrangement.

[0029] The estimated value of the reconstructed flow field is:

[0030]

[0031] in: For the estimated value of the reconstructed flow field, M is the set of the number of apertures, H is the flow field degradation strategy, V is the reconstructed flow field, γ is the graph Laplace regularization parameter, Tr is the matrix trace, U is the Laplace transform matrix, and L is the matrix after Laplace transform decomposition.

[0032] The reconstructed flow field in step S3-3 is as follows:

[0033]

[0034] in: The reconstructed optical flow motion vector is given by the set of pixels i that match the motion vector j. The purpose is to suppress basis vectors at the same position through gradient operations. ikm V is the reconstructed optical flow motion vector under the set of pixels i that matches the motion vector k under graph constraints. ijn V is the reconstructed optical flow motion suppression vector for the set of pixels i that matches motion vector j. ikn V is the optical flow motion suppression vector under the pixel set i that matches the motion vector k under graph constraints. ijm The reconstructed optical flow motion vector is the set of pixels i that match the motion vector j.

[0035] Step S4 specifically includes:

[0036] By embedding key points of scale-invariant feature transformation using a Transformer network to capture the relationships between key points and enhance the expressive power of features, and by using a multilayer perceptron to approximate the descriptor space, fast extraction of scale-invariant feature transformation is achieved.

[0037] The Transformer network takes N local features from the same image as input and outputs an enhanced feature descriptor.

[0038] In the attention computation of the Transformer network, the connection pattern is only used in key or value computation, and element-wise multiplication is used instead of matrix multiplication. The mathematical expression is as follows:

[0039]

[0040] Where: f i (X) is the improved Transformer attention function, σ(Q) i ) represents the sigmoid function operation, Q i K is the key point in the i-th row of Q. j For the j-th key point of Q, V j Let V be the key point in the j-th row, where softmax is the activation function, K is the key, and ⊙ is the element-wise multiplication operation.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. A multi-aperture topological sparse representation framework is introduced, and sparse coding technology is incorporated into array aperture optical flow motion estimation. A generalized modulation framework for array optical flow is presented, establishing dictionary mapping relationships and learning important motion features in the field of view. This enables pixel-level processing of flow field estimation errors caused by occlusion and viewpoint changes. Compared with traditional variational methods and deep learning optical flow estimation methods, this approach eliminates the need for complex labeling to process visual motion data, ensuring high-precision array flow field estimation.

[0043] 2. Aperture-correlated robust optical flow estimation is adopted. Considering the characteristics of overlapping imaging arrays in multi-aperture imaging systems, GSP (Graph Signal Processing) technology is introduced to further characterize the optical flow motion vector. Combined with sparse coding technology, the flow field is jointly sparsely represented. The correlation motion characteristics between apertures are fully considered, and regional aperture adaptive estimation is achieved, which greatly reduces the burden of computational resources.

[0044] 3. A fast SIFT feature extraction network is introduced, utilizing a lightweight neural network to accelerate SIFT feature search. Compared to traditional feature representation operators, it not only improves the accuracy of keypoint descriptors through a self-enhancing mechanism but also requires less computational resources to be embedded in multi-aperture array imaging systems, enabling adaptive array optical flow extraction. Through morphological feature keypoint selection, local robustness enhancement of the flow field can be achieved. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating the technical principle of the present invention;

[0046] Figure 2 This is an aperture constraint optimization method based on graph signal processing;

[0047] Figure 3 This is a visualization comparison of the optical flow field estimation performance of the multi-aperture array image of the present invention with other methods;

[0048] Figure 4 This is a schematic diagram of the main steps of the present invention. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0050] A local optical flow activation method for a multi-aperture array imaging system within a sparse topological framework, such as... Figure 1 and Figure 4 As shown, it includes:

[0051] Step S1: Use a multi-aperture array imaging system to capture a continuous array of images, select the source image and the target image, and calculate the initial optical flow vector motion field;

[0052] Using a multi-aperture array imaging system to capture a continuous array of images, selecting source and target images, the optical flow vector motion field requires frame-by-frame processing of the multi-aperture images to obtain multiple sets of multi-aperture image sequences at time t and t-1; the vertical and horizontal increments of the compound eye image sequences at time t and t-1 are calculated using the formula I. t (x+v)-I t-1 (x). The initial velocity field is calculated according to the calculation formula, including the initial motion increment in the vertical direction and the initial motion increment in the horizontal direction.

[0053] The mathematical expression for the optical flow vector motion field is:

[0054]

[0055] Where: E(v(x)) is the energy function, C(x,v(x)) is the fundamental energy function in the optical flow vector motion field, G(v(x)) is the regular energy function in the optical flow vector motion field, x is the gray value of a pixel, v(x) is the local motion-activated flow field, and image I t-1 Scene global pixels and image I t The regularization term of the motion vectors in the equation is minimized to obtain the dense motion vector field v. D (x).

[0056] The fundamental energy function in the optical flow vector motion field can be expressed as:

[0057] C(x,v(x))=|I t (x+v)-I t-1 (x)|+β(x)|▽I t (x+v)-▽I t-1 (x)|

[0058] Where: β(x)=1-ρ(|▽I t (x)|), β(x) is a robust adjustment term used to increase the matching robustness of textureless regions in array imaging, ρ is an adjustment factor that controls the weights, I t (x+v) represents the brightness value of pixel x at displacement v at time t, I t-1 (x) represents the brightness value of pixel x at time t-1, ▽I t (x+v) represents the gradient value of pixel x at displacement v at time t, ▽I t-1 (x) represents the gradient value of pixel x at v at time t.

[0059] The method for calculating the canonical energy function in the optical flow vector motion field can be expressed as:

[0060]

[0061] Where: ω(x) is used to penalize discontinuities, D represents the flow field set of the entire image, and v(x) d )-v(x d-1 ) represents the difference between adjacent flow fields.

[0062] Step S2: Based on the initial optical flow vector motion field, and combined with the number and arrangement of apertures in the multi-aperture array imaging system, construct the optical flow vector motion field under the sparse framework;

[0063] The process of obtaining the optical flow vector motion field under the sparse framework includes:

[0064] Step S2-1: Based on the number and arrangement of apertures, and combined with the current optical flow vector motion field, update the local motion activation flow field:

[0065]

[0066] Where: v n (x) represents the optical flow field of a single aperture, and N represents the number of apertures in the multi-aperture array imaging system.

[0067] Step S2-2: Activate the flow field based on the updated local motion and update the optical flow vector motion field;

[0068] Step S2-3: Repeat steps S2-1 and S2-2 until convergence, to obtain the optical flow vector motion field under the sparse frame.

[0069] Step S3: Based on the optical flow vector motion field, calculate the optical flow sparse matrix using GSP analysis, including:

[0070] Step S3-1: Based on the arrangement characteristics of the aperture array in a multi-aperture imaging system, the image information can be represented... N represents the data dimension, and M represents the number of apertures. The image is viewed as a set of nodes, where each node represents a pixel, and edges represent the spatial proximity and similarity between nodes. Graph signal processing (GSP) is used to operate on the graph structure formed by the pixels of the optical flow. A representation method G for array-type aperture optical flow maps is defined. OF (V,Y,W), where V represents a set of N aperture flow field nodes, Y represents the boundary of the aperture set, and W represents the neighbor weight matrix. For an image containing optical flow information, the weight w of the edge connecting nodes i and j is... i,j The Gaussian kernel can be used for calculation, as shown in the following formula:

[0071]

[0072] Where δ1 and δ2 are weight adjustment factors, and x represents the set of pixels in the current image.

[0073] To calculate the sparse matrix characteristics of optical flow based on GSP, we define the angle matrix D, where D = d i,j =Σ j w i,j The combined graph Laplace matrix L is:

[0074] L=DW=(∑ j w i,j )-w i,j

[0075] Based on the matrix properties, L is symmetric. Therefore, it can be proven using the spectral theorem that it can be eigendecomposed into L = UΛU. T The eigenvector U can be interpreted as the corresponding graph frequency components. The set of eigenvectors U of L together forms the Graph Fourier Transform (GFT).

[0076] The optical flow vector motion field is represented as a flow field matrix, and by combining this matrix with a predefined dictionary matrix, the estimated values ​​of the coefficient matrix are obtained:

[0077]

[0078] in: Let X be the estimated value of the coefficient matrix, D be the dictionary matrix, A be the coefficient matrix, and λ be the sparse constraint for regularization. is one regularization calculation method, ||·||1 is another regularization calculation method, and a is an element in the coefficient matrix.

[0079] Step S3-2: Introduce the Graph Laplace regularization term, combine it with the energy function of the optical flow estimation, and combine it with the estimated value of the coefficient matrix to obtain the estimated value of the reconstructed flow field;

[0080] Specifically, a graph signal X is defined on the graph to represent the set of flow field vectors in the image frame. Aperture position distribution constraints are implemented. A graph Laplacian regularization term is introduced based on the sparse coding structure to achieve graph structure constraints, thereby solving the aperture correlation problem. The coefficient matrix A is replaced by the energy function of optical flow estimation, and an appropriate degradation strategy is selected. Therefore, the solution expression for minimizing the sparse representation optical flow field based on GSP analysis can be reformulated as:

[0081]

[0082] in: For the estimated value of the reconstructed flow field, M is the set of the number of apertures, H is the flow field degradation strategy, V is the reconstructed flow field, γ is the graph Laplace regularization parameter, Tr is the matrix trace, U is the Laplace transform matrix, and L is the matrix after Laplace transform decomposition.

[0083] Step S3-3: Based on the gradient adaptive strategy, a gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap region. For the reconstructed flow field V, the flow field is reconstructed according to the array arrangement.

[0084] Specifically, a gradient adaptive strategy was designed to avoid gradient explosion and increase the topological sparsity of the framework. A gradient suppression weight factor was introduced during the energy function solution process to suppress the activation of receptive fields in overlapping aperture regions. For the reconstructed flow field V, flow field reconstruction was performed according to the array arrangement, as shown in the equation:

[0085]

[0086] in: The reconstructed optical flow motion vector is given by the set of pixels i that match the motion vector j. The purpose is to suppress basis vectors at the same position through gradient operations. ikm V is the reconstructed optical flow motion vector under the set of pixels i that matches the motion vector k under graph constraints. ijn V is the reconstructed optical flow motion suppression vector for the set of pixels i that matches motion vector j. ikn V is the optical flow motion suppression vector under the pixel set i that matches the motion vector k under graph constraints. ijm The reconstructed optical flow motion vector is the set of pixels i that match the motion vector j.

[0087] Gradient error can be calculated using the following formula:

[0088]

[0089] in: Let be the gradient value of the pixel set i that matches the motion vector j, and corr2 is the operation for calculating the correlation coefficient.

[0090] It can not only provide greater weight support for the positions of discontinuous motion, but also constrain the edge of the aperture, thereby obtaining a robust global optical flow field.

[0091] Step S4: Use a multilayer perceptron to map the original descriptor to an adaptive descriptor through scale-invariant feature transformation, capturing the relationships between key points;

[0092] To achieve adaptive aperture optical flow extraction, the SIFT operator is used for feature matching. To improve the accuracy of outlier selection, a secondary selection mechanism based on symmetric optical flow is added to the detected dynamic region. This involves calculating the optical flow vector of the feature point at time t with reference to time t, and the optical flow vector of the feature point at time t with reference to time t. The calculation results must satisfy the characteristics of equal modulus and opposite direction. If the above characteristics are not met, the feature point can be regarded as an outlier and rejected.

[0093] The original descriptor is mapped to a new descriptor using an MLP (Multilayer Perceptron). A Transformer network is used to embed SIFT keypoints to capture the complex relationships between them, enhancing feature expressiveness. An MLP is used to approximate the descriptor space, enabling fast SIFT extraction. A Transformer is used to capture spatial contextual cues of sparse local features extracted from the same image. The entire array aperture flow field calculation process can be represented as:

[0094]

[0095] The Transformer takes N local features from the same image as input and outputs an enhanced feature descriptor. By leveraging the attention mechanism in the Transformer, all local feature information can be aggregated to form a global context. Through integrating this global context, the local feature descriptor can have a larger receptive field. The sparse attention pattern is improved in the following ways:

[0096] By replacing the matmul matrix operation in the ordinary Transformer with an efficient sparse improved Transformer, the connection pattern is only used in the key (K) or value (V) computation during attention computation, and element-wise multiplication is used instead of matrix multiplication, as shown in the equation:

[0097]

[0098] Where: f i (X) is the improved Transformer attention function, σ(Q) i ) represents the sigmoid function operation, Q i K is the key point in the i-th row of Q. j For the j-th key point of Q, V j Let V be the key point in the j-th row, where softmax is the activation function and ⊙ is the element-wise multiplication operation.

[0099] The complexity of attention computation is reduced by mapping attention queries (Q), keys (K), or values ​​(V) to a lower-dimensional space. This is further mitigated by introducing structured sparse patterns and leveraging a graph-based sparse structure model. A sparse encoder and decoder are used to transform the attention matrix, reducing the number of attention weights that need to be computed.

[0100] By combining sparse representations through the above steps to enhance the topological sparsity of the optical flow framework, lightweight transformers can be applied to enhance feature representations using sieve operators. This effectively solves the problems of abnormal estimation and dynamic feature selection in visual tracking tasks, thereby improving the accuracy and robustness of tracking.

[0101] like Figure 2 As shown, the energy function of the multi-aperture optical flow can be expressed as:

[0102] E = E R +λE H +γE G +δE T (Basic reconstruction + aperture distribution constraints + topological sparsity)

[0103] Among them, E R E represents the basic aperture reconstruction energy function. H EG E represents the sparse representation of the optical flow field and aperture constraint term based on GSP analysis. T This represents the adaptive SIFT screening process, where λ, γ, and δ are all weighting factors from previous steps.

[0104] Step S5: Combine the relationships between key points and the optical flow sparse matrix to achieve local optical flow activation.

[0105] Finally, the joint sparse local optical flow activation model of this embodiment is minimized, and the iteration termination condition is set as the change in photometric consistency error being less than a threshold λ. θ =10 -6 .

[0106] After completing the iterative solution of the optical flow field, the final adaptive estimation result of the local optical flow field of the array aperture image can be obtained.

[0107] The effectiveness of this method is verified through comparative experiments. The experimental platform used was an unmanned vehicle equipped with a multi-aperture vision sensor, a position reference, and a stabilized gimbal. The algorithm was compared with optical flow estimation methods without adaptive SITF filtering, LK, and LK+SIFT. Figure 3 The visualization of the optical flow estimation results is shown in the first set of multi-aperture images. The method presented in this example can accurately estimate the optical flow field of each aperture, while LK exhibits flow field overflow. Under the adaptive filtering of the SIFT operator, LK+SIFT achieves adaptive estimation of motion details, but still has some error; LK deviates significantly from the true value. The second set of multi-aperture image data is used for adaptive optical flow field prediction using this method. This reveals the magnitude and direction of displacement of each moving target in the scene, and effectively removes static motion information from the apertures. Overall, this example can accurately estimate and filter the optical flow field when multi-aperture images contain complex displacements and multiple objects.

[0108] In summary, compared with traditional variational methods and deep learning optical flow estimation methods, this invention does not require complex labeling to process visual motion data, ensuring high-precision array flow fields. This allows for pixel-level processing of flow field estimation errors caused by occlusion and viewpoint changes. Furthermore, it fully considers the inter-aperture motion characteristics to achieve adaptive estimation of regional apertures, significantly reducing computational resource burden. The lightweight neural network used to accelerate SIFT feature search not only improves the accuracy of keypoint descriptors through a self-enhancing mechanism but also allows for more efficient embedding into multi-aperture array imaging systems with lower computational resources. This enables morphological feature keypoint selection, ultimately achieving robust local enhancement of the flow field.

[0109] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework, characterized in that, include: Step S1: Use a multi-aperture array imaging system to capture a continuous array of images, select the source image and the target image, and calculate the initial optical flow vector motion field; Step S2: Based on the initial optical flow vector motion field, and combined with the number and arrangement of apertures in the multi-aperture array imaging system, construct the optical flow vector motion field under the sparse framework; Step S3: Based on the optical flow vector motion field, calculate the optical flow sparse matrix using GSP analysis; Step S4: Use a multilayer perceptron to map the original descriptor to an adaptive descriptor through scale-invariant feature transformation, capturing the relationships between key points; Step S5: Combine the relationships between key points and the optical flow sparse matrix to achieve local optical flow activation; Step S3 includes: Step S3-1: Represent the optical flow vector motion field as a flow field matrix, and combine it with the set dictionary matrix to obtain the estimated value of the coefficient matrix: in: The coefficient matrix is ​​an estimate. Here is the flow field matrix. It is a dictionary matrix. The coefficient matrix, To implement sparse constraints for regularization constraints, This is a regularization calculation method. This is another regularization calculation method. a These are the elements in the coefficient matrix; Step S3-2: Introduce the Graph Laplace regularization term, combine it with the energy function of the optical flow estimation, and combine it with the estimated value of the coefficient matrix to obtain the estimated value of the reconstructed flow field; Step S3-3: Based on the gradient adaptive strategy, a gradient suppression weight factor is introduced in the process of solving the energy function to suppress the activation of the receptive field in the aperture overlap region. For the reconstructed flow field V, the flow field is reconstructed according to the array arrangement. The estimated value of the reconstructed flow field is: in: To reconstruct the estimated flow field, M The set of the number of apertures, H For flow field degradation strategy, V To reconstruct the flow field, γ For the graph Laplacian regularization parameters, Tr Find the trace of the matrix. U Here is the Laplace transformation matrix. L The matrix is ​​the result of the Laplace transform decomposition; The reconstructed flow field in step S3-3 is as follows: in: The reconstructed optical flow motion vector is given by the set of pixels i that match the motion vector j. The purpose is to suppress basis vectors at the same position through gradient operations. The reconstructed optical flow motion vector is the set of pixels i that matches the motion vector k under graph constraints. The reconstructed optical flow motion suppression vector is the set of pixels i that match the motion vector j. Let i be the set of pixels i that match the motion vector k under graph constraints, representing the optical flow motion suppression vector. The reconstructed optical flow motion vector is the set of pixels i that match the motion vector j.

2. The method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework according to claim 1, characterized in that, The mathematical expression for the optical flow vector motion field is: in: Let be the energy function. Let be the fundamental energy function in the optical flow vector motion field. Let be the canonical energy function in the optical flow vector motion field. x The grayscale value of a pixel. Activate the flow field for local motion.

3. The method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework according to claim 1, characterized in that, The process of obtaining the optical flow vector motion field under the sparse frame in step S2 includes: Step S2-1: Based on the number and arrangement of apertures, and combined with the current optical flow vector motion field, update the local motion activation flow field: Step S2-2: Activate the flow field based on the updated local motion and update the optical flow vector motion field; Step S2-3: Repeat steps S2-1 and S2-2 until convergence, to obtain the optical flow vector motion field under the sparse frame.

4. The method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework according to claim 3, characterized in that, The locally activated flow field is updated as follows: in: For the optical flow field of a single aperture, N This represents the number of apertures in a multi-aperture array imaging system.

5. The method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework according to claim 1, characterized in that, Step S4 specifically includes: By embedding key points of scale-invariant feature transformation using a Transformer network to capture the relationships between key points and enhance the expressive power of features, and by using a multilayer perceptron to approximate the descriptor space, fast extraction of scale-invariant feature transformation is achieved.

6. The method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework according to claim 5, characterized in that, The Transformer network takes N local features from the same image as input and outputs an enhanced feature descriptor.

7. The method for local optical flow activation in a multi-aperture array imaging system under a sparse topological framework according to claim 6, characterized in that, In the attention computation of the Transformer network, the connection pattern is only used in key or value computation, and element-wise multiplication is used instead of matrix multiplication. The mathematical expression is as follows: in: For the improved Transformer attention function, For the sigmod function operation, For the i-th row of Q, For the j-th row of Q, V j This is the key point in the j-th row of V. For activation function, K As key, This is an element-wise multiplication operation.

Images