A method for motion optical flow estimation for low-light scenes

Abstract

This invention discloses a method for estimating motion optical flow in low-light scenes, belonging to the field of computer vision technology. The method includes constructing a spatial domain frequency decomposition encoder to decompose the initial features of the low-light image into high- and low-frequency components, encode local and global motion features respectively, and fuse them. Feature refinement is completed by downsampling and multilayer perceptron (MLP) residual fusion operation. A frequency domain sensing motion enhancement module is constructed to map the spatial domain motion features to the frequency domain for modulation and fusion. After frequency sensing channel attention optimization, enhanced motion features are obtained through gated residual connections. A motion prior sensing attention module is constructed to perform high-dimensional semantic space mapping on the enhanced features of adjacent frames. After calculating semantic similarity weights, prior motion vectors are obtained by coordinate weighted aggregation. A 4D cost volume is constructed based on the enhanced features. Combined with the prior motion vectors, the prior motion vectors are iteratively updated using a gated recurrent unit (GRU) update operator to obtain the motion optical flow estimation result.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for estimating motion optical flow in low-light scenes. Background Technology

[0002] Motion optical flow estimation is one of the fundamental tasks in computer vision. Its core objective is to obtain the motion information of targets in a scene by calculating the motion trajectories of pixels between adjacent frames in an image sequence. It is widely used in many key fields such as autonomous driving, video surveillance, robot navigation, and medical image analysis. In practical applications, low-light environments (such as nighttime, tunnels, and rainy weather) are common. The imaging process in such environments is easily affected by insufficient light, leading to image quality degradation problems such as noise superposition, reduced contrast, and blurred target outlines, posing a serious challenge to the optical flow estimation task.

[0003] Existing general optical flow estimation methods (such as deep learning-based RAFT and FlowNet models) are mainly designed for normal lighting conditions. Their feature extraction and matching mechanisms rely on clear texture information and stable pixel gray-level distribution in the image. However, in low-light scenes, these methods have significant drawbacks: on the one hand, strong noise caused by low light can destroy the gray-level consistency between pixels, interfering with the establishment of inter-frame pixel correspondences and causing traditional optical flow estimation models based on the assumption of constant brightness to fail; on the other hand, target contour degradation and loss of detail caused by insufficient lighting can make it difficult for the model to accurately extract discriminative features, resulting in blurred motion boundaries and enhanced motion discontinuity, ultimately leading to a significant decrease in optical flow estimation accuracy and insufficient robustness.

[0004] Furthermore, some existing improvement methods attempt to adapt to low-light scenarios through image enhancement preprocessing or adjusting the loss function, but these methods often have limitations: image enhancement preprocessing may amplify noise or introduce spurious features, further interfering with optical flow calculation. Simply adjusting the loss function fails to fundamentally resolve the core contradiction between feature extraction and matching in low-light environments, making it difficult to balance estimation accuracy and algorithm real-time performance. Summary of the Invention

[0005] The purpose of this invention is to provide a motion optical flow estimation method for low-light scenes, which can effectively suppress noise interference, enhance effective feature representation, and improve the accuracy and robustness of optical flow estimation.

[0006] To achieve the above objectives, the present invention provides a method for estimating motion optical flow in low-light scenes, comprising the following steps: S1. Construct a spatial domain frequency decomposition encoder to decompose the initial features of the low-light image into high and low frequency components, encode local and global motion features respectively and fuse them, and complete feature refinement through downsampling and multilayer perceptron (MLP) residual fusion operation. S2. Construct a frequency domain sensing motion enhancement module to map spatial motion features to the frequency domain for modulation and fusion. After frequency sensing channel attention optimization, enhanced motion features are obtained through gated residual connections. S3. Construct a motion prior perception attention module, perform high-dimensional semantic space mapping on the enhanced features of adjacent frames, calculate semantic similarity weights, and then obtain the prior motion vector through coordinate weighted aggregation. S4. Construct a 4D cost volume based on the enhanced features, combine it with the prior motion vector, and iteratively update it using the gated recurrent unit (GRU) update operator to obtain the motion optical flow estimation result.

[0007] Preferably, S1 specifically includes the following: S11. The spatial domain frequency decomposition encoder achieves the decomposition of low-frequency and high-frequency components of spatial domain features in low-light images through differential modeling of dual-path convolution combined with dynamic weight segmentation; the dual-path convolution specifically refers to: S111. Use 1×1 convolutions to construct local detail paths and 3×3 dilated convolutions to construct structure-aware paths, respectively used to extract features from the initial features. F Capture local detail information and region structure information in 0; S112. Calculate the difference between the two path outputs and compare the response differences of the two path outputs at each pixel position. This process is represented as follows: (1) In formula (1) Indicates initial features, H Indicates the height of the input image. W Indicates the width of the input image. C Indicates the number of feature channels. This represents a standard convolution with a kernel size of 1×1. This represents a dilated convolution with a kernel size of 3×3 and a dilation rate of 2. This represents the difference in output features between two convolutional paths; The larger the absolute value, the stronger the inconsistency between the local details and the overall structure at that location; conversely, it indicates that they have the same trend of change. S12. Use the Sigmoid function to generate dynamic weights to adaptively adjust the response differences of the dual-path output at each pixel position, and from... F High-frequency and low-frequency components were separated from 0: (2) (3) In formula (2) This represents the Sigmoid activation function. Indicates the weight; in formula (3) This represents element-wise multiplication. Represents high-frequency components, Indicates low-frequency components; S13. Spatial-channel joint feature reconstruction is achieved through a combination of 1×1 and 3×3 convolutions, and residual connections and Sigmoid gating mechanisms are used to extract features from high-frequency features. F H After filtering out structural information related to local motion, the final output is the local motion features. ;for Global motion information is encoded through orientation-aware convolution and residual fusion mechanisms. ; S14. Use a channel-wise weighted adaptive method to fuse local and global motion features in the spatial domain to obtain initial fused features. ; S15. To facilitate subsequent optical flow regression, downsampling is performed. F LG For 1 / 8 resolution, and refinement using residual fusion operations based on a multilayer perceptron (MLP) mechanism: (4) In formula (4) This refers to MLPs based on attention mechanisms. This represents a refined spatial motion feature with a size of 1 / 8 of the input image.

[0008] Preferably, the frequency domain sensing motion enhancement module in S2 sequentially performs position sensing frequency transformation, frequency domain dynamic feature fusion, frequency sensing channel attention optimization, and gated residual connection, as detailed below: S21. Position-aware frequency transformation specifically involves using Fast Fourier Transform to transform the input spatial motion features. F SM Map to the frequency domain and decompose explicitly; use depthwise separable convolution to perform position-encoded modulation of the frequency domain features: (5) (6) In formula (5), Represents the Fast Fourier Transform. This represents the extraction of the real part of the complex tensor. Represents the extraction of the imaginary part of the complex tensor, symbol Indicates feature splicing; in equation (6) This indicates a depthwise separable convolution with a kernel size of 3×3. Represents the frequency domain characteristics of position sensing; S22. Perform frequency domain dynamic feature fusion, specifically... Multiple parallel pointwise convolutions are used to simulate the response characteristics of different frequency components. Dynamic weights are generated through an attention mechanism, and the output features of each convolution are weighted and fused to achieve frequency domain feature modulation. (7) in, This represents the modulated frequency domain feature tensor. K i Indicates the first i One candidate pointwise convolution kernel, w Represents the attention weight vector. This represents a pointwise convolution operation; in obtaining... Then, it is divided into real parts. F R and the virtual part F I And reconstructed into a complex frequency domain tensor Then to Applying a two-dimensional orthogonal inverse fast Fourier transform, we map it back to the spatial domain, retaining only the real part. F spat To eliminate numerical artifacts in complex number operations; S23. Fine-tuning the discriminative power of feature channels through frequency-aware channel attention optimization, specifically by optimizing the spatial features in the initial stage. F spat Perform the Fourier transform again to obtain the frequency domain features. ;calculate The amplitude spectrum is used to obtain the distribution difference between low-frequency motion energy and high-frequency noise; adaptive average pooling is performed on the amplitude spectrum, and channel attention weights are generated through a fully connected layer; finally, the attention weights are... F spat Element-wise multiplication yields F FACA ; S24. Gated residual connections enhance frequency-aware motion features F FACA Characteristics of airspace motion F SM Perform dual-domain feature fusion: (8) Where γ is a learnable parameter, This indicates the enhanced motion characteristics after fusion.

[0009] Preferably, the motion prior perception attention module in S3 is used to solve the feature matching error problem caused by uneven illumination under low light conditions, specifically including the following: S31, Order and These represent the motion features of the first and second frames of the input image sequence, enhanced by the spatial domain frequency decomposition encoder and the frequency domain sensing motion enhancement module, respectively. any region in , its in The corresponding neighborhood in can be represented as ,in N Represents the size of the neighborhood window; through a linear projection matrix. W Q , W K , W v from and Generate queries, keys, and values, mapping input features to a unified high-dimensional semantic space: (9); in, Represents any region The corresponding query vector, Indicates the neighborhood region The key vector, Indicates the neighborhood region The value vector; S32. Utilize an attention mechanism to quantify the similarity between the semantic query of the current frame and the semantic neighborhood of the next frame, generating a discriminative weight matrix to focus on real motion associations in low-light scenes: (10) In formula (10), This represents the spatial coordinate index on the feature map of the current frame. Indicates the spatial coordinate index within the neighborhood of the next frame. C Q for Q The dimension of 0 Represents semantic similarity weights. Indicates the activation function; S33. Obtain semantic similarity weights Then, through coordinate weighted aggregation and displacement calculation, the spatial position change of the feature region in the current frame in the next frame is modeled; specifically, a two-dimensional coordinate graph is created. Record the relative position of each pixel in the feature map, where each position in the current frame... The corresponding coordinates are Then utilize For the next frame's neighborhood coordinate subset Perform a weighted summation to obtain the current frame. Estimated matching coordinates of the region in the next frame ; S34. By calculating the difference between the current frame coordinates and the estimated matching coordinates of the next frame, the prior motion vector can be obtained. : (11).

[0010] Preferably, S4 specifically includes the following: S41. Given the enhanced features obtained by the spatial domain frequency decomposition encoder and the frequency domain sensing motion enhancement module. , The 4D cost volume is constructed by calculating the dot product between all eigenvector pairs: (12) In formula (12), and They represent , Spatial coordinates in This represents the dot product operation. This represents the calculated 4D cost product; S42. Using the obtained 4D cost product, combined with... M P An update operator based on a gated recurrent unit (GRU) is used to iteratively update the optical flow estimation results by retrieving values ​​from the relevant volume: (13) In formula (13) This represents the estimated moving optical flow. K Indicates the number of iterations. The contextual features are obtained from the first frame of the input image sequence through the contextual basic feature extraction module, the spatial frequency decomposition encoder, and the frequency domain sensing motion enhancement module.

[0011] The present invention also provides a motion optical flow estimation system for low-light scenes, comprising: The spatial domain frequency decomposition coding module realizes high and low frequency decomposition of low-light image features, local and global motion feature coding, and feature refinement. The frequency domain sensing motion enhancement module realizes frequency domain enhancement of spatial motion features and dual-domain feature fusion; The motion prior perception attention module enables semantic matching of features from adjacent frames and estimation of prior motion vectors. The motion optical flow estimation module constructs a 4D cost volume and combines it with the GRU update operator to iteratively output the motion optical flow estimation results; The spatial domain frequency decomposition coding module, the frequency domain sensing motion enhancement module, the motion prior sensing attention module, and the motion optical flow estimation module are sequentially connected by signals, and the motion prior sensing attention module and the motion optical flow estimation module are bidirectionally connected by signals.

[0012] Therefore, this invention employs the aforementioned motion optical flow estimation method for low-light scenes, decomposing the spatial features of low-light images into high-frequency components with structural details and low-frequency components with regional motion, and correspondingly encoding local and overall motion features. A frequency-aware motion enhancement module based on Fast Fourier Transform is constructed, projecting the encoded spatial motion features into the frequency domain. Through frequency domain decomposition modulation and dual-domain feature fusion enhancement, noise suppression and contour enhancement are achieved. Finally, by modeling the regional correlation between adjacent frames and estimating prior motion vectors, motion discontinuities caused by brightness fluctuations are mitigated. This method overcomes the problems of low accuracy and poor robustness of general motion optical flow estimation methods in low-light environments.

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0014] Figure 1 This is a diagram of the local motion feature encoding network structure according to an embodiment of the present invention; Figure 2 This is a diagram of the global motion feature encoding network structure according to an embodiment of the present invention; Figure 3 This is the overall network architecture of this invention embodiment; Figure 4 This is the low-light image sequence as input in an embodiment of the present invention; Figure 5 This is a visualization of the optical flow estimation results corresponding to the low-light image sequence input in an embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0016] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0017] Example 1 like Figure 1-3 As shown, in this embodiment, images from the standard low-light optical flow evaluation dataset FCDN are used to experimentally illustrate the motion optical flow estimation method for low-light scenes described in this invention. In this embodiment, for the first frame of the input image sequence... Second frame ,in H Indicates the height of the image, W This represents the width of the image, and 3 represents the number of channels in the image. In this embodiment, we first... I t and I t+1 The dimensions are set to 640×192, that is... W =640, H =192 (below) W and H Both refer to 640 and 192. I t and I t+1 The common input basic feature extraction module extracts initial features. F 0, F The size of 0 is 1 / 4 of the input image size, and the number of channels is... C It is 128, that is (The following) C (All are 128).

[0018] The specific steps are as follows: S1. Construct a spatial domain frequency decomposition encoder to decompose the initial features of the low-light image into high- and low-frequency components, encode local and global motion features respectively, and fuse them. Feature refinement is then achieved through downsampling and multilayer perceptron (MLP) residual fusion operations. Specifically, this includes the following: S11. The spatial domain frequency decomposition encoder achieves the decomposition of low-frequency and high-frequency components of spatial domain features in low-light images through differential modeling of dual-path convolution combined with dynamic weight segmentation; the dual-path convolution is specifically as follows: S111. Use 1×1 convolutions to construct local detail paths and 3×3 dilated convolutions to construct structure-aware paths, respectively used to extract features from the initial features. F Capture local detail information and region structure information in 0; S112. Calculate the difference between the two path outputs and compare the response differences of the two path outputs at each pixel position. This process is represented as follows: (1) In formula (1) Indicates initial features, H Indicates the height of the input image. W Indicates the width of the input image. C Indicates the number of feature channels. This represents a standard convolution with a kernel size of 1×1. This represents a dilated convolution with a kernel size of 3×3 and a dilation rate of 2. This represents the difference in output features between two convolutional paths; The larger the absolute value, the stronger the inconsistency between the local details and the overall structure at that location; conversely, it indicates that they share the same trend of change.

[0019] S12. Use the Sigmoid function to generate dynamic weights to adaptively adjust the response differences of the dual-path output at each pixel position, and from... F High-frequency and low-frequency components were separated from 0: (2) (3) In formula (2) This represents the Sigmoid activation function. Indicates the weight; in formula (3) This represents element-wise multiplication. Represents high-frequency components, This indicates the low-frequency component.

[0020] S13. Spatial-channel joint feature reconstruction is achieved through a combination of 1×1 and 3×3 convolutions, and residual connections and Sigmoid gating mechanisms are used to extract features from high-frequency features. F H After filtering out structural information related to local motion, the final output is the local motion features. The structure of a local motion feature encoding network is as follows: Figure 1 As shown; for Global motion information is encoded through orientation-aware convolution and residual fusion mechanisms. The global motion feature encoding network structure is as follows: Figure 2 As shown.

[0021] S14. Use a channel-wise weighted adaptive method to fuse local and global motion features in the spatial domain to obtain initial fused features. .

[0022] S15. To facilitate subsequent optical flow regression, downsampling is performed. F LG For 1 / 8 resolution, and refinement using residual fusion operations based on a multilayer perceptron (MLP) mechanism: (4) In formula (4) This refers to MLPs based on attention mechanisms. This represents a refined spatial motion feature with a size of 1 / 8 of the input image.

[0023] S2. Construct a frequency-domain sensing motion enhancement module to map spatial motion features to the frequency domain for modulation and fusion. After frequency-sensing channel attention optimization, enhanced motion features are obtained through gated residual connections. The frequency-domain sensing motion enhancement module sequentially performs position-sensing frequency transformation, frequency-domain dynamic feature fusion, frequency-sensing channel attention optimization, and gated residual connections. The specific content is as follows: S21. Position-aware frequency transformation specifically involves using Fast Fourier Transform to transform the input spatial motion features. F SM Map to the frequency domain and decompose explicitly; use depthwise separable convolution to perform position-encoded modulation of the frequency domain features: (5) (6) In formula (5), Represents the Fast Fourier Transform. This represents the extraction of the real part of the complex tensor. Represents the extraction of the imaginary part of the complex tensor, symbol Indicates feature splicing; in equation (6) This indicates a depthwise separable convolution with a kernel size of 3×3. This represents the frequency domain characteristics of location awareness.

[0024] S22. Perform frequency domain dynamic feature fusion, specifically... Multiple parallel pointwise convolutions are used to simulate the response characteristics of different frequency components. Dynamic weights are generated through an attention mechanism, and the output features of each convolution are weighted and fused to achieve frequency domain feature modulation. (7) in, This represents the modulated frequency domain feature tensor. K i Indicates the first i One candidate pointwise convolution kernel, w Represents the attention weight vector. This represents a pointwise convolution operation; in obtaining... Then, it is divided into real parts. F R and the virtual part F I And reconstructed into a complex frequency domain tensor Then to Applying a two-dimensional orthogonal inverse fast Fourier transform, we map it back to the spatial domain, retaining only the real part. F spat This is to eliminate numerical artifacts in complex number operations.

[0025] S23. Although position-aware frequency transformation and frequency-domain dynamic feature fusion compensate for the limitations of spatial domain processing in low-light and complex motion scenes by utilizing the sensitivity of the frequency domain to global information and periodic patterns through frequency domain decomposition and modulation, neither directly performs fine-tuning of the discriminative power of the feature channels. Therefore, fine-tuning of the discriminative power of the feature channels is achieved through frequency-aware channel attention optimization, specifically by optimizing the initially optimized spatial features. F spat Perform the Fourier transform again to obtain the frequency domain features. ;calculate The amplitude spectrum is used to obtain the distribution difference between low-frequency motion energy and high-frequency noise; adaptive average pooling is performed on the amplitude spectrum, and channel attention weights are generated through a fully connected layer; finally, the attention weights are... F spat Element-wise multiplication yields F FACA .

[0026] S24. Gated residual connections enhance frequency-aware motion features F FACA Characteristics of airspace motion F SM Perform dual-domain feature fusion: (8) Where γ is a learnable parameter, This indicates the enhanced motion characteristics after fusion.

[0027] S3. Construct a motion prior awareness attention module to perform high-dimensional semantic space mapping on the enhanced features of adjacent frames, calculate semantic similarity weights, and then obtain the prior motion vector through coordinate weighted aggregation. The motion prior awareness attention module is used to solve the feature matching error problem caused by uneven lighting under low light conditions, and specifically includes the following: S31, Order and These represent the motion features of the first and second frames of the input image sequence, enhanced by the spatial domain frequency decomposition encoder and the frequency domain sensing motion enhancement module, respectively. any region in In this embodiment C Set to 128, it is in The corresponding neighborhood in can be represented as ,in N In this embodiment, the size of the neighborhood window is indicated. N =11; through the linear projection matrix W Q , W K , W v from F ME1 and F ME2 Generate queries, keys, and values, mapping input features to a unified high-dimensional semantic space: (9); in, Represents any region The corresponding query vector, Indicates the neighborhood region The key vector, Indicates the neighborhood region The value vector.

[0028] S32. Utilize an attention mechanism to quantify the similarity between the semantic query of the current frame and the semantic neighborhood of the next frame, generating a discriminative weight matrix to focus on real motion associations in low-light scenes: (10) In formula (10), This represents the spatial coordinate index on the feature map of the current frame. Indicates the spatial coordinate index within the neighborhood of the next frame. C Q for Q The dimension of 0 Represents semantic similarity weights. This represents the activation function.

[0029] S33. Obtain semantic similarity weights Then, through coordinate weighted aggregation and displacement calculation, the spatial position change of the feature region in the current frame in the next frame is modeled; specifically, a two-dimensional coordinate graph is created. Record the relative position of each pixel in the feature map, where each position in the current frame... The corresponding coordinates are Then utilize For the next frame's neighborhood coordinate subset Perform a weighted summation to obtain the current frame. Estimated matching coordinates of the region in the next frame .

[0030] S34. By calculating the difference between the current frame coordinates and the estimated matching coordinates of the next frame, the prior motion vector can be obtained. : (11).

[0031] S4. Construct a 4D cost volume based on enhanced features, combine it with prior motion vectors, and iteratively update using the GRU update operator to obtain the motion optical flow estimation results. S4 specifically includes the following: S41. Given the enhanced features obtained by the spatial domain frequency decomposition encoder and the frequency domain sensing motion enhancement module. , The 4D cost volume is constructed by calculating the dot product between all eigenvector pairs: (12) In formula (12), and They represent , Spatial coordinates in This represents the dot product operation. This represents the calculated 4D cost product.

[0032] S42. Using the obtained 4D cost product, combined with... M P An update operator based on a gated recurrent unit (GRU) is used to iteratively update the optical flow estimation results by retrieving values ​​from the relevant volume: (13) In formula (13) This represents the estimated moving optical flow. K This indicates the number of iterations; in this embodiment, it is set to 12. F c Representing contextual features; in this embodiment ,Depend on It It is obtained through the contextual basic feature extraction module, the spatial frequency decomposition encoder, and the frequency domain sensing motion enhancement module.

[0033] like Figure 3 As shown, this invention combines the aforementioned spatial domain frequency decomposition encoder, frequency domain sensing motion enhancement module, motion prior sensing attention module, and joint... M P A holistic system for motion optical flow estimation in low-light scenes is constructed using a GRU-based update operator. Specifically, it includes: The spatial domain frequency decomposition coding module realizes high and low frequency decomposition of low-light image features, local and global motion feature coding, and feature refinement. The frequency domain sensing motion enhancement module realizes frequency domain enhancement of spatial motion features and dual-domain feature fusion; The motion prior perception attention module enables semantic matching of features from adjacent frames and estimation of prior motion vectors. The motion optical flow estimation module constructs a 4D cost volume and combines it with the GRU update operator to iteratively output the motion optical flow estimation results; The spatial domain frequency decomposition coding module, the frequency domain sensing motion enhancement module, the motion prior sensing attention module, and the motion optical flow estimation module are sequentially connected by signals, and the motion prior sensing attention module and the motion optical flow estimation module are bidirectionally connected by signals.

[0034] In this embodiment, as Figure 4 As shown, taking the low-light image sequence in the FCDN dataset as an example, (a1) and (a2) correspond to the first and second frames of the first group of image sequences, (b1) and (b2) correspond to the first and second frames of the second group of image sequences, and (c1) and (c2) correspond to the first and second frames of the third group of image sequences. All images contain a lot of noise and have low contrast, resulting in a degradation of the target contour. First, the two image sequences are used by the basic feature extraction module to obtain initial features. Then, the initial features are fed into the spatial domain frequency decomposition encoder to decompose high-frequency and low-frequency components, which are then encoded as local and global motion features and fused. Next, the spatial domain motion features are fed into the frequency domain aware motion enhancement module for enhancement. The enhanced features are fed into the motion prior aware attention module to estimate the prior motion vector. On the other hand, they are fed into the 4D cost product to construct the cost product. Finally, the 4D cost product, context features, and prior motion vector are used together with the GRU update operator to estimate the optical flow, such as... Figure 5 As shown, Figure 5 The first column represents the input image; in this embodiment, we take... Figure 4Taking the first frame as an example, the second column shows the visualization results of the actual optical flow; the third and fourth columns are visualizations of the optical flow estimation results estimated by the optical flow estimation methods LRFlow and GMFlowNet, developed from normal illumination image sequences, respectively; and the fifth column is the visualization of the optical flow estimation results in this embodiment. The magnified areas in the figures indicate regions where there are significant differences between the optical flow estimation results of different comparison methods. Figure 5 As can be seen, the optical flow estimated in this embodiment is closer to the true value in areas with thin strip structures or severe contour degradation. This demonstrates that this method is more suitable for optical flow estimation tasks in low-light scenes compared to existing optical flow methods.

[0035] Therefore, this invention employs the aforementioned motion optical flow estimation method for low-light scenes, decomposing the spatial features of low-light images into high-frequency components with structural details and low-frequency components with regional motion, and correspondingly encoding local and overall motion features. A frequency-aware motion enhancement module based on Fast Fourier Transform is constructed, projecting the encoded spatial motion features into the frequency domain. Through frequency domain decomposition modulation and dual-domain feature fusion enhancement, noise suppression and contour enhancement are achieved. Finally, by modeling the regional correlation between adjacent frames and estimating prior motion vectors, motion discontinuities caused by brightness fluctuations are mitigated. This method overcomes the problems of low accuracy and poor robustness of general motion optical flow estimation methods in low-light environments.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for estimating motion optical flow in low-light scenes, characterized in that, Includes the following steps: S1. Construct a spatial domain frequency decomposition encoder to decompose the initial features of the low-light image into high and low frequency components, encode local and global motion features respectively and fuse them, and complete feature refinement through downsampling and multilayer perceptron (MLP) residual fusion operation. S2. Construct a frequency domain sensing motion enhancement module to map spatial motion features to the frequency domain for modulation and fusion. After frequency sensing channel attention optimization, enhanced motion features are obtained through gated residual connections. S3. Construct a motion prior perception attention module, perform high-dimensional semantic space mapping on the enhanced features of adjacent frames, calculate semantic similarity weights, and then obtain the prior motion vector through coordinate weighted aggregation. S4. Construct a 4D cost volume based on the enhanced features, combine it with the prior motion vector, and iteratively update it using the gated recurrent unit (GRU) update operator to obtain the motion optical flow estimation result.

2. The method for estimating motion optical flow in low-light scenes according to claim 1, characterized in that, Specific in S1 Includes the following: S11. The spatial domain frequency decomposition encoder achieves the decomposition of low-frequency and high-frequency components of spatial domain features in low-light images through differential modeling of dual-path convolution combined with dynamic weight segmentation; the dual-path convolution specifically refers to: S111. Use 1×1 convolutions to construct local detail paths and 3×3 dilated convolutions to construct structure-aware paths, respectively used to extract features from the initial features. F Capture local detail information and region structure information in 0; S112. Calculate the difference between the two path outputs and compare the response differences of the two path outputs at each pixel position. This process is represented as follows: (1) In formula (1) Indicates initial features, H Indicates the height of the input image. W Indicates the width of the input image. C Indicates the number of feature channels. This represents a standard convolution with a kernel size of 1×1. This represents a dilated convolution with a kernel size of 3×3 and a dilation rate of 2. This represents the difference in output features between two convolutional paths; The larger the absolute value, the stronger the inconsistency between the local details and the overall structure at that location; conversely, it indicates that they have the same trend of change. S12. Use the Sigmoid function to generate dynamic weights to adaptively adjust the response differences of the dual-path output at each pixel position, and from... F High-frequency and low-frequency components were separated from 0: (2) (3) In formula (2) This represents the Sigmoid activation function. Indicates the weight; in formula (3) This represents element-wise multiplication. Represents high-frequency components, Indicates low-frequency components; S13. Spatial-channel joint feature reconstruction is achieved through a combination of 1×1 and 3×3 convolutions, and residual connections and Sigmoid gating mechanisms are used to extract features from high-frequency features. F H After filtering out structural information related to local motion, the final output is the local motion features. ;for Global motion information is encoded through orientation-aware convolution and residual fusion mechanisms. ; S14. Use a channel-wise weighted adaptive method to fuse local and global motion features in the spatial domain to obtain initial fused features. ; S15. To facilitate subsequent optical flow regression, downsampling is performed. F LG For 1 / 8 resolution, and refinement using residual fusion operations based on a multilayer perceptron (MLP) mechanism: (4) In formula (4) This refers to MLPs based on attention mechanisms. This represents a refined spatial motion feature with a size of 1 / 8 of the input image.

3. The method for estimating motion optical flow in low-light scenes according to claim 2, characterized in that, The frequency domain sensing motion enhancement module described in S2 sequentially performs position-aware frequency transformation, frequency domain dynamic feature fusion, frequency-aware channel attention optimization, and gated residual connection, as detailed below: S21. Position-aware frequency transformation specifically involves using Fast Fourier Transform to transform the input spatial motion features. F SM Map to the frequency domain and decompose explicitly; use depthwise separable convolution to perform position-encoded modulation of the frequency domain features: (5) (6) In formula (5), Represents the Fast Fourier Transform. This represents the extraction of the real part of the complex tensor. Represents the extraction of the imaginary part of the complex tensor, symbol Indicates feature splicing; In equation (6) This indicates a depthwise separable convolution with a kernel size of 3×3. Represents the frequency domain characteristics of position sensing; S22. Perform frequency domain dynamic feature fusion, specifically... Multiple parallel pointwise convolutions are used to simulate the response characteristics of different frequency components. Dynamic weights are generated through an attention mechanism, and the output features of each convolution are weighted and fused to achieve frequency domain feature modulation. (7) in, This represents the modulated frequency domain feature tensor. K i Indicates the first i One candidate pointwise convolution kernel, w Represents the attention weight vector. This represents a pointwise convolution operation; in obtaining... Then, it is divided into real parts. F R and the virtual part F I And reconstructed into a complex frequency domain tensor Then on Applying a two-dimensional orthogonal inverse fast Fourier transform, we map it back to the spatial domain, retaining only the real part. F spat To eliminate numerical artifacts in complex number operations; S23. Fine-tuning the discriminative power of feature channels through frequency-aware channel attention optimization, specifically by optimizing the spatial features in the initial stage. F spat Perform the Fourier transform again to obtain the frequency domain features. ;calculate The amplitude spectrum is used to obtain the distribution difference between low-frequency motion energy and high-frequency noise; adaptive average pooling is performed on the amplitude spectrum, and channel attention weights are generated through a fully connected layer; finally, the attention weights are... F spat Element-wise multiplication yields F FACA ; S24. Gated residual connections enhance frequency-aware motion features F FACA Characteristics of airspace motion F SM Perform dual-domain feature fusion: (8) Where γ is a learnable parameter. This indicates the enhanced motion characteristics after fusion.

4. The method for estimating motion optical flow in low-light scenes according to claim 3, characterized in that, The motion prior perception attention module in S3 is used to address the feature matching error problem caused by uneven lighting under low light conditions. Specifically, it includes the following: S31, Order and These represent the motion features of the first and second frames of the input image sequence, enhanced by the spatial domain frequency decomposition encoder and the frequency domain sensing motion enhancement module, respectively. any region in , its in The corresponding neighborhood in can be represented as ,in N Indicates the size of the neighborhood window; Through linear projection matrix W Q , W K , W v from and Generate queries, keys, and values, mapping input features to a unified high-dimensional semantic space: (9)； in, Represents any region The corresponding query vector, Indicates the neighborhood region The key vector, Indicates the neighborhood region The value vector; S32. Utilize an attention mechanism to quantify the similarity between the semantic query of the current frame and the semantic neighborhood of the next frame, generating a discriminative weight matrix to focus on real motion associations in low-light scenes: (10) In formula (10), This represents the spatial coordinate index on the feature map of the current frame. Indicates the spatial coordinate index within the neighborhood of the next frame. C Q for Q The dimension of 0 Indicates semantic similarity weight. Indicates the activation function; S33. Obtain semantic similarity weights Then, through coordinate weighted aggregation and displacement calculation, the spatial position change of the feature region in the current frame in the next frame is modeled; specifically, a two-dimensional coordinate graph is created. Record the relative position of each pixel in the feature map, where each position in the current frame... The corresponding coordinates are Then utilize For the next frame's neighborhood coordinate subset Perform a weighted summation to obtain the current frame. Estimated matching coordinates of the region in the next frame ; S34. By calculating the difference between the current frame coordinates and the estimated matching coordinates of the next frame, the prior motion vector can be obtained. : (11)。 5. The method for estimating motion optical flow in low-light scenes according to claim 4, characterized in that, S4 specifically includes the following: S41. Given the enhanced features obtained by the spatial domain frequency decomposition encoder and the frequency domain sensing motion enhancement module. , The 4D cost volume is constructed by calculating the dot product between all eigenvector pairs: (12) In formula (12), and They represent , Spatial coordinates in This represents the dot product operation. This represents the calculated 4D cost product; S42. Using the obtained 4D cost product, combined with... M P An update operator based on a gated recurrent unit (GRU) is used to iteratively update the optical flow estimation results by retrieving values ​​from the relevant volume: (13) In formula (13) This represents the estimated moving optical flow. K Indicates the number of iterations; The contextual features are obtained from the first frame of the input image sequence through the contextual basic feature extraction module, the spatial frequency decomposition encoder, and the frequency domain sensing motion enhancement module.

6. A motion optical flow estimation system for low-light scenes, characterized in that, include: The spatial domain frequency decomposition coding module realizes high and low frequency decomposition of low-light image features, local and global motion feature coding, and feature refinement. The frequency domain sensing motion enhancement module realizes frequency domain enhancement of spatial motion features and dual-domain feature fusion; The motion prior perception attention module enables semantic matching of features from adjacent frames and estimation of prior motion vectors. The motion optical flow estimation module constructs a 4D cost volume and combines it with the GRU update operator to iteratively output the motion optical flow estimation results; The spatial domain frequency decomposition coding module, the frequency domain sensing motion enhancement module, the motion prior sensing attention module, and the motion optical flow estimation module are sequentially connected by signals, and the motion prior sensing attention module and the motion optical flow estimation module are bidirectionally connected by signals.

Images