A video compression sensing reconstruction method based on frequency domain separation

Abstract

This invention relates to the field of video compression and transmission methods, specifically to a video compression-sensory reconstruction method based on frequency domain separation. The method includes: acquiring a video frame sequence and performing compressed-sensory sampling on each frame to obtain measurement values; obtaining an initial reconstruction result for the current frame based on the measurement values; calculating the hidden layer state of the current frame using a hidden layer state calculation module, whereby the hidden layer state includes low-frequency and high-frequency hidden layer states; and fusing the hidden layer states with the initial reconstruction result to obtain the reconstruction result of the video frame. A frequency domain separation strategy is employed to process low-frequency and high-frequency components independently. In the calculation of the low-frequency hidden layer state, an attention mechanism is used to globally correlate low-frequency information between frames, thereby improving the fidelity and stability of low-frequency information. In the calculation of the high-frequency hidden layer state, optical flow calculation is introduced to align and weight the high-frequency components between frames, effectively suppressing the propagation of high-frequency noise.

Description

Technical Field

[0001] This invention relates to the field of video compression and transmission methods, and specifically to a video compression and sensing reconstruction method based on frequency domain separation. Background Technology

[0002] With the development of technologies such as video communication, intelligent surveillance, and unmanned systems, the demand for efficient video compression and high-quality reconstruction is increasing. Traditional video compression and sensing technologies reduce transmission and storage costs by sparsely sampling the original video data and reconstructing it at the decoding end. However, existing methods suffer from severe limitations in reconstruction quality and efficiency when facing high-dynamic scenes, complex backgrounds, and low signal-to-noise ratio environments.

[0003] The video compressed sensing system consists of two parts: an encoder and a decoder. The encoder can be implemented using a single-pixel camera, equivalent to the following equation: y = Φx (1), where Φ is a non-adaptive measurement matrix, and the video frame signal to be observed is x ∈ R. N ×1 The measured value is y∈R M×1 Then the measurement matrix Φ∈R M×N And M << N. M / N is the sampling rate, and this sampling rate is much smaller than the Nyquist sampling rate. As can be seen from equation (1), sampling and compression are combined into one, and the computational complexity at the encoding end is extremely low.

[0004] In order to recover the complete video signal x to be observed from a very small number of sampled samples y, an efficient and accurate compressed sensing reconstruction algorithm is needed.

[0005] Reconstruction algorithms can be divided into two main categories: 1) traditional algorithms based on optimization, and 2) deep learning methods based on neural networks. Traditional algorithms are less commonly used because they usually require iterative solutions and have high computational complexity.

[0006] Deep learning methods can currently be broadly categorized into two main architectures: 1) networks using sliding windows as input, and 2) recurrent neural networks (RNNs). Because sliding window input networks have a limited temporal visibility, RNNs are currently the mainstream approach. The core principle of RNNs is to pass the reconstruction result or hidden state of the current frame to the next frame, thereby enabling the interaction of temporal information and aiding in reconstruction.

[0007] In existing technologies, common methods are reconstructions based on sparse representations and optimization algorithms, such as the Iterative Shrink Thresholding (ISTA) algorithm and block matching methods. These methods typically rely on strong sparsity assumptions, recovering the original video data by optimizing a loss function. However, these methods suffer from the following problems:

[0008] 1. High noise sensitivity: In low signal-to-noise ratio environments, optimization algorithms are easily affected by noise when recovering high-frequency information, leading to blurred or even lost edge details. This phenomenon is particularly pronounced in dynamic video scenes, resulting in poor temporal consistency of the reconstruction results and a significant decrease in visual quality.

[0009] 2. High computational complexity: Traditional methods, such as iterative methods based on sparse optimization, require significant computational resources to solve nonlinear optimization problems, resulting in low reconstruction efficiency and difficulty in meeting the needs of real-time applications. This problem is particularly prominent in high-resolution video processing scenarios.

[0010] 3. Insufficient utilization of inter-frame correlation: Existing methods often ignore the temporal correlation between video frames during decoding, reconstructing each frame independently. This strategy, lacking temporal correlation, fails to fully utilize the motion information between video frames, resulting in low reconstruction quality, especially making it difficult to accurately reconstruct moving targets in complex scenes.

[0011] 4. Insufficient frequency domain information fusion: In practical applications, the processing of high-frequency and low-frequency information is often unified, ignoring the different impacts of high-frequency and low-frequency components on reconstruction quality. High-frequency components are crucial for details and edges, while low-frequency components form the structural basis of the scene. Existing technologies fail to distinguish and optimize the characteristics of these two types of components, resulting in noise mixing with high-frequency details and limiting the reconstruction effect.

[0012] In summary, existing methods struggle to meet the reconstruction quality and efficiency requirements of practical applications in high-dynamic, low-signal-to-noise ratio environments. These issues significantly limit the application of video compressed sensing technology in resource-constrained scenarios such as intelligent transportation, drone surveillance, and public safety. Summary of the Invention

[0013] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a video compression and sensing reconstruction method based on frequency domain separation. Through frequency domain separation, efficient hidden layer state calculation and the introduction of optical flow information, high-quality video reconstruction is achieved.

[0014] This invention protects a video compressed sensing reconstruction method based on frequency domain separation, comprising:

[0015] Acquire a video frame sequence and perform compressed sensing sampling on each frame of the video frame sequence to obtain measurement values;

[0016] Based on the measured values, the initial reconstruction result of the current frame is obtained through initial reconstruction;

[0017] The hidden layer state of the current frame is calculated using the hidden layer state calculation module, wherein the hidden layer state includes a low-frequency hidden layer state and a high-frequency hidden layer state.

[0018] The hidden layer state is fused with the initial reconstruction result to obtain the reconstruction result of the video frame.

[0019] Preferably, the calculation of the low-frequency hidden layer state includes:

[0020] Through attention mechanisms and By fusing the components, the low-frequency hidden layer components are obtained. Features of each pixel It can be calculated using the following formula:

[0021]

[0022] in, for The i-th pixel, for For the j-th pixel, f(·,·) is a binary function mapping, through The implementation is as follows: θ(·) and φ(·) are two univariate linear functions implemented through convolutional layers to calculate the correlation between two points; g(·) is a linear mapping function implemented through a convolutional layer; and C(·) is a normalization function, denoted as . In networks, this can be achieved using softmax.

[0023] and Similarity through The implementation is achieved by θ(·) and φ(·), which are two univariate linear functions implemented through convolutional layers. The kernel size of the convolutional layer is 1×1, the stride is 1, and the padding is 0.

[0024] In the attention mechanism, The linear mapping is Value. As Key, For Query.

[0025] Preferably, the low-frequency component is obtained through a linear low-pass filter, which includes, but is not limited to, a Gaussian blur filter or other linear filtering methods.

[0026] Preferably, the calculation of the high-frequency hidden layer state includes:

[0027] According to optical flow information F t-1 Through motion compensation Align to The result is calculate and Similarity at each point

[0028]

[0029] in, for and The dot product, p(·) and q(·), are two linear functions implemented through two convolutional layers, and sigmoid(·) is the sigmoid function used to restrict the output to the range [0,1]. As a weight, with Perform pixel-by-pixel multiplication, that is, multiply corresponding elements to obtain high-frequency hidden layer components.

[0030] Preferably, the fusion method includes: calculating the hidden layer state. Depend on and Adding them together, we get:

[0031]

[0032] By fusing the hidden layer state and the current frame reconstruction state, With S t The layers are cascaded, and then the fused features are obtained through a convolutional layer, which is used as the input to the next reconstruction module.

[0033] Preferably, the optical flow information F t-1 Obtained through SpyNet network or other optical flow estimation algorithms.

[0034] Preferably, the fused hidden layer state is concatenated with the initial reconstruction result of the current frame, processed by a convolutional layer, and then input into the next reconstruction module to obtain the final reconstruction result of the frame.

[0035] Preferably, the method further includes:

[0036] The hidden layer states of multiple video frames are calculated and fused sequentially, and the reconstructed results of the continuous video are obtained by processing each frame.

[0037] Preferably, the method is applicable to resource-constrained video sensor network scenarios, including but not limited to intelligent transportation, public safety monitoring, field exploration, and emergency rescue.

[0038] Specifically, the present invention solves the problems in the prior art in the following ways:

[0039] 1. Frequency Domain Separation Strategy: This invention employs a frequency domain separation strategy, processing low-frequency and high-frequency components independently. In the calculation of low-frequency hidden layer states, an attention mechanism is used to globally correlate low-frequency information between frames, thereby improving the fidelity and stability of low-frequency information. In the calculation of high-frequency hidden layer states, optical flow calculation is introduced to align and weight high-frequency components between frames, effectively suppressing the propagation of high-frequency noise.

[0040] 2. Optical Flow Estimation and Motion Compensation: To address the issue of insufficient utilization of inter-frame temporal information, this invention introduces optical flow estimation and aligns high-frequency components from the previous frame through motion compensation. This method significantly improves the consistency of inter-frame information, ensuring that moving targets in dynamic scenes can be accurately reconstructed.

[0041] 3. Fusion of Hidden Layer States: This invention resolves the contradiction between high-frequency noise and low-frequency structural information by weighted fusion of low-frequency and high-frequency hidden layer states. The fused hidden layer states combine global and local correlation information between frames, significantly improving the detail and overall consistency of the reconstructed frames.

[0042] 4. High efficiency and real-time performance: In the algorithm implementation, this invention uses a lightweight convolutional neural network to realize key steps such as feature extraction of low-frequency and high-frequency components and optical flow estimation. At the same time, the Sigmoid function is used to constrain the dynamic weights to ensure that the computational complexity is controlled within an acceptable range and meets the requirements of real-time processing.

[0043] Through the synergy of the above-mentioned technical features, the present invention has significant beneficial effects in the following aspects:

[0044] (1) Improved reconstruction quality: Through frequency domain separation and efficient optical flow alignment, the low-frequency components provide clear structural information, while the high-frequency components achieve a balance between detail preservation and noise suppression. Compared with traditional methods, this invention can achieve higher PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) in complex dynamic scenes.

[0045] (2) Enhanced temporal consistency: Optical flow estimation and motion compensation ensure the motion consistency between frames, solve the problem of frame skipping caused by noise or motion distortion, and improve the visual smoothness of video sequences.

[0046] (3) Optimized computational efficiency: Through lightweight network design and efficient dynamic weight calculation, this invention significantly reduces computational complexity. Compared with traditional sparse optimization methods, processing time is reduced by about 50%, enabling real-time processing of high-resolution video.

[0047] (4) Adaptability and robustness: Even in low-bandwidth, high-noise scenarios, this invention can still guarantee the quality of video reconstruction. This makes it extremely valuable in resource-constrained environments such as UAV monitoring, disaster relief, and smart cities.

[0048] In summary, this invention solves the core problems in the prior art at the algorithm level. Through frequency domain separation strategy, optical flow estimation and optimization of hidden layer state, it provides a video compression and perception reconstruction method that combines high quality and high efficiency, bringing a breakthrough improvement to the fields of video communication and intelligent monitoring. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the video compression sensing method of the present invention.

[0050] Figure 2 This is a schematic diagram of the hidden layer state calculation process of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0052] like Figure 1-2 As shown, this invention provides a video compressed sensing reconstruction method based on frequency domain separation, including acquiring a video frame sequence and performing compressed sensing sampling on each frame to obtain measurement values. Preferably, these measurement values ​​can be generated by a single-pixel camera or other low-sampling devices to reduce the complexity and cost of the sampling hardware. After acquiring the measurement values, an initial reconstruction module obtains the initial reconstruction result of the current frame. The initial reconstruction can employ a simple linear interpolation method or other fast reconstruction algorithms.

[0053] Subsequently, the hidden layer state of the current frame is calculated using the hidden layer state calculation module. The hidden layer state is divided into low-frequency and high-frequency hidden layer states. This separation strategy effectively reduces the impact of noise on high-frequency details while preserving low-frequency structural information. Finally, the hidden layer state is fused with the initial reconstruction result to obtain the final reconstruction result of the video frame.

[0054] In one embodiment of the present invention, the encoding end of the present invention: uses an existing method to perform compressed sensing sampling to obtain the measurement value y of each frame. t ∈R M×1 M represents the number of measurements.

[0055] Decoding end: First, through the initial reconstruction module, the measured value y of each frame is... t Recover the initial reconstruction Where h×w represents the spatial dimension of the video frame. Subsequent reconstructions will maintain the h×w spatial dimension until the final output result. Generally, subsequent reconstructions typically consist of multiple network blocks (such as residual blocks). For a specific reconstruction state or several reconstruction states in subsequent reconstructions, the following hidden states are calculated and fused:

[0056] Assume the current reconstruction state and the previous frame reconstruction state are S. t and S t-1 The objective is based on S. t and S t-1 Calculate the hidden layer state containing the effective temporal information of the previous frame. Then combine it with S t Reconstruction of the fusion. To reduce the impact of noise on this step, the present invention processes high-frequency components and low-frequency components separately. The specific steps are as follows:

[0057] First, calculate S t-1 To S t Optical flow information F t-1 The methods that can be used include, but are not limited to, the SpyNet network.

[0058] Then, calculate S. t and S t-1 high frequency components and low frequency components The calculation method is as follows: First, calculate the low-frequency components. Methods that can be used include, but are not limited to, linear low-pass filters and Gaussian blurring. Then, calculate the low-frequency components. and Afterwards, high-frequency components can be obtained through and get.

[0059] Then, the low-frequency hidden layer components are calculated. This is done through an attention mechanism. and By fusing the components, the low-frequency hidden layer components are obtained. Features of each pixel It can be calculated using the following formula:

[0060]

[0061] in, for The i-th pixel, for For the j-th pixel, f(·,·) is a binary function mapping that can be obtained through... The implementation is as follows: θ(·) and φ(·) are two univariate linear functions implemented through convolutional layers, used to calculate the correlation between two points. g(·) is a linear mapping function, which can be implemented through a convolutional layer, and C(·) is a normalization function, which can be set as... In networks, this can be achieved using softmax.

[0062] and Similarity through The implementation is achieved by θ(·) and φ(·), which are two univariate linear functions implemented through convolutional layers. The kernel size of the convolutional layer is 1×1, the stride is 1, and the padding is 0.

[0063] Next, the high-frequency hidden layer components are calculated. This is based on the optical flow information F. t-1 Through motion compensation Align to The result is calculate and Similarity at each point

[0064]

[0065] in, for and The dot product, p(·) and q(·), are two linear functions, which can be implemented using two separate convolutional layers. sigmoid(·) is the sigmoid function, used to restrict the output to the range [0,1]. As a weight, with Perform pixel-by-pixel multiplication, that is, multiply corresponding elements to obtain high-frequency hidden layer components.

[0066] Next, calculate the hidden layer state. Depend on and Adding them together, we get:

[0067]

[0068] Fuse the hidden layer state with the current frame reconstruction state. With S t The layers are cascaded, and then the fused features are obtained through a convolutional layer, which is used as the input to the next reconstruction module.

[0069] For example, in intelligent transportation scenarios, video frame data can be transmitted from roadside cameras to cloud servers, and high-quality video reconstruction with low bandwidth transmission can be achieved through the method of this invention.

[0070] In this invention, the calculation of the low-frequency hidden layer state is one of the key steps in the reconstruction process. Specifically, pixel-level feature information is extracted from the low-frequency components of the current frame and the previous frame, and the low-frequency hidden layer state of the current frame is calculated using formula (2):

[0071] Preferably, the Softmax function can be dynamically adjusted according to computing resources, and an empirical value range of [0.1, 1] is usually selected.

[0072] In the process of acquiring low-frequency components, the present invention preferably uses a linear low-pass filter to filter the video frame. The low-pass filter can include various methods such as Gaussian blurring and mean filtering. The standard deviation σ of the Gaussian blur filter parameters is preferably 1.5 to ensure that the main structural information of the video frame is preserved while effectively filtering out high-frequency noise.

[0073] Taking public safety monitoring as an example, in complex nighttime monitoring scenarios, low-frequency components can clearly retain the outline of large objects while filtering out small light spots and noise, which helps improve the accuracy of target recognition.

[0074] The high-frequency hidden layer state is achieved by aligning the optical flow information. The optical flow information can be calculated by the SpyNet network. After alignment, the high-frequency hidden layer state is calculated using formula (3).

[0075] High-frequency components are calculated using the following formula:

[0076]

[0077] Among them, S t S is the initial reconstruction feature of the current frame. t-1 This is the reconstructed state of the previous frame. and Represents high-frequency components, and The components are low-frequency components. Preferably, the present invention can dynamically adjust the initial reconstruction weight ratio in different application scenarios. For example, in a field exploration environment, higher weights can be assigned to high-frequency components to enhance the edge details of small targets.

[0078] First, to improve the accuracy of video frame reconstruction, this invention separates the hidden layer states into low-frequency and high-frequency hidden layer states and solves them separately. This separation strategy significantly reduces high-frequency information distortion caused by noise accumulation, while ensuring the stability of the low-frequency components.

[0079] Specifically, based on the initial reconstruction results of the current frame and the previous frame, high-frequency components and low-frequency components are calculated separately. The low-frequency components are obtained through a low-pass filter, such as Gaussian blur or mean filtering, to ensure the extraction of the main structural information in the frame. The high-frequency components are obtained by subtracting the low-frequency components from the original frame, thereby preserving edge details and texture information.

[0080] In calculating the hidden layer state, this invention combines an attention mechanism with a motion compensation strategy. Figure 2 As shown, the optical flow information obtained through the optical flow calculation module provides the basis for inter-frame motion compensation. Optical flow can accurately capture the movement of objects between frames, thus supporting the alignment of high-frequency components.

[0081] In processing high-frequency components, this invention aligns the high-frequency components of the previous frame using motion compensation to obtain the aligned high-frequency hidden layer state. Furthermore, a dynamic weighting strategy is employed to calculate the weight matrix based on the similarity of pixels between frames. This dynamic weighting mechanism significantly suppresses the propagation of high-frequency noise and effectively improves the edge detail fidelity of the reconstructed frame.

[0082] Unlike high-frequency processing, the low-frequency hidden layer state is obtained through global pixel-level correlation calculation. Each pixel in the current frame is fused with all pixels in the previous frame. This method captures the dynamic features of low-frequency information through global correlation, enhancing the consistency of information between frames while maintaining the overall image structure.

[0083] This invention significantly improves the quality and efficiency of video compressed sensing reconstruction through a frequency domain separation strategy and efficient hidden layer state calculation. Low-frequency and high-frequency information are processed separately, effectively suppressing noise propagation while retaining more useful temporal information. Through these innovations, this invention is particularly suitable for smart cities, security monitoring, and resource-constrained video sensor networks, providing high-quality reconstructed video even with low bandwidth.

[0084] A key aspect of this invention is the modeling of inter-frame motion using optical flow information. Preferably, the optical flow information is obtained using a SpyNet network or other lightweight optical flow estimation algorithms. The SpyNet network offers significant advantages in resource-constrained scenarios due to its efficiency and low computational complexity.

[0085] Optical flow information F t-1 The role of optical flow is to capture the displacement and deformation relationships between frames, providing a basis for high-frequency component alignment. In urban intelligent transportation systems, cameras are mounted on moving vehicles, resulting in significant motion and displacement between video frames. Optical flow information can be used to precisely align the video details of the previous frame to the current frame, ensuring that the reconstruction process is unaffected by motion.

[0086] For example, in a scene where a vehicle is traveling from west to east, the positions of the road edge and traffic signs from the previous frame will shift in the current frame. The role of optical flow information is to accurately capture this shift, thereby minimizing motion errors between frames.

[0087] In this invention, the fusion of hidden layer states is the core guarantee of reconstruction quality. Low-frequency and high-frequency hidden layer states are weighted and superimposed using formula (4):

[0088] Low-frequency hidden layer states provide key structural information about video frames, while high-frequency hidden layer states enhance detail and edge sharpness.

[0089] The calculated high-frequency and low-frequency hidden layer states are fused together. This fusion operation ensures the full integration of high-frequency and low-frequency information. The final hidden layer state is shown below. Compared with the initial reconstruction result S t After cascading, the data is input into subsequent convolutional layers to further extract features and obtain optimized reconstruction results.

[0090] This invention significantly improves the reconstruction quality of video frames by organically combining frequency domain separation, optical flow estimation, and dynamic weighting strategies. The separation and processing of low-frequency and high-frequency components, along with differentiated processing for different noise sources, enhances the robustness of the algorithm. Even in low-bandwidth environments, it ensures high-quality video transmission, providing effective technical support for fields such as intelligent transportation and public safety monitoring.

[0091] In practical applications, such as in intelligent transportation scenarios, the processing method of this invention can reconstruct high-quality video streams under limited network bandwidth conditions, thereby improving the accuracy of target detection and behavior analysis.

[0092] Preferably, during fusion, the present invention can dynamically adjust the weights of low-frequency and high-frequency components according to the specific scenario. For example, in public safety monitoring, low-frequency components can be used to identify activities over a large area, while high-frequency components can be used to capture subtle anomalies, such as the specific posture or gestures of a suspect.

[0093] The merged hidden layer state The initial reconstruction result S of the current frame t Cascading processing specifically includes the following steps:

[0094] First, convolutional layers are used to reduce the dimensionality and extract features from the cascaded features, thereby minimizing the impact of redundant information and obtaining simplified reconstructed features. Then, these reconstructed features are input into subsequent reconstruction modules to generate the final video frame reconstruction result.

[0095] For example, in disaster emergency scenarios, the image quality captured by cameras deployed on drones may be unstable due to altitude and wind conditions. The fusion strategy of this invention combines the coarse features in the initial reconstruction result with temporal information from the hidden layer state, thereby significantly improving the clarity and accuracy of frame reconstruction.

[0096] A key feature of this invention is its support for frame-by-frame processing of video frame sequences. Each frame in the video frame sequence undergoes a complete process from compressed sensing sampling to reconstruction, ensuring video continuity and quality. By fully utilizing the temporal information between frames, the inconsistency problem of consecutive frames caused by low sampling rates or noise introduction is eliminated.

[0097] Taking field exploration as an example, when multiple cameras are used to monitor the mining environment in real time, video quality may degrade due to bandwidth limitations. This invention, through frame-by-frame processing, can still provide high-quality reconstructed video under limited bandwidth conditions, ensuring the continuity of safety monitoring in the mining area.

[0098] This invention is applicable to resource-constrained video sensor network scenarios, including but not limited to intelligent transportation, public safety monitoring, field exploration, and emergency rescue. Below are some examples of specific application scenarios:

[0099] Intelligent Transportation: Camera systems deployed in intelligent vehicles require real-time video data transmission. However, due to network bandwidth limitations, traditional video transmission methods cannot meet the demands of high-resolution video. This invention utilizes compressed sensing and frequency domain separation technologies to achieve high-quality video reconstruction under low bandwidth conditions, providing support for intelligent transportation.

[0100] Public safety monitoring: In cities, thousands of cameras record large amounts of video data in real time. The method of this invention can significantly reduce the bandwidth requirements for video transmission, while improving the clarity of reconstructed videos, which facilitates subsequent event review and analysis.

[0101] Field exploration: In remote areas such as mining areas and forests, the transmission and storage of video data face challenges due to weak communication network infrastructure. This invention can complete data acquisition with a lower sampling rate and ensure the quality of exploration videos through efficient reconstruction algorithms, providing technical support for environmental monitoring and natural resource management.

[0102] Emergency Rescue: In disaster emergencies, drones and ground-based sensing equipment need to transmit video in real time. This invention can provide high-quality reconstructed video with limited bandwidth, helping rescue command centers to understand the disaster site situation in a timely manner and improve decision-making efficiency.

[0103] Through the above description, this invention demonstrates significant technical advantages in multiple application scenarios. The strategy combining optical flow estimation and frequency domain separation effectively improves video reconstruction quality. Frame-by-frame optimization and fusion of hidden layer states ensures temporal consistency and spatial detail representation in the video. In low-bandwidth, high-noise environments, this invention exhibits excellent robustness and adaptability, providing advanced video processing solutions for smart cities, security monitoring, and other fields.

[0104] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A video compressed sensing reconstruction method based on frequency domain separation, characterized in that, include: Acquire a video frame sequence and perform compressed sensing sampling on each frame of the video frame sequence to obtain measurement values; Based on the measured values, the initial reconstruction result of the current frame is obtained through initial reconstruction; The hidden layer state of the current frame is calculated using the hidden layer state calculation module, wherein the hidden layer state includes a low-frequency hidden layer state and a high-frequency hidden layer state. The hidden layer state is fused with the initial reconstruction result to obtain the reconstructed video frame; the calculation of the low-frequency hidden layer state includes: Through attention mechanisms and By fusing the components, the low-frequency hidden layer components are obtained. Features of each pixel It can be calculated using the following formula: in, Reconstructing the current frame state S t The low-frequency components, Reconstructing state S of the previous frame t-1 The low-frequency components, for The i-th pixel, for For the j-th pixel, f(·,·) is a binary function mapping, through The implementation is as follows: θ(·) and φ(·) are two univariate linear functions implemented through convolutional layers to calculate the correlation between two points; g(·) is a linear mapping function implemented through a convolutional layer; and C(·) is a normalization function, denoted as . In networks, this can be achieved using softmax. and Similarity through The implementation is as follows: θ(·) and φ(·) are two univariate linear functions implemented by a convolutional layer. The convolutional kernel size of the convolutional layer is 1×1, the stride is 1, and the padding is 0. The calculation of the high-frequency hidden layer state includes: According to optical flow information F t-1 Through motion compensation Align to The result is calculate and Similarity at each point in, Reconstructing the current frame state S t High-frequency components, Reconstructing state S of the previous frame t-1 High-frequency components, for and The dot product, p(·) and q(·), are two linear functions implemented through two convolutional layers respectively. sigmoid(·) is the sigmoid function used to restrict the output to the range [0,1]. As a weight, with Perform pixel-by-pixel multiplication, that is, multiply corresponding elements to obtain high-frequency hidden layer components.

2. The method according to claim 1, characterized in that, In the attention mechanism, The linear mapping is Value. As Key, For Query.

3. The method according to claim 1, characterized in that, The low-frequency component is obtained through a linear low-pass filter, which includes, but is not limited to, a Gaussian blur filter or other linear filtering methods.

4. The method according to claim 1, characterized in that, The fusion method includes: calculating the hidden layer state. Depend on and Adding them together, we get: By fusing the hidden layer state and the current frame reconstruction state, With S t Cascade, S t The current frame is reconstructed, and then the fused features are obtained through a convolutional layer and used as the input of the next reconstruction module.

5. The method according to claim 1, characterized in that, The optical flow information F t-1 Obtained through SpyNet network or other optical flow estimation algorithms.

6. The method according to claim 4, characterized in that, The fused hidden layer state is concatenated with the initial reconstruction result of the current frame, processed by a convolutional layer, and then input into the next reconstruction module to obtain the final reconstruction result of the frame.

7. The method according to claim 1, characterized in that, The method further includes: sequentially calculating and fusing the hidden layer states of multiple video frames, processing frame by frame, to obtain the reconstruction result of the continuous video.

8. The video compressed sensing reconstruction method based on frequency domain separation according to any one of claims 1 to 4, characterized in that, The method is applicable to resource-constrained video sensor network scenarios, including but not limited to intelligent transportation, public safety monitoring, field exploration, and emergency rescue.

Images