Motion texture combined threshold method

Abstract

The invention provides a motion texture joint threshold (WTJT) method, which comprises the following steps of: firstly, comparing the motion speed and the motion vector of a current frame with the motion speed and the motion vector of a previous frame and a next frame through the sum of absolute differences (SAD) in a frame interpolation information acquisition mode, and screening candidate jump blocks; secondly, extracting two textural features of direction and frequency details from adjacent frames based on a local binary pattern (LBP), and screening candidate hop blocks again in combination with Gabor filtering; and finally, performing spatial-temporal feature decomposition on an input video sequence, establishing a proportional relation between motion information and texture feature information, and calculating a joint weighting coefficient (JWC) so as to dynamically adjust a block jumping threshold value of each frame and realize adaptive allocation of a block jumping proportion. Through united weighting after motion and texture features are normalized, the method provided by the invention improves the accuracy of block jumping judgment, optimizes the distribution of block jumping rate, and achieves balance between algorithm complexity and reconstruction quality.

Description

Technical Field

[0001] The present invention relates to the field of image communication, and in particular to a motion-texture joint threshold method. Background Art

[0002] With the rapid development of wireless visual sensor networks (WVSNs), the demand for real-time video surveillance and transmission is growing. Distributed Video Compressive Sensing (DVCS) technology significantly reduces encoding complexity and transmission costs by dividing video frames into non-overlapping blocks and adopting a block compressive sensing (BCS) strategy, making it a mainstream solution for resource-constrained scenarios.

[0003] However, existing technologies still face significant challenges when processing dynamic scenes (such as fast motion or complex textures). Specifically, existing methods rely on single motion features (such as absolute difference and SAD) or static spatiotemporal correlations to determine block skipping, resulting in significant loss of detail in fast-motion scenes and inefficient transmission of redundant information in static scenes. Furthermore, most methods fail to fully exploit texture similarities between frames. While recent research has attempted to incorporate deep learning, these approaches suffer from high decoding complexity and high hardware resource consumption, making them incapable of meeting the real-time requirements of WVSNs. Setting a precise block skip threshold and assigning an appropriate block skip rate to each frame are key issues in video compressed sensing. Distributed video compressed sensing methods offer promising non-keyframe reconstruction methods, and recent research has focused on optimizing these methods, such as by referencing adjacent frames and using iterative multi-hypothesis frameworks. However, existing methods typically use a fixed block skip threshold, making it difficult to assign an appropriate block skip rate. Therefore, it is necessary to find a method that adaptively optimizes the block skip threshold based on the motion and texture features of different frames in a video sequence, combining them according to specific rules. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: how to improve the accuracy of the block skipping threshold judgment, assign an appropriate block skipping rate to each video frame, and improve the reconstruction quality.

[0005] To solve the above problems, the technical solution of the present invention is to propose a motion texture joint threshold method, which includes the following steps:

[0006] Step 1: Input the video sequence, divide the image into non-overlapping blocks and initialize the block skipping parameters. Divide the video frame into non-overlapping blocks of size 16×16, set the initial block skipping ratio and key frame sampling rate;

[0007] Step 2: Use the sum of absolute differences (SAD) to compare the motion speed of the current frame with the two adjacent frames to obtain motion edge information and motion vectors; preferentially skip blocks with small SAD values ​​and flat motion vectors to reduce detail loss in fast-moving scenes;

[0008] Step 3: Use local binary pattern (LBP) and Gabor filtering to extract the texture features of the current frame, including direction and frequency details, and enhance the correlation between frames;

[0009] Step 4: Joint motion-texture adaptive weight adjustment, perform the following steps:

[0010] Step 4 includes the following specific steps:

[0011] Step 4.1, normalize motion information and texture feature information, and calculate joint weighting coefficients;

[0012] Step 4.2: Dynamically adjust the block skipping threshold based on the joint weighting coefficient;

[0013] Step 5: Skip redundant blocks based on adaptive decision and retain key blocks;

[0014] Step 6: Adjust the current average value of the block skipping rate according to the set block skipping rate value and transmit it to the decoding end;

[0015] Step 7: Reconstruction and optimization;

[0016] Step 7.1: At the decoding end, the weighted residual sparse (RRS) algorithm is used to iteratively reconstruct the video frame;

[0017] Step 7.2: Combine motion-texture information to improve the reconstructed peak signal-to-noise ratio (PSNR);

[0018] Step 8: Output the reconstructed image.

[0019] Preferably, the formula of the sum of absolute differences method used in step 2 is as follows:

[0020]

[0021] Among them, f k (x,y) is the current pixel value, f k-1 (x+a,y+b) is the decoded pixel value, X×Y is the size of the block, and a and b are relative displacements.

[0022] Preferably, the local binary pattern in step 3 is formulated as follows:

[0023]

[0024] Among them, (x c ,yc ) is the coordinate of the center pixel, l n is the current pixel brightness, l c is the brightness of the adjacent pixels, p is the number of adjacent pixels compared to the central pixel, and s is the sign function.

[0025] Preferably, when normalizing the motion information and texture feature information in step 4.1, deviation normalization is adopted, and the formula is as follows:

[0026]

[0027] Among them, V norm is the normalized motion speed, T norm is the normalized texture feature, V max , V min , T max , T min are the maximum and minimum values ​​of motion speed and texture features respectively, and JWC is the joint weighting coefficient.

[0028] Preferably, when adjusting the block skipping threshold in step 4.2, the formula is as follows:

[0029] T adaptive =T base ×(1+α×(1-JWC))

[0030] Among them, T adaptive is the adjusted block skipping threshold, T base is the basic block skipping threshold, and α is the adjustment factor.

[0031] Preferably, when adjusting the block jumping rate mean in step 6, the block jumping rates of all frames are summarized, the deviation between the mean and the target value is calculated, and the block jumping rate scaling factor is derived, which is as follows:

[0032]

[0033] skratio ultimate =skratio initial ×adjust

[0034] Among them, adjust is the scaling factor of the skipping rate, skip specified is the target skipping rate, skip average is the current average block skipping rate, skratio ultimate is the final determined block skipping rate.

[0035] The method proposed in the present invention is different from the existing distributed video compression sensing method. It uses the sum of absolute differences method to determine motion information and motion vectors, uses local binary patterns and Gabor filtering to extract texture feature information, and determines the joint weighting coefficient according to the corresponding weight after normalization. The block skipping threshold of each frame is determined by the difference in motion information and texture feature information between the current frame and the adjacent frame and the joint weighting coefficient, and the corresponding block skipping rate is assigned to different frames. In order to ensure that no requirements are placed on the decoder, this method is implemented at the encoding end through simple information, which not only effectively improves the reconstruction quality, but also does not increase the algorithm complexity. Combining the above measures, the method proposed in the present invention can obtain good reconstruction quality for different video sequences, and has lower algorithm complexity, better overall performance and versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a distributed video compression sensing flow chart;

[0037] Figure 2 It is a reference frame edge information generation framework;

[0038] Figure 3 is a flow chart of the motion-texture joint thresholding method;

[0039] Figure 4 It is the process of dynamically adjusting the block skipping threshold for each frame. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings. It should be understood that these embodiments are intended only to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalents also fall within the scope limited by the appended claims of the application.

[0041] like Figure 1 As shown, distributed video compression sensing is mainly divided into two parts: the encoding end and the decoding end. The technical solution of the present invention proposes a motion texture joint threshold method, including the following steps:

[0042] Step 1: Input a standard football video test sequence. First, each frame is segmented into non-overlapping blocks and the block skipping parameters are initialized. The video frame is divided into non-overlapping blocks of 16×16 size, and keyframes and non-keyframes are determined. The initial block skipping ratio and keyframe sampling rate are set.

[0043] Step 2: The segmented video sequence blocks are then subjected to block-distributed compressed sensing. In the motion speed estimation module, the sum of absolute differences (SAD) is used to compare the motion speed of the current frame with the two adjacent frames to obtain motion edge information and motion vectors. Blocks with small SAD values ​​and flat motion vectors are preferentially skipped to reduce detail loss in fast-moving scenes.

[0044] Step 3. At the same time, the video sequence block will also be used in the feature extraction module to extract the texture features of the current frame, including direction and frequency details, using local binary pattern (LBP) and Gabor filtering to enhance the correlation between frames;

[0045] Step 4: Perform joint motion-texture adaptive weight adjustment on the video sequence block, and perform the following steps:

[0046] Step 4 includes the following specific steps:

[0047] Step 4.1, normalize the motion information and texture feature information, and calculate the joint weighting coefficient according to the size of the two;

[0048] Step 4.2: Modify the basic block skipping threshold in combination with the joint weighting coefficient, dynamically adjust the block skipping threshold, and ensure the accuracy of the block skipping threshold;

[0049] Step 5: Classify different blocks into redundant blocks and key blocks according to the modified block skipping threshold, and decide to skip redundant blocks and retain key blocks;

[0050] Step 6: Adjust the current average value of the block skipping rate according to the set block skipping rate value and transmit it to the decoding end;

[0051] Step 7: Reconstruction and optimization;

[0052] Step 7.1: At the decoding end, the weighted residual sparse (RRS) algorithm is used to iteratively reconstruct the video frame;

[0053] Step 7.2: Combine motion-texture information to improve the reconstructed peak signal-to-noise ratio (PSNR);

[0054] Step 8: Output the reconstructed image.

[0055] Furthermore, the formula of the sum of absolute differences method used in step 2 is as follows:

[0056]

[0057] Among them, f k (x,y) is the current pixel value, f k-1 (x+a,y+b) is the decoded pixel value, X×Y is the size of the block, and a and b are relative displacements.

[0058] Furthermore, the local binary pattern in step 3 is formulated as follows:

[0059]

[0060] Among them, (x c ,y c ) is the coordinate of the center pixel, l n is the current pixel brightness, l c is the brightness of the adjacent pixels, p is the number of adjacent pixels compared to the central pixel, and s is the sign function.

[0061] Furthermore, when normalizing the motion information and texture feature information in step 4.1, deviation normalization is adopted, and the formula is as follows:

[0062]

[0063] Among them, V norm is the normalized motion speed, T norm is the normalized texture feature, V max , V min , T max , T min are the maximum and minimum values ​​of motion speed and texture features respectively, and JWC is the joint weighting coefficient.

[0064] Furthermore, when adjusting the block skipping threshold in step 4.2, the formula is as follows:

[0065] T adaptive =T base ×(1+α×(1-JWC))

[0066] Among them, T adaptive is the adjusted block skipping threshold, T base is the basic block skipping threshold, and α is the adjustment factor.

[0067] Furthermore, preferably, when adjusting the block jumping rate mean in step 6, the block jumping rates of all frames are summarized, the deviation between the mean and the target value is calculated, and the block jumping rate scaling factor is derived, which is formulated as follows:

[0068]

[0069] skratio ultimate =skratio initial ×adjust

[0070] Among them, adjust is the scaling factor of the skipping rate, skip specified is the target skipping rate, skip average is the current average block skipping rate, skratioultimate is the final determined block skipping rate.

[0071] Figure 2 A reference frame edge information generation framework is proposed, which requires that when calculating the block skipping rate of the current frame, the motion information and texture feature information of the two previous and next frames should be referred to at the same time. In addition, when calculating the block skipping rate of the two previous and next frames, the current frame will also serve as a reference frame to provide information for the two previous and next frames.

[0072] like Figure 3 As shown, the process of the motion-texture joint threshold method proposed in the present invention is as follows: input the current video sequence, determine the adaptive block skipping threshold, assign corresponding block skipping rates to different blocks, iteratively reconstruct the video frame at the decoding end, and output the reconstructed image. The determination of the adaptive block skipping threshold and the assignment of the block skipping rate include the following steps:

[0073] (1) Divide the input video sequence into non-overlapping blocks and set the frame sampling rate and basic block skipping parameters.

[0074] (2) Compare the motion information of the current frame with the two adjacent frames. The process of comparing motion information is as follows:

[0075] First, calculate the SAD of the current frame and the previous frame, and then calculate the SAD of the current frame and the next frame. Use motion estimation to get the motion vector, and select the block with small SAD and flat motion vector as the skip block. The calculation formula of SAD is:

[0076]

[0077] (3) Compare the texture feature information of the current frame with the texture feature information of the two adjacent frames. The process of comparing texture feature information is as follows:

[0078] First, LBP is used to extract the texture features of the current frame, and the chi-square distance is used to judge the texture similarity between the current frame and the previous and next frames. Blocks with more texture similarity are selected as skip blocks. The formula of the chi-square distance is:

[0079]

[0080] (4) The obtained motion information and texture feature information are first normalized to facilitate the subsequent weight allocation. The normalization formula is as follows:

[0081]

[0082] (5) Combination Figure 4Since motion information is more important than texture feature information when determining the block skipping threshold, a higher weight should be assigned to motion information. Therefore, the normalized motion information and texture feature information are first compared. Based on the different results, different joint weighting coefficients (JWCs) are determined, and the adaptive block skipping threshold is determined based on the JWCs. The calculation formulas for JWC and the adaptive threshold determination formulas are as follows:

[0083]

[0084] T adaptive =T base ×(1+α×(1-JWC)) (6)

[0085] (6) To obtain the target value of the block skipping rate, the allocated value needs to be continuously corrected during the process of allocating the block skipping rate. The block skipping rate scaling factor is obtained by using the target value and the average value. Finally, the target value can be obtained by using the scaling factor. The correction process formula is as follows:

[0086]

[0087] skratio ultimate =skratio initial ×adjust (8)

[0088] (7) At the decoding end, the weighted residual sparse (RRS) algorithm is used to reconstruct the video frame, the orthogonal matching pursuit (OMP) algorithm is used to generate the initial reconstructed frame, the residual between the current frame and the reference frame is calculated, and the reconstruction result is iteratively updated by minimizing the objective function.

[0089] (8) Finally, the reconstructed video frame is output.

[0090] Experiments using football video sequences were conducted, using the peak signal-to-noise ratio (PSNR) as an objective metric for evaluating the quality of non-keyframe reconstruction. The proposed method outperformed existing distributed video compressed sensing methods in both PSNR and subjective quality. Using the football test sequence, the proposed method achieved an average improvement of 0.3dB compared to existing state-of-the-art methods. The proposed method more accurately determined the adaptive block skipping threshold, ensuring a reasonable block skipping rate distribution. Furthermore, the proposed method leverages lightweight metrics such as SAD and LBP to ensure decoder versatility while maintaining algorithmic complexity.

Claims

1. A motion texture joint threshold method, characterized in that: The following steps are involved: Step 1: Use the sum of absolute differences (SAD) to compare the motion speed of the current frame with the two adjacent frames to obtain motion edge information and motion vectors; preferentially skip blocks with small SAD values ​​and flat motion vectors to reduce detail loss in fast-moving scenes; Step 2: Use local binary pattern (LBP) and Gabor filtering to extract the texture features of the current frame, including direction and frequency details, and enhance the correlation between frames; Step 3: Normalize the motion information and texture feature information and calculate the joint weighting coefficient; Step 4: Combine the joint weighting coefficient to dynamically adjust the block skipping threshold, skip redundant blocks based on adaptive decision, and retain key blocks; Step 5: At the decoding end, the weighted residual sparse (RRS) algorithm is used to iteratively reconstruct the video frame; Step 6: Combine motion-texture side information to improve the reconstructed peak signal-to-noise ratio (PSNR).

2. A motion texture joint threshold method according to claim 1, characterized in that: The formula for the sum of absolute differences method used in step 1 is as follows: Among them, f k (x,y) is the current pixel value, f k-1 (x+a, y+b) is the decoded pixel value, X×Y is the size of the block, and a and b are relative displacements.

3. The motion texture joint threshold method according to claim 1, characterized in that: The local binary pattern in step 2 is formulated as follows: Among them, (x c ,y c ) is the coordinate of the center pixel, l n is the current pixel brightness, l c is the brightness of the adjacent pixels, p is the number of adjacent pixels compared to the central pixel, and s is the sign function.

4. The motion texture joint threshold method according to claim 1, wherein: When normalizing the motion information and texture feature information in step 3, deviation normalization is adopted, and the formula is as follows: Among them, V norm is the normalized motion speed, T norm is the normalized texture feature, V max , V min , T max , T min are the maximum and minimum values ​​of motion speed and texture features respectively, and JWC is the joint weighting coefficient.

5. The motion texture joint threshold method according to claim 1, characterized in that: When adjusting the block skipping threshold in step 4, the formula is as follows: T adaptive =T base ×(1+α×(1-JWC)) Among them, T adaptive is the adjusted block skipping threshold, T base is the basic block skipping threshold, and α is the adjustment factor.

6. The motion texture joint threshold method according to claim 1, characterized in that: When adjusting the block skipping rate mean in step 4, the block skipping rates of all frames are summarized, the deviation between the mean and the target value is calculated, and the block skipping rate scaling factor is derived. The formula is as follows: shortened ultimate =shortened initial ×adjust Among them, adjust is the scaling factor of the skipping rate, skip specified is the target skipping rate, skip average is the current average block skipping rate, skratio ultimate is the final determined block skipping rate.

Images