Video inter-frame redundancy adaptive compression method based on space-time perception graph neural network

Abstract

The invention discloses a video inter-frame redundancy adaptive compression method based on a space-time perception graph neural network, and the method comprises the following steps: S1, obtaining continuous frame image data of a to-be-compressed video, carrying out the preprocessing, and constructing a frame sequence feature tensor; s2, extracting region candidate blocks in each frame of image, and constructing an inter-frame adjacency graph; s3, combining the motion vector and the local gradient features to construct a space-time perception feature map structure; s4, executing graph convolution calculation, and outputting a redundant feature score value; s5, setting a compression threshold, and generating an inter-frame redundancy mask; s6, performing variable bit rate quantization compression and reference frame reconstruction compression on the region candidate blocks with different redundancies to generate compressed frame data; according to the method, the space-time perception graph neural network is introduced, so that accurate recognition and adaptive compression of the redundant region between the video frames are realized, and the compression efficiency and the image restoration quality are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video coding and compression technology, and particularly relates to a video inter-frame redundancy adaptive compression method based on a spatio-temporal perception graph neural network. BACKGROUND

[0002] Under the background of rapid development of video coding and compression technology, in order to reduce the storage and transmission cost of video data, the academic and industrial circles have proposed various video compression methods. Traditional video compression technologies, such as H.264 and H.265 standards, mainly rely on block matching motion compensation, intra-frame and inter-frame prediction, and transform quantization mechanisms to realize compression by reducing temporal redundancy and spatial redundancy. However, these methods are generally based on fixed rules and linear models, and have weak adaptability to complex motion scenes and unstructured areas, which can easily cause compression distortion and visual quality degradation.

[0003] In recent years, deep learning technology has been introduced into the field of video compression, which uses convolutional neural networks to model the similarity and redundancy between images, and improves the compression rate and restoration quality. Some methods achieve compression control by introducing attention mechanisms or using end-to-end compression neural networks. However, the convolutional structure is naturally suitable for smooth areas in Euclidean space, and has limited ability to model complex non-Euclidean spatio-temporal relationships between video frames, making it difficult to effectively handle video content with irregular motion, occlusion changes or semantic mutations.

[0004] In addition, in the redundancy area identification process, existing methods mostly use global features or static thresholds for compression decision-making, lacking fine modeling ability at the region level. The redundancy estimation at the region granularity is not accurate, which can easily lead to over-compression of important areas or insufficient compression of low-value areas, limiting the further improvement of overall compression efficiency.

[0005] Currently, some researches have tried to introduce graph neural networks to model the structural relationship between video frames, but most of them focus on video target detection and behavior recognition tasks, and there is still a lack of systematic methods in the compression field that combine spatio-temporal perception and graph structure optimization. Especially in how to extract the spatio-temporal structural features of region candidate blocks, how to construct the graph structure representation, and how to realize the linkage of adaptive redundancy discrimination and compression strategy, there are still obvious deficiencies.

[0006] Therefore, how to provide a video inter-frame redundancy adaptive compression method based on a spatio-temporal perception graph neural network is a problem that those skilled in the art need to solve. SUMMARY

[0007] One purpose of the present application is to propose a video inter-frame redundancy adaptive compression method based on a spatio-temporal perception graph neural network, which fully utilizes the ability of the graph neural network to model non-Euclidean structures and the fusion expression of the spatio-temporal features of video frames, and describes in detail the whole process from region candidate block extraction, adjacency graph construction, redundancy score calculation to compression control execution, which has the advantages of high compression rate, low image quality loss and strong adaptability.

[0008] The video inter-frame redundancy adaptive compression method based on a spatio-temporal perception graph neural network according to the embodiment of the present application comprises the following steps:

[0009] S1, acquiring continuous frame image data of a video to be compressed, and performing preprocessing to extract pixel feature information of each frame image and construct a frame sequence feature tensor;

[0010] S2, based on the frame sequence feature tensor, extracting region candidate blocks in each frame image, and calculating spatial position offsets and pixel similarities between adjacent frames to construct an inter-frame adjacency graph;

[0011] S3, based on the inter-frame adjacency graph, combining the motion vector and local gradient feature of each region candidate block in the frame sequence to construct a spatio-temporal perception feature graph structure, and generating a graph structure representation with edge weights;

[0012] S4, inputting the spatio-temporal perception feature graph structure into a graph neural network for graph convolution calculation, and outputting redundancy feature score values of each region candidate block;

[0013] S5, setting a compression threshold based on the redundancy feature score values, generating an inter-frame redundancy mask, and screening out image regions that need to be compressed and regions that do not need to be compressed;

[0014] S6, according to the inter-frame redundancy mask, performing variable bit rate quantization compression on region candidate blocks identified as high redundancy, and performing reference frame reconstruction compression on region candidate blocks identified as low redundancy, to generate compressed frame data;

[0015] S7, combining the compressed frame data in the order of the frame sequence, and outputting the compressed video frame sequence.

[0016] Optionally, the continuous frame image data comprises a video frame sequence arranged in time sequence, and each frame image is in the form of a two-dimensional matrix with fixed resolution.

[0017] Optionally, the preprocessing comprises frame processing, size unification, grayscale normalization and noise filtering operations.

[0018] Optionally, the region candidate block is a set of image sub-blocks obtained by dividing the frame sequence feature tensor according to a fixed size and a sliding step, each image frame is divided into a plurality of region candidate blocks, each region candidate block retains the frame number, spatial coordinate information and corresponding pixel feature, and is used for inter-frame similarity calculation and graph structure construction.

[0019] Optionally, the S2 specifically comprises:

[0020] S21, performing frame-by-frame feature extraction on the preprocessed video frame image sequence to obtain a pixel feature matrix of each frame image;

[0021] S22, dividing each frame image according to a fixed size and a sliding step to obtain a plurality of region candidate blocks, each region candidate block retaining its pixel feature and position coordinates in the frame;

[0022] S23, for the region candidate blocks in any adjacent frames, extracting the center coordinates and calculating the position offset as an inter-frame motion indicator;

[0023] S24, calculating the inter-frame pixel similarity score based on the feature difference degree of the two candidate blocks at each pixel point;

[0024] S25, measuring the similarity between all candidate blocks according to the position offset and the pixel similarity score, and constructing an inter-frame adjacency graph with the candidate blocks as nodes, the connection edges in the inter-frame adjacency graph are determined according to a preset threshold, and the edge weight representing the similarity is added.

[0025] Optionally, the S3 specifically comprises:

[0026] S31, on the basis of the constructed inter-frame adjacency graph, extracting the inter-frame motion vector of each region candidate block in each frame and recording the displacement in the horizontal direction and the vertical direction;

[0027] S32, performing gradient calculation on each region candidate block to obtain the average gradient value of the local region of the image, which is used to measure the structure change degree;

[0028] S33, combining the motion vector of each region candidate block with the local gradient feature to form a feature vector;

[0029] S34, for any pair of connected region candidate blocks in the inter-frame adjacency graph, calculating the edge weight according to the difference degree of the feature vectors between the two, the edge weight representing the similarity degree of the region candidate block pair in the space-time structure;

[0030] S35, combining all region candidate blocks and connection relationships and edge weights into a graph structure representation, i.e. a space-time perception feature graph.

[0031] Optionally, the S4 specifically comprises:

[0032] S41, inputting a space-time perception graph structure comprising node features and edge weights to a graph convolutional neural network, and setting an initial feature vector of each region candidate block to be composed of motion vectors and local gradient features;

[0033] S42, in the graph convolutional neural network, performing multi-layer information transmission and aggregation operations according to edge connection relationships between nodes and corresponding edge weights, and repeatedly updating graph feature representations of the nodes;

[0034] S43, in a graph convolution output stage, inputting a final graph feature of each region candidate block to a redundancy score function to generate a redundancy score value of each region candidate block, the redundancy score value being used to represent whether the region has compressibility, and the redundancy score value being a continuous value in a range of 0 to 1.

[0035] Optionally, the S5 specifically includes:

[0036] S51, performing statistical processing on redundancy feature score values of all region candidate blocks, calculating a mean value and a standard deviation, and setting a fixed compression threshold based on an adjustment factor;

[0037] S52, performing threshold comparison on the redundancy feature score value of each region candidate block to generate a binary mask, and marking a region candidate block with a redundancy feature score value not lower than the compression threshold as a region to be compressed, and marking the rest as a region to be retained;

[0038] S53, combining two-dimensional masks of region candidate blocks in all frames into an inter-frame redundancy mask matrix, which is used to indicate image region screening results in a compression process.

[0039] Optionally, the binary mask generates a marked value of 0 or 1 according to a size relationship between the redundancy feature score value of each region candidate block and the compression threshold, and indicates whether the region candidate block needs to perform a compression operation.

[0040] Optionally, the S6 specifically includes:

[0041] S61, judging each region candidate block according to the inter-frame redundancy mask matrix, if the region candidate block is marked as a high redundancy region, performing variable bit rate quantization compression, and if the region candidate block is marked as a low redundancy region, performing reference frame reconstruction compression;

[0042] S62, in the variable bit rate quantization process, using a corresponding quantization coefficient to scale and round pixel features of an image block to obtain a compressed expression of a high redundancy block;

[0043] S63, in the reference frame reconstruction compression process, selecting a similar region block in a previous frame as a reference, calculating a residual compensation value between the current block and the reference block, and performing weighted reconstruction on the reference block.

[0044] S64, combine all compressed blocks and reconstructed blocks according to the frame number and block number to which they belong into a new compressed frame data sequence.

[0045] The beneficial effects of the present application are:

[0046] Firstly, by means of the inter-frame feature division of the region granularity and the adjacent graph construction mechanism, the present application can accurately capture the spatial position offset and pixel similarity between video frames, providing a structural basis for subsequent spatio-temporal feature fusion, and effectively improving the local accuracy of redundancy recognition.

[0047] Secondly, by constructing a spatio-temporal perception graph structure that fuses motion vectors and gradient features, and completing multi-layer graph convolution propagation in the graph neural network, the modeling capability for non-linear motion relationships and semantic redundancy of video frames is significantly enhanced, realizing region-level redundancy scoring and compression judgment.

[0048] Finally, combined with the redundancy scoring results and the adaptive threshold, a mask matrix is generated, and then the variable bit rate quantization compression and reference frame reconstruction compression strategy is executed. The present application improves the compression rate while maintaining the subjective perception quality of the image, has stronger inter-frame structure adaptive ability and universality, and is suitable for efficient compression needs in various complex video scenes. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation on the present application. In the drawings:

[0050] Figure 1 a flowchart of the video inter-frame redundancy adaptive compression method based on the spatio-temporal perception graph neural network proposed by the present application;

[0051] Figure 2 a video preprocessing and graph structure construction flowchart of the video inter-frame redundancy adaptive compression method based on the spatio-temporal perception graph neural network proposed by the present application;

[0052] Figure 3 a redundancy scoring and mask generation flowchart of the video inter-frame redundancy adaptive compression method based on the spatio-temporal perception graph neural network proposed by the present application;

[0053] Figure 4 a compression strategy execution and result output flowchart of the video inter-frame redundancy adaptive compression method based on the spatio-temporal perception graph neural network proposed by the present application; DETAILED DESCRIPTION

[0054] The application will be described in further detail below with reference to the drawings. These drawings are simplified schematic diagrams and only show the basic structure of the application in a schematic manner, and thus only show the components relevant to the application.

[0055] Reference Figures 1-4 The video inter-frame redundancy adaptive compression method based on a space-time perception graph neural network comprises the following steps:

[0056] S1. Obtain continuous frame image data of a video to be compressed, and perform preprocessing to extract pixel feature information of each frame image and construct a frame sequence feature tensor;

[0057] S2. Based on the frame sequence feature tensor, extract a region candidate block in each frame image, and calculate the spatial position offset and pixel similarity between adjacent frames to construct an inter-frame adjacency graph;

[0058] S3. Based on the inter-frame adjacency graph, combine the motion vector and local gradient feature of each region candidate block in the frame sequence to construct a space-time perception feature graph structure and generate a graph structure representation with edge weights;

[0059] S4. Input the space-time perception feature graph structure into a graph neural network to perform graph convolution calculation and output a redundancy feature score value of each region candidate block;

[0060] S5. Set a compression threshold based on the redundancy feature score value, generate an inter-frame redundancy mask, and screen out image regions that need to be compressed and regions that do not need to be compressed;

[0061] S6. According to the inter-frame redundancy mask, perform variable bit rate quantization compression on the region candidate blocks identified as high redundancy, and perform reference frame reconstruction compression on the region candidate blocks identified as low redundancy, to generate compressed frame data;

[0062] S7. Combine the compressed frame data in the order of the frame sequence to output a compressed video frame sequence.

[0063] The application realizes accurate identification and adaptive compression of inter-frame redundancy regions of a video by constructing a complete compression process and combining a graph neural network and a space-time perception mechanism, thereby significantly improving compression efficiency and image fidelity.

[0064] In this embodiment, the continuous frame image data comprises a video image frame sequence arranged in chronological order, and each frame image is in the form of a two-dimensional matrix with a fixed resolution.

[0065] The application uses a fixed resolution frame sequence as input, which guarantees the normativity and consistency of feature extraction and structure division, and is conducive to high-quality modeling of subsequent graph structures.

[0066] In the embodiment, the preprocessing process comprises: performing frame processing on the obtained video frame image, splitting the original video into single frame images arranged in time sequence; performing size unification operation on each frame image to adjust the image to a preset standard resolution; performing gray scale normalization processing on the image after size unification to normalize the pixel value to the interval [0, 1]; and applying a median filter to the normalized image to perform noise filtering, remove high-frequency noise interference and retain edge information, to obtain a preprocessed image frame sequence.

[0067] The application effectively eliminates interference factors in the original image through the preprocessing process of image frame processing, size unification, gray scale normalization and noise filtering, and provides a clear and stable feature basis for redundancy detection.

[0068] In the embodiment, the region candidate block is a set of image sub-blocks obtained by dividing the frame sequence feature tensor according to a fixed size and a sliding step length, each image frame is divided into a plurality of region candidate blocks, and each region candidate block retains the frame number, spatial coordinate information and corresponding pixel feature thereof, and is used for inter-frame similarity calculation and graph structure construction.

[0069] The application divides the frame image according to a fixed size and a sliding step length, improves the structure expression capability of the region candidate block, and provides an accurate input unit for subsequent similarity calculation and graph structure construction.

[0070] In the embodiment, the S2 specifically comprises:

[0071] S21, performing frame extraction operation on the frame sequence feature tensor T to obtain the pixel feature matrix F of each frame image i , satisfying wherein H is the image height, W is the image width, and C is the feature channel number;

[0072] S22, dividing the pixel feature matrix F of each frame image i into a plurality of region candidate blocks B i,m according to a sliding window size k x k and a sliding step length s, wherein m is the candidate block number, represents the region candidate block numbered m in the i-th frame image in the video, and k represents the edge length size of the region candidate block;

[0073] S23, for the region candidate blocks B i,m and B i+1,n in adjacent frames i and i+1, calculating the spatial position offset Δx=x i+1,n -x i,m and Δy=y i+1,n -y i,m , wherein (x i,m , y i,m ) and (x i+1,ny i+1,n is the position center coordinate of the corresponding candidate block in the frame;

[0074] S24, based on the region candidate block B i,m and the pixel feature of B i+1,n , the pixel similarity score S

[0075]

[0076] is calculated, wherein S i,m,n represents the pixel similarity score between the mth region candidate block in the ith frame and the nth region candidate block in the i+1th frame, and the larger the value is, the more similar the two region candidate blocks are, B i,m (p,q,c) represents the pixel feature value of the pth row, qth column and cth channel of the mth region candidate block in the ith frame, B i+1,n (p,q,c) represents the pixel feature value of the pth row, qth column and cth channel of the nth region candidate block in the i+1th frame;

[0077] S25, based on the spatial position offset Δx, Δy and the pixel similarity score S i,m,n , the adjacency relationship between the region candidate blocks is defined, and the inter-frame adjacency graph G=(V, E) is constructed, wherein the node set V represents all region candidate blocks, the edge set E represents the inter-frame edge relationship between the region candidate blocks that meet the threshold condition, and each edge is attached with an edge weight w i,m,n =S i,m,n .

[0078] The present application constructs an inter-frame adjacency graph by spatial position offset and pixel similarity score, so that the graph structure more accurately reflects the inter-frame motion correlation, and effectively enhances the structural perception ability of the region redundant information. In the embodiment, the S3 specifically includes:

[0079] S31, based on the inter-frame adjacency graph G=(V, E), the motion vector of each node B i,m ∈V is extracted wherein Δx i,m and Δy i,m represent the inter-frame displacement values of the mth region candidate block in the ith frame in the horizontal direction and the vertical direction;

[0080] S32, the local gradient feature g i,m of the region candidate block B i,m is calculated:

[0081]

[0082] wherein B i,m(p,q) represents the gray value of the pixel in the pth row and qth column in the mth region candidate block, and k represents the side length dimension of the region candidate block;

[0083] S33, the motion vector is combined with the local gradient feature g i,m to form a candidate block feature vector h i,m . i,m i,m i,m .

[0084] S34, for any adjacent region candidate blocks B i,m and B j,n , and (B i,m , B j,n ) ∈ E, the edge weight w i,m,j,n is calculated:

[0085]

[0086] wherein B j,n represents the region candidate block numbered n in the jth frame image in the video, α is a positive scaling coefficient, used to control the influence degree of feature difference on the edge weight, ‖·‖2 represents the Euclidean distance norm, exp(·) represents the natural exponential function with e as the base, h i,m represents the spatiotemporal perception feature vector of the region candidate block B i,m , h j,n represents the spatiotemporal perception feature vector of the region candidate block B j,n , and has the same form as h i,m .

[0087] S35, the node set V, the edge set E and the corresponding edge weight w i,m,j,n are combined to synthesize a spatiotemporal perception feature graph structure, and the graph structure is represented as G ′ =(V,E,W), wherein W is the set of all edge weights.

[0088] The application fuses the motion vector and the gradient feature to construct the node feature, generates the edge weight according to the feature difference, forms the spatiotemporal perception graph structure, and effectively improves the expression ability and the generalization ability of the interframe graph modeling.

[0089] In the embodiment, the S4 specifically comprises:

[0090] S41, input the graph structure represented as G ′ =(V,E,W) into a graph convolutional neural network, and set the initial feature vector of each node B i,m ∈ V as , which is defined as: wherein Δx i,m and Δy i,m ​​denotes the inter-frame displacement value of the mth region candidate block in the i-th frame in the horizontal and vertical directions, g i,m denotes the local gradient feature;

[0091] S42, in the l-th layer graph convolution, the message passing and aggregation are performed by using the adjacency relationship of the nodes and the edge weight, and the node B i,m the l-th layer feature representation of the node B is updated according to the following formula:

[0092]

[0093] wherein, denotes the set of nodes adjacent to the node B i,m , w i,m,j,n denotes the edge weight, W (l) is the trainable weight matrix of the l-th layer, is the (l-1)-th layer feature representation of the node B j,n , and sigma is a nonlinear activation function;

[0094] S43, after the L-layer convolution of the graph neural network is completed, the final feature representation of the node B i,m is input into a redundancy scoring function f, to obtain a redundancy feature score value of the region candidate block, and the calculation formula is:

[0095]

[0096] wherein, r i,m denotes the redundancy feature score value of the region candidate block B i,m , which is a continuous real number ranging from 0 to 1, w r is a redundancy scoring weight vector, b r is a bias term, and Sigmoid is an activation function.

[0097] S44, all region candidate blocks are counted, and the redundancy feature score value of each region candidate block is output.

[0098] According to the multi-layer propagation mechanism of the graph convolutional neural network, the region redundancy features are aggregated and refined to generate a continuous score result, so that the refinement and quantification of the region compression decision are realized.

[0099] In the embodiment, the S5 specifically includes:

[0100] S51, the redundancy feature score value of each region candidate block is input into a threshold compression function to obtain a compression threshold tau:

[0101] tau = mu + lambda * sigma;

[0102] ​Wherein, μ is the average value of all redundant feature score values, σ is the standard deviation, λ is the coefficient adjustment factor, and the compression threshold τ is a fixed value.

[0103] S52, generating a region candidate block B i,m For example, according to the redundant feature score value r i,m The comparison result with the compression threshold τ generates a binary mask M i,m :

[0104]

[0105] Wherein, M i,m =1 indicates that the mth region candidate block in the ith frame is identified as a high redundancy region, and the compression operation needs to be performed, otherwise no compression is needed.

[0106] S53, combining the two-dimensional mask of the region candidate block in all frames into an inter-frame redundancy mask matrix M, which is used to identify the compression determination result of all region candidate blocks in the video sequence.

[0107] The present application sets an adaptive compression threshold based on statistical parameters, generates an inter-frame redundancy mask, improves the dynamic adaptability of compression screening, and ensures the reasonable division of redundant regions and reserved regions.

[0108] In the embodiment, the compression threshold function generates a fixed compression threshold by calculating the linear combination of the weighted mean and standard deviation of all redundant feature score values, and performs binary determination on each region candidate block according to the threshold. All determination results are filled into the corresponding position in order of frame number and block number, and combined into an inter-frame redundancy mask matrix.

[0109] The present application uses a binary mask to mark the compression state of each region, realizes the division of the compression control unit at the candidate block level, and enhances the accuracy and structural controllability of the compression decision.

[0110] In the embodiment, the S6 specifically comprises:

[0111] S61, traversing and judging the inter-frame redundancy mask matrix M If M i,m =1, the region candidate block B i,m needs to perform variable bit rate quantization compression; if M i,m =0, the region candidate block B i,m needs to perform reference frame reconstruction compression.

[0112] S62, for the region candidate block satisfying M i,m =1, performing variable bit rate quantization operation, assuming that the original pixel value tensor of the region candidate block is The quantization coefficient is The compression output is:

[0113]

[0114] wherein, C i,m represents the quantization compression result of the mth high redundancy region of the ith frame, and Round represents a rounding function;

[0115] S63, for the region candidate block satisfying M i,m = 0, reference frame reconstruction compression is performed, wherein the reference frame is the (i-1)th frame, the region reference block is B i-1,n , and the reference reconstruction result is:

[0116] R i,m = B i-1,n + δ i,m ; δ i,m = P i,m -B i-1,n ;

[0117] wherein, δ i,m represents an inter-frame residual error compensation vector, P i,m is the original block of the current frame, and R i,m is the reconstruction output block;

[0118] S64, the quantization compression result C i,m is combined with the reconstruction output block R i,m according to the original frame number i and the block number m to generate a compressed frame data sequence:

[0119]

[0120] According to the mask result, the application selects different compression strategies, and combines variable bit rate quantization and reference frame reconstruction, so that the compression ratio and the restoration quality are effectively balanced, and the intelligent level of the overall video coding is improved.

[0121] Embodiment 1:

[0122] In order to verify the feasibility of the application in implementation, the application is applied to a certain high-definition video compression processing task, and the task scene is that the intelligent monitoring terminal compresses and stores the video data shot for 24 hours without interruption in real time. In this scene, there are a large number of background static and occasional motion of foreground targets in the monitoring video. The traditional compression algorithm is prone to over-compression or insufficient compression when processing video frames mixed with static and dynamic content, which not only wastes the coding bit rate, but also cannot guarantee the image restoration quality, especially in video playback, there are often phenomena such as partial area mosaic, edge blur or motion ghosting.

[0123] To address the above problems, the present invention introduces a video inter-frame redundancy adaptive compression method based on a spatiotemporal perception graph neural network, and adopts a full-process mechanism that integrates regional candidate block division, graph structure modeling, redundancy score discrimination, and dynamic compression strategy, which has been effectively applied in this scenario. The video frame image is input at a fixed resolution (1920×1080), and the frame rate is set to 30 frames per second. By performing a sliding window block operation on each frame image, it is divided into regional candidate blocks of size 32×32, and each frame is divided into 2040 candidate blocks. After inter-frame mapping and feature extraction, the redundancy score of each candidate block is output through the graph convolutional network, and the redundancy score threshold is set to half of the global mean plus the standard deviation to generate an inter-frame redundancy mask matrix, and further perform compression strategy selection.

[0124] During the variable bitrate quantization process, candidate blocks with scores above a threshold are compressed at a high rate (bitrates are controlled between 12% and 30% of the original value). For candidate blocks with scores below the threshold, a reference frame reconstruction compression strategy is implemented, recording only the difference information, effectively reducing the transmission of duplicate data. Finally, the compressed frames are sequentially combined to output a compressed video frame sequence.

[0125] In order to verify the compression effect, three sections of surveillance video data (daytime, nighttime, and periods of frequent human activity) were selected as sample data, and the performance of the method of the present invention on the same video clips was compared with the traditional H.264 compression algorithm and the video compression method based on convolutional neural network (CNN-Codec). As can be seen from the data in the table, the present invention is superior to the comparison method in terms of image quality indicators such as PSNR and SSIM, and also achieves better control in terms of compression ratio. Especially in scenes with intensive target activities, the traditional method has a high compression rate but severe image distortion. The present invention can effectively retain the clear edges of the moving subject while improving the compression ratio, avoiding the loss of key areas. The following are the specific data test results:

[0126] Table 1 Comparison of video compression performance between the present invention and the comparative method in different scenarios

[0127]

[0128]

[0129] As can be seen from Table 1, in daytime static scenes, the proposed method improves the compression ratio by 12.4% while maintaining image quality. In nighttime low-light scenes, it still maintains a high SSIM value. In dynamic activity scenes, even though the compression rate is relatively low, the image edges and the structure of moving objects are well preserved, and the distortion rate of the restored frame is less than half of that of the traditional method.

[0130] In summary, the implementation results fully prove that the method can effectively improve the compression efficiency and reduce the image loss in various complex video scenes, and has strong practical value and popularization significance.

[0131] The above merely describes the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical solutions and the inventive concept of the present application within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network, characterized in that: The steps include: S1. Obtain continuous frame image data of the video to be compressed, perform preprocessing, extract pixel feature information of each frame image, and construct a frame sequence feature tensor; S2. Based on the frame sequence feature tensor, extract the region candidate blocks in each frame image, calculate the spatial position offset and pixel similarity of the region candidate blocks between adjacent frames, and construct an inter-frame adjacency graph; S3. Based on the inter-frame adjacency graph, the motion vector and local gradient features of each candidate block in the frame sequence are combined to construct a spatiotemporal feature graph structure and generate a graph structure representation with edge weights. S4. Input the spatiotemporal perception feature map structure into the graph neural network for graph convolution calculation, and output the redundant feature score value of each region candidate block; S5. Setting a compression threshold based on the redundant feature score value, generating an inter-frame redundancy mask, and screening out image areas that need to be compressed and reserved areas that do not need to be compressed; S6. Perform variable bit rate quantization compression on the candidate blocks in the region identified as high redundancy according to the inter-frame redundancy mask, and perform reference frame reconstruction compression on the candidate blocks in the region identified as low redundancy to generate compressed frame data; S7. Combining the compressed frame data in a frame sequence order, and outputting a compressed video frame sequence.

2. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that The continuous frame image data includes a sequence of video image frames arranged in time sequence, and each frame image is in the form of a two-dimensional matrix with a fixed resolution.

3. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that The preprocessing includes frame processing, size unification, grayscale normalization and noise filtering operations.

4. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that: The region candidate block is a set of image sub-blocks obtained by dividing the frame sequence feature tensor according to a fixed size and sliding step size. Each image frame is divided into multiple region candidate blocks. Each region candidate block retains its frame number, spatial coordinate information and corresponding pixel features for inter-frame similarity calculation and graph structure construction.

5. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that: The S2 specifically includes: S21, performing frame-by-frame feature extraction on the preprocessed video frame image sequence to obtain a pixel feature matrix for each frame image; S22, dividing each frame image into regions according to a fixed size and sliding step size to obtain multiple region candidate blocks, each region candidate block retaining its pixel features and position coordinates in the frame; S23, for the region candidate blocks in any adjacent frames, extract their center coordinates and calculate the position offset as an inter-frame motion index; S24, calculating the inter-frame pixel similarity score based on the degree of feature difference between the two candidate blocks at each pixel point; S25. Measure the similarity between all candidate blocks based on the position offset and the pixel similarity score, and construct an inter-frame adjacency graph with the candidate blocks as nodes. The connecting edges in the inter-frame adjacency graph are determined based on a preset threshold, and edge weights representing similarity are added.

6. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that: The S3 specifically includes: S31, based on the constructed inter-frame adjacency graph, extracting the inter-frame motion vector of the candidate block in each frame, and recording the displacement in the horizontal and vertical directions; S32, performing gradient calculation on each candidate block in the region to obtain the average gradient value of the local region of the image, which is used to measure the degree of structural change; S33, combining the motion vector of each candidate block in the region with the local gradient feature to form a feature vector; S34. For any pair of region candidate blocks connected in the inter-frame adjacency graph, calculate an edge weight based on the degree of difference between the feature vectors therebetween, where the edge weight represents the degree of similarity between the region candidate block pairs in terms of temporal and spatial structure. S35. Combining all region candidate blocks, connection relationships, and edge weights into a graph structure representation, i.e., a spatiotemporal perception feature graph.

7. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that: The S4 specifically includes: S41, inputting the spatiotemporal perception graph structure including node features and edge weights into a graph convolutional neural network, setting the initial feature vector of each region candidate block to consist of a motion vector and a local gradient feature; S42. In the graph convolutional neural network, according to the edge connection relationship and corresponding edge weight between nodes, multi-layer information transmission and aggregation operations are performed to repeatedly update the graph feature representation of the nodes; S43. In the graph convolution output stage, the final graph features of each region candidate block are input into the redundancy scoring function to generate a redundancy score value for each region candidate block. The redundancy score value is used to indicate whether the region is compressible, and the value range is a continuous value between 0 and 1.

8. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that: The S5 specifically includes: S51, performing statistical processing on the redundant feature score values ​​of all candidate blocks in the region, calculating the average value and standard deviation, and setting a fixed compression threshold based on the adjustment factor; S52, performing a threshold comparison on the redundant feature score value of each candidate region block to generate a binary mask, marking the candidate region blocks whose redundant feature score value is not less than the compression threshold as regions to be compressed, and marking the rest as reserved regions; S53. Combining the two-dimensional masks of the candidate blocks in all frames into an inter-frame redundant mask matrix, which is used to indicate the image region screening result in the compression process.

9. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 8 is characterized in that: The binary mask generates a flag value of 0 or 1 according to the relationship between the redundant feature score value of each region candidate block and the compression threshold, indicating whether the region candidate block needs to be compressed.

10. The video inter-frame redundancy adaptive compression method based on spatiotemporal perception graph neural network according to claim 1 is characterized in that: The S6 specifically includes: S61, judging each region candidate block according to the inter-frame redundancy mask matrix, and performing variable bit rate quantization compression if the region candidate block is marked as a high redundancy region; and performing reference frame reconstruction compression if the region candidate block is marked as a low redundancy region; S62. In a variable bit rate quantization process, scaling and rounding pixel features of the image block using corresponding quantization coefficients to obtain a compressed representation of the high-redundancy block; S63, in the reference frame reconstruction and compression process, selecting a similar region block in the previous frame as a reference, calculating a residual compensation value between the current block and the reference block, and performing weighted reconstruction on the reference block; S64. Combine all compressed blocks and reconstructed blocks into a new compressed frame data sequence according to their frame numbers and block numbers.

Images