Adaptive image compression method, device, equipment and storage medium

Abstract

The embodiments of the present disclosure provide an adaptive image compression method, device, equipment and storage medium. Applied to the technical field of image processing and compression technology, the method includes: using a hybrid motion estimation algorithm to perform motion estimation on the original video stream to obtain global motion information; according to the global motion information, using an image segmentation model to segment each frame of the original image in the original video stream into multiple blocks with different motion degrees and characteristics; performing motion compensation and feature extraction on each block to obtain the residual and residual features of each block; based on the residual and residual features of each block, using a dynamic quantization table, quantizing the residual of each block; according to the statistical characteristics of the residual of each block after quantization, adaptively adjusting the compression parameters; and performing entropy coding compression on the residual of each block after quantization according to the adjusted compression parameters. In this way, high-quality and low-latency compression of the video stream can be achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing and compression technology, and in particular to an adaptive image compression method, device, equipment and storage medium. Background Art

[0002] At present, efficient compression of multimedia data, especially images and videos, has always been one of the core challenges in the field of information technology. Although traditional coding standards have achieved remarkable results in data compression, they still face the following technical challenges:

[0003] Spatial redundancy and temporal redundancy cannot be fully utilized, motion compensation accuracy is insufficient, coding efficiency is not adapted to dynamic scene changes, and it is difficult to balance quality and compression ratio. Summary of the invention

[0004] In view of this, the present disclosure provides an adaptive image compression method, apparatus, device and storage medium.

[0005] According to a first aspect of the present disclosure, an adaptive image compression method is provided. The method comprises:

[0006] Perform motion estimation on the original video stream to obtain global motion information; based on the global motion information, use an image segmentation model to segment each frame of the original image in the original video stream into multiple blocks with different motion degrees and characteristics; perform motion compensation and feature extraction on each block to obtain the residual and residual features of each block; based on the residual and residual features of each block, use a dynamic quantization table to quantize the residual of each block; according to the statistical characteristics of the residual of each block after quantization, adaptively adjust the compression parameters; and perform entropy coding compression on the residual of each block after quantization according to the adjusted compression parameters.

[0007] In some implementations of the first aspect, performing motion estimation on the original video stream to obtain global motion information includes:

[0008] A hybrid motion estimation algorithm is used to calculate the motion vector field of each frame image in the original video stream, and global motion information is obtained from the motion vector field information; the hybrid motion estimation algorithm is a motion estimation algorithm that is a mixture of a block matching algorithm and an optical flow method.

[0009] In some implementations of the first aspect, based on the global motion information, each frame of the original image in the original video stream is segmented into a plurality of blocks with different motion degrees and characteristics using an image segmentation model, including:

[0010] According to the global motion information, the motion characteristics of each frame image are obtained;

[0011] Each frame of image and its motion features are input into the image segmentation model. The image segmentation model divides each frame of original image in the original video stream into multiple blocks with different motion degrees and features according to the predicted segmentation mask.

[0012] In some implementations of the first aspect, performing motion compensation and feature extraction on each block to obtain a residual and a residual feature of each block includes:

[0013] For each block of each frame image in the original video stream, the optimal motion vector in the global motion information is called to compensate the reference frame to obtain the predicted pixel value of each block of the current frame;

[0014] Calculate the difference between the predicted pixel value and the actual pixel value of each block to obtain the residual;

[0015] Based on a three-dimensional convolutional neural network combined with a long short-term memory network, feature extraction is performed on the residual of each block to obtain the spatial and temporal related features of the corresponding residual.

[0016] In some implementations of the first aspect, based on the residual and residual features of each block, a dynamic quantization table is used to quantize the residual of each block, including:

[0017] If the residual features of any block are complex and the complexity of other information is also high, the dynamic quantization table will adaptively update a smaller quantization parameter to quantize the residual of the block to retain the detail information and prevent distortion;

[0018] If the residual feature of any block is simple and the complexity of other information is also low, the dynamic quantization table will adaptively update the larger quantization parameter to quantize the residual of the block to reduce the amount of data and improve the compression rate;

[0019] The dynamic quantization table is adaptively updated according to the residual and residual characteristics of each block, information complexity, and historical data.

[0020] In some implementations of the first aspect, adaptively adjusting a compression parameter according to statistical characteristics of a residual of each block after quantization; and performing entropy coding compression processing on the residual of each block after quantization according to the adjusted compression parameter, including:

[0021] According to the statistical characteristics of the residual of each block after quantization, for blocks with high entropy values ​​and complex textures, a finer code table and context adaptive coding mode are called; for blocks with low entropy values ​​and simple textures, a coarser code table and simple adaptive coding mode are called;

[0022] According to the code table and coding mode, the residual of each block is entropy coded and compressed;

[0023] The entropy coding compression process includes:

[0024] Transform coding is used to transform the residual of each block after quantization, converting the pixel value in the spatial domain into the coefficient in the frequency domain to compress the redundant information;

[0025] According to the code table and the coding mode, entropy coding is performed on the residual of each block after the transformation;

[0026] The codewords obtained by entropy coding are packaged into a code stream in a specified format for storage and transmission.

[0027] In some implementations of the first aspect, before performing motion estimation on the original video stream to obtain global motion information, the method further includes:

[0028] Motion compensation, quantization, and entropy coding compression processing are integrated into an end-to-end deep learning model. According to the rate-distortion optimization principle, back propagation and gradient optimization are used to find the optimal balance between compression rate and reconstruction quality, so that the model has the ability to adaptively adjust the relevant parameters in the motion compensation, quantization, and entropy coding compression processing steps.

[0029] According to a second aspect of the present disclosure, an adaptive image compression device is provided. The device comprises:

[0030] The motion estimation module is used to perform motion estimation on the original video stream and obtain global motion information;

[0031] An image segmentation module is used to segment each frame of the original image in the original video stream into multiple blocks with different motion degrees and characteristics using an image segmentation model according to global motion information;

[0032] The motion compensation and feature extraction module is used to perform motion compensation and feature extraction on each block to obtain the residual and residual features of each block;

[0033] A quantization processing module, used for quantizing the residual of each block based on the residual and residual characteristics of each block using a dynamic quantization table;

[0034] The compression processing module is used to adaptively adjust the compression parameters according to the statistical characteristics of the residual of each block after quantization; and perform entropy coding compression processing on the residual of each block according to the adjusted compression parameters.

[0035] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0036] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described above.

[0037] In the present disclosure, image segmentation based on global motion information divides each frame of the original image into multiple blocks with different motion degrees and characteristics, effectively separates the spatial and temporal correlations in the video, and facilitates subsequent processing steps; motion compensation is performed on each block to eliminate inter-frame redundancy, so that the residual only contains the unpredictable part, effectively reducing the amount of data; residual features are extracted to clarify the internal structure and distribution law of the data in the block, providing good basic information for subsequent quantization and compression; the residual of each block is quantized using a dynamic quantization table to ensure that the number of bits is minimized while maintaining visual quality; according to the statistical characteristics of the block residuals after quantization, the compression parameters are adaptively adjusted, the high-entropy block residuals are carefully encoded, and the low-entropy block residuals are roughly encoded, achieving a balance between overall bit rate control and local detail retention, and optimizing coding efficiency.

[0038] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0040] Figure 1 A flowchart of an adaptive image compression method provided by an embodiment of the present disclosure is shown;

[0041] Figure 2 A block diagram of an adaptive image compression device according to an embodiment of the present disclosure is shown;

[0042] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0044] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0045] In response to the problems mentioned in the background technology, the present disclosure provides an adaptive image compression method, device, equipment and storage medium.

[0046] Specifically, motion estimation is performed on the original video stream to obtain global motion information; based on the global motion information, an image segmentation model is used to segment each frame of the original image in the original video stream into a plurality of blocks with different motion degrees and characteristics; motion compensation and feature extraction are performed on each block to obtain the residual and residual characteristics of each block; based on the residual and residual characteristics of each block, a dynamic quantization table is used to quantize the residual of each block; based on the statistical characteristics of the residual of each block after quantization, compression parameters are adaptively adjusted; and entropy coding compression is performed on the residual of each block after quantization according to the adjusted compression parameters.

[0047] In this way, the spatial, temporal and statistical characteristics of video data can be effectively mined and utilized, and the distortion of the reconstructed video can be reduced as much as possible within a limited bit rate budget, achieving the best match between compression rate and reconstruction quality.

[0048] Figure 1 FIG. 4 is a flow chart of an adaptive image compression method provided by an embodiment of the present disclosure. Figure 1 As shown, the adaptive image compression method 100 may include the following steps:

[0049] S110, performing motion estimation on the original video stream to obtain global motion information.

[0050] Specifically, a hybrid motion estimation algorithm is used to calculate the motion vector field of each frame in the original video stream, and the global motion information is obtained from the motion vector field information; the hybrid motion estimation algorithm is a motion estimation algorithm that is a mixture of the block matching algorithm and the optical flow method. Among them:

[0051] Common motion estimation algorithms include block matching algorithm, optical flow method, feature tracking method, phase correlation method and pixel-level motion estimation algorithm. Block matching algorithm and optical flow method are commonly used in video stream processing.

[0052] The block matching algorithm is a commonly used motion estimation algorithm, which mainly divides the current frame into several blocks of fixed size; sets a search area in the reference frame (usually a rectangular area around the corresponding position of the current block in the previous frame or the next frame); calculates the distortion between the current block and all possible candidate blocks in the search area; selects the candidate block with the smallest distortion as the best matching block, and the corresponding offset is the motion vector of the block.

[0053] For distortion, mean square error is often used as a distance metric, for example:

[0054] {D}_{MSE ({B}_{t}, {B}_{r})}=\frac {1} {{N}^{2}}\sum ^{N-1}_{i=0} {\sum ^{N-1}_{j=0} {\left [ {{I}_{t}\left ( {i,j} \right )-{I}_{r}\left ( {i+u,j+v} \right )} \right ]^{2}}} ;

[0055] in, Indicates the current block B t and candidate block B in the reference frame r The mean square error between them; N is the side length of the image block, indicating that each block has N×N pixels; Indicates the current block B t The pixel value at position (i, j); Represents candidate block B r The pixel value at a position offset by (u, v) relative to the current block, where (u, v) is the offset of the candidate block relative to the center of the current block.

[0056] The optical flow method is a method for estimating the motion between pixels based on the continuity assumption of the image sequence. It takes into account the continuous change of the pixel gray value between consecutive frames and assumes that the pixel brightness remains unchanged in a short time. Based on this assumption, an optical flow equation about the spatial coordinates and time coordinates is established:

[0057] ;

[0058] Among them, I x and I y are the gradients of the image in the x and y directions of space, representing the local rate of change of the image grayscale in these directions; I tIt is the gradient of the image in the time t direction, reflecting the rate of change of the pixel gray value over time; u' and v' represent the movement speed of the pixel in the horizontal and vertical directions respectively, and u' and v' are the motion vectors of the pixel from time t to t+Δt.

[0059] Solving the above optical flow equation usually requires the use of numerical methods, such as the Lucas-Kanade algorithm, the Horn-Schunck algorithm, or an optical flow network based on deep learning (FlowNet, PWC-Net, etc.), which is not specifically limited in this disclosure.

[0060] For application scenarios with high real-time requirements, the block matching algorithm is more popular due to its high computational efficiency and good compatibility, but its accuracy is limited; the optical flow method is often used in application scenarios with high requirements for motion estimation accuracy, such as professional film and television production, video analysis and understanding, but it has high complexity and poor real-time performance.

[0061] Therefore, the block matching algorithm and the optical flow method can be combined. According to the characteristics of each frame of the original video stream, the optical flow method is used in complex motion areas, and the block matching is used in simple motion or sparse texture areas. In this way, by properly calling different algorithms, the accuracy of motion estimation can be improved and errors can be reduced while maintaining high coding efficiency.

[0062] The motion vector field is a collection of spatially distributed motion vectors that together describe the relative motion of all pixels in the entire image or a specific area between video frames, reflecting the complex motion patterns and changes of the video sequence in the spatiotemporal domain.

[0063] A motion vector (MV) is a vector used to describe the relative displacement of pixel blocks between adjacent frames in a video sequence. It can be used to guide motion compensation. It usually consists of two components: horizontal displacement (i.e., horizontal pixel offset) and vertical displacement (i.e., vertical pixel offset).

[0064] Furthermore, a motion vector can be expressed as: ; Where u and v represent the pixel offset in the horizontal and vertical directions when the image area moves from the current frame to the reference frame.

[0065] The following is an example of using a hybrid motion estimation algorithm to calculate the motion vector field of each frame of an original video stream:

[0066] 1. Preprocessing: For each frame of the original video stream, first divide it into multiple blocks B i .

[0067] 2. Feature evaluation: For each block B i, based on the standard deviation of the gradient amplitude, calculate its motion complexity index C i , which can be expressed as:

[0068] ;

[0069] Among them, G x and G y Block B i The gradient values ​​in the horizontal and vertical directions, N is the number of pixels in the block.

[0070] 3. Decision: Set a threshold θ, if block B i The complexity index C i If it is greater than the threshold θ, the block is considered to be in a complex motion area, and the optical flow method is used to calculate its motion vector Otherwise, the motion vector is calculated using the block matching algorithm .

[0071] Combined motion vector: finally get the mixed motion vector MV i :

[0072] .

[0073] Further, the obtained hybrid motion vector MV can be used i Perform inter-frame prediction coding.

[0074] The above is an example of a fixed threshold. The following is an example of an adaptive threshold:

[0075] 1. Define a time window W, including the most recent L frames (for example, L=3, i.e. the current frame F t And the first two frames F t-1 and F t-2 ).

[0076] 2. Calculate the average motion complexity index of each frame in window W:

[0077] ;

[0078] Wherein, C(T, x, y) represents the motion complexity index of the pixel at the position (x, y) in the frame numbered T.

[0079] 3. According to the average complexity index C avg (t) to decide whether the current frame t uses the optical flow method or the block matching method. Set an adaptive threshold θ avg , when C avg (t) exceeds θ avg When , the optical flow method is used; otherwise, the block matching method is used, which can be specifically expressed as:

[0080] .

[0081] In order to enable the system to adapt to changes in video stream content, the adaptive threshold (\theta_{avg}) can also be set to dynamically adjust according to the statistical characteristics of the motion complexity index over a period of time. For example, it can be set to the average value of the complexity index over a period of time plus or minus a certain standard deviation, or the threshold can be continuously optimized through online learning.

[0082] According to the embodiments of the present disclosure, by flexibly selecting motion estimation strategies, it is possible to ensure high motion estimation accuracy in complex motion areas, while avoiding unnecessary high computational complexity in simple motion or sparse texture areas, thereby improving video coding efficiency.

[0083] S120, based on the global motion information, using an image segmentation model to segment each frame of the original image in the original video stream into a plurality of blocks with different motion degrees and characteristics.

[0084] Specifically, according to the global motion information, the motion features of each frame image are obtained; each frame image and its motion features are input into the image segmentation model, and the image segmentation model divides each frame of the original image in the original video stream into multiple blocks with different motion degrees and features according to the predicted segmentation mask.

[0085] Furthermore, the above content is better explained below with an example.

[0086] First, define a motion complexity index to decide when to use which motion estimation method. The complexity index is:

[0087] ;

[0088] Among them, C(x,y) is the motion complexity index, ▽I i is the gradient strength at the image pixel i, N is the total number of pixels, E BM (x, y) is the block matching error, E OF (x, y) is the optical flow residual, w1, w2, w3 are the corresponding indicator weights.

[0089] When the motion complexity exceeds a predetermined threshold, the optical flow method is used; otherwise, the block matching method is used.

[0090] For each frame in the video stream, the motion vector is first calculated by block matching (for example, for a block B in the current frame F t and the reference frame F t-1 The position of the best matching block is found (Δx, Δy) and recorded as motion vector MV = (Δx, Δy).

[0091] At the same time, the optical flow method is used to calculate the dense motion field and obtain the motion vector OFV (x, y) of each pixel.

[0092] Next, motion feature extraction is performed. Based on the results of the above two methods, a motion feature map M is constructed. M can be expressed as:

[0093] ;

[0094] Among them, U OF (x, y) is the optical flow uncertainty, C BM (x, y) is the block matching confidence.

[0095] Then, the original image I and the motion feature map M are combined to form a multi-channel input X, which is input into the pre-trained image segmentation model S, where X(x, y) = [I(x, y), M(x, y)].

[0096] The image segmentation model S predicts a segmentation mask P based on the input X, where each channel of P corresponds to a motion level category, P(x, y) = S(X(x, y)); , N is the number of segmentation categories. (In this example, the main focus is on distinguishing categories with different degrees of motion.)

[0097] The softmax function is used for activation so that the output probability distribution is normalized on each motion category, which can be specifically expressed as:

[0098] .

[0099] According to the predicted segmentation mask P, each frame of the original image is divided into multiple blocks. Assuming there are K types of motion levels, then block B k It can be expressed as: B k ={(x, y)|P(x, y)=k}. Assume there are three categories of motion: 0 still, 1 slow motion, 2 fast motion. By finding the category index corresponding to the maximum probability of each pixel in the segmentation mask P, the image is segmented into different blocks. Specifically, it can be expressed as:

[0100] ; According to Label(x, y), the image can be divided into three blocks B k :B k ={(x, y)|Label(x, y)=k}, k=0 (stationary), 1 (slow-speed movement), 2 (high-speed movement)].

[0101] Furthermore, the above image segmentation model can adopt a fully convolutional network (FCN) or its variants in deep learning.

[0102] The following are several examples of FCN model structures:

[0103] a. U-Net model: U-Net consists of a contraction path and an expansion path, in which the shallow features are fused with the deep features through skip-layer connections.

[0104] In this example, the contraction path of the U-Net model consists of a series of convolutional layers, batch normalization layers, and ReLU activation functions, and 2x downsampling is performed after each convolution; the expansion path of the model is symmetrical to the contraction path, and consists of a series of upsampling, convolutional layers, BN, and ReLU activation functions. After each upsampling, a skip-layer connection is made with the corresponding contraction path layer to add shallow features to deep features. The last layer of the model is a 1x1 convolution output layer, and the number of output channels is equal to the number of segmentation categories K. The Softmax activation function is used to obtain the pixel-level category probability distribution.

[0105] Taking the convolution operation of layer l as an example, the input is X l , the convolution kernel is W l , with a bias of b l , the output is Y l , the ReLU activation function is:

[0106] Y l =f(BN(X l * W l +b l ));

[0107] Among them, * represents the convolution operation and BN represents batch normalization.

[0108] The training parameters of the model can be as follows:

[0109] Learning rate: The initial value is 0.001, and a learning rate decay strategy (e.g., cosine annealing, exponential decay) is used.

[0110] Loss function: The cross entropy loss function is used for multi-category segmentation tasks.

[0111] Regularization: L2 regularization prevents overfitting.

[0112] Data enhancement: Random flipping, rotation, scaling, cropping, etc. to increase the diversity of the training set.

[0113] b. DeepLabV3+ model:

[0114] DeepLabV3+ is based on ResNet as the encoder, combining dilated convolution, atrous spatial pyramid pooling (ASPP) and multi-scale feature fusion to achieve efficient segmentation.

[0115] In this example, the encoder ResNet is used to extract deep features. The ASPP module is used to perform dilated convolutions at different dilation rates to capture multi-scale contextual information. The decoder can restore spatial resolution by fusing shallow features with ASPP outputs through dilated convolutions and upsampling. The model output layer is set to 1x1 convolution followed by Softmax to obtain pixel-level probability distribution.

[0116] In the ASPP module, the dilated convolution operation of the i-th branch is:

[0117] ;

[0118] Among them, (ar) is the expansion rate, and different branches have different (ar).

[0119] The training parameters of the model can be similar to U-Net, including learning rate, loss function, regularization, and data augmentation strategy.

[0120] c. HRNet model: HRNet achieves accurate segmentation by maintaining a high-resolution feature flow throughout the entire network and performing multi-scale fusion between feature maps of different resolutions.

[0121] In this example, the backbone network of the HRNet model consists of multiple parallel resolution branches, and multiple downsampling and upsampling are performed within each branch to always maintain high-resolution features. The fusion module of the model performs cross-stage partial connections (CSPCs) between branches of different resolutions to fuse multi-scale features. The output layer of the model performs 1x1 convolution and Softmax on the highest resolution branch to obtain the segmentation result.

[0122] In the CSPCs module, the high-resolution branch feature H is fused with the low-resolution branch feature L: F=H+U P Sample (L);

[0123] Among them, UpSample represents an upsampling operation.

[0124] The training parameters of the model can be similar to U-Net and DeepLabV3+, including learning rate, loss function, regularization, and data augmentation strategy.

[0125] The above models are all capable of inputting and segmenting images containing motion features. According to the embodiments of the present disclosure, by fusing the original image and motion features, a deep learning image segmentation model is used to achieve refined motion partitioning of each frame of the original video stream, so that targeted operations can be performed later based on the motion characteristics of different blocks.

[0126] S130, performing motion compensation and feature extraction on each block to obtain a residual and residual feature of each block.

[0127] Specifically, for each block of each frame image in the original video stream, the optimal motion vector in the global motion information is called to compensate the reference frame to obtain the predicted pixel value of each block in the current frame; the difference between the predicted pixel value and the actual pixel value of each block is calculated to obtain the residual; based on the three-dimensional convolutional neural network combined with the long short-term memory network, the residual of each block is feature extracted to obtain the spatial and temporal related features of the corresponding residual.

[0128] For example, assuming that for any block B in the current frame image k , obtain the optimal motion vector MV from the global motion information k , using the optimal motion vector MV k For the reference frame F t-1 Compensate to predict the current frame F t Block B k The pixel value of .

[0129] In this example, since the motion vector may contain displacement at the sub-pixel level, a bilinear interpolation method is used to compensate the reference frame to improve prediction accuracy.

[0130] Further assume that F t-1 (x, y)) represents the pixel value of the reference frame at position (x, y), then for the current frame F t Block B k For each pixel point (x, y) in , can be calculated by the following compensation formula:

[0131] ;

[0132] Among them, m and n are the interpolation factors in the horizontal and vertical directions respectively. and They are the optimal motion vector MV k The horizontal and vertical components, F t-1 (x, y) represents the reference frame F t-1 The pixel value at position (x,y).

[0133] In this example, bilinear interpolation is used to ensure that even if the motion vector points to non-integer coordinates, it can be accurately obtained from the reference frame F. t-1 Get the approximate value of the pixel in .

[0134] According to the above compensation formula, the current frame F t Block B k Calculate all the pixels (x, y) in the block to get the predicted pixel value matrix of the block This matrix is ​​consistent with the actual pixel value matrix (F t (B k )) are of the same size, and each element It represents the predicted pixel value of each pixel point (x, y) at the corresponding position obtained by compensation of the reference frame in the current frame.

[0135] For the current frame F t Block B k For each pixel point (x, y), calculate the difference between its predicted pixel value and the actual pixel value to get the residual R(x, y): , clarify the time span of the residual sequence (number of frames T), calculate the residuals between adjacent frames in sequence, and form the current frame F t Block B k The residual matrix R is: R = (R t , R t-1， …, R t-T+1 ).

[0136] A three-dimensional convolutional neural network combined with a long short-term memory network is used to extract residual features, and C3D is selected as the three-dimensional convolutional neural network.

[0137] In this example, the C3D network structure specifically includes several three-dimensional convolutional layers, maximum pooling layers, batch normalization layers, etc. Each convolutional layer is followed by an activation function (such as ReLU), and parameters such as the convolution kernel size, number of channels, step size, and padding are specified. Before the output layer of the C3D network, a long short-term memory network (LSTM) layer is added to capture the long-term dependencies of the residual sequence in the time dimension. The LSTM layer receives the feature sequence output by the C3D network, and captures the long-term dependencies of the residual sequence in the time dimension through four gating mechanisms: input gate, forget gate, output gate, and cell state update. The hidden state (h_t) of each time step contains both the feature information of the current time step and the memory of the past time steps, forming a feature sequence with time dependence.

[0138] The input of the C3D network is a tensor of ((T, H, W, C)) dimensions, where T is the time dimension (number of frames), H and W are the spatial dimensions (height and width), and C is the number of channels.

[0139] For the current block Bk The residual sequence R = (R t , R t-1， …, R t-T+1 ), stack it into a tensor of ((T, H, W, C=1)) as the network input.

[0140] The final output dimension is a feature sequence of (T', H', W', C'), where T', H', W' are the reduced sizes in time, height, and width dimensions, respectively, and C' is the number of feature channels.

[0141] The C3D network extracts spatial features from the input residual sequence through a series of three-dimensional convolutional layers. Each convolutional layer captures the local features of the residual at different spatial scales and time steps through convolution operations across time and space dimensions, further reduces the dimension and normalizes the features through pooling layers and batch normalization layers, and finally outputs a feature sequence.

[0142] The input of the LSTM layer is the feature sequence output by the C3D network, that is, a tensor with dimensions (T', H', W', C'). After passing through the LSTM layer, the feature sequence is converted into a time-dependent feature sequence. The LSTM layer can be set to bidirectional (Bi-LSTM). Through forward propagation, the C3D network and the LSTM layer can jointly process the residual sequence to better capture the dependency between the previous and next time steps. The output of the LSTM layer is the hidden state h of the last time step. T or the sequence of hidden states at all time steps (h1, h2, ..., h T ), whose output is the desired spatial and temporal correlation characteristics of the residual.

[0143] According to the embodiments of the present disclosure, the reference frame is first compensated using global motion information to obtain the predicted pixel value of the current frame block, thereby reducing redundant information during actual encoding. Subsequently, a three-dimensional convolutional neural network is used to capture the rich features of the spatiotemporal dimension, and a long short-term memory network is combined to perform cross-frame spatiotemporal context modeling, so that the model can process the residual sequence more accurately and extract the spatial and temporal related features of the residual, effectively reducing the spatial and temporal redundancy of the video data, effectively reducing the amount of data, and improving the encoding efficiency.

[0144] S140 , based on the residual and residual feature of each block, a dynamic quantization table is used to quantize the residual of each block.

[0145] Specifically, if the residual characteristics of any block are complex and the complexity of other information is also high, the dynamic quantization table will adaptively update a smaller quantization parameter and quantize the residual of the block to retain detail information and prevent distortion; if the residual characteristics of any block are simple and the complexity of other information is also low, the dynamic quantization table will adaptively update a larger quantization parameter and quantize the residual of the block to reduce the amount of data and improve the compression rate; wherein the dynamic quantization table is adaptively updated according to the residual and residual characteristics of each block, information complexity, and historical data.

[0146] An example is provided below to illustrate the above content in detail.

[0147] For each block B of each frame image in the original video stream k Residual k , use a three-dimensional convolutional neural network combined with a long short-term memory network to process the residual sequence, extract the spatial and temporal related features of the residual, and calculate the complexity CF of the residual feature k ; At the same time, consider other information complexity indicators, such as texture complexity, edge density, color change, etc. Calculate other information complexity IC through image analysis algorithms (such as gradient histogram, entropy calculation, color clustering, etc.) k . The complexity of combining residual features CF k and other information complexity IC k , calculate the comprehensive complexity score CS k :[CS k =w1·CF k +w2·IC k ];

[0148] Among them, w1 and w2 are weight coefficients, which are used to balance the influence of residual feature complexity and information complexity. The weight coefficients can be adjusted according to actual application requirements and experimental results.

[0149] Create a dynamic quantization table QTable, which stores the quantization parameter QP for each block k For the currently pending block B k , according to its comprehensive complexity score CS k And historical data (such as the quantization parameter value QP of the block in the previous frame prev , update the quantization parameter QP of the current block k :QP k =f(CS k , QP prev , HistoryData).

[0150] Furthermore, if CS kLarger (indicating high complexity), in order to retain more details, the QP can be appropriately reduced k :

[0151] QP k =min(QP max ,QP prev -α·(CS K -CS threshold ));

[0152] Among them, QP max is the maximum allowed quantization parameter value, QP max is the complexity threshold, and α is the adjustment coefficient.

[0153] Furthermore, if CS k Small (indicating low complexity), in order to improve the compression rate, QP can be appropriately increased k :

[0154] QP k =max(QP min ,QP prev +β·(CS threshold -CS K ));

[0155] Among them, QP min is the minimum allowed quantization parameter value, CS threshold is the complexity threshold, and β is the adjustment coefficient.

[0156] Further, using the updated quantization parameter QP k For block B k The residual is quantized:

[0157] ;

[0158] Among them, Residual k It is block B K The residual matrix of , round is the rounding operation.

[0159] At this point, the embodiment of the present disclosure has completed a process of adaptive quantization processing for blocks of different complexities. Through continuous iteration, the dynamic quantization table QTable will be adaptively updated according to the residual characteristics, information complexity and historical data of each block, thereby realizing refined compression control of each block in the video stream, which helps to effectively reduce the amount of video data and improve coding efficiency while ensuring visual quality.

[0160] S150, adaptively adjusting the compression parameters according to the statistical characteristics of the residual of each block after quantization; and performing entropy coding compression processing on the residual of each block after quantization according to the adjusted compression parameters.

[0161] Specifically, according to the statistical characteristics of the residual of each block after quantization, for blocks with high entropy values ​​and complex textures, a finer code table and context adaptive coding mode are called; for blocks with low entropy values ​​and simple textures, a coarser code table and simple adaptive coding mode are called; according to the code table and coding mode, entropy coding compression processing is performed on the residual of each block; wherein,

[0162] The entropy coding compression process includes:

[0163] Transform coding is used to transform the residual of each block after quantization, converting the pixel value in the spatial domain into the coefficient in the frequency domain to compress the redundant information;

[0164] According to the code table and the coding mode, entropy coding is performed on the residual of each block after the transformation;

[0165] The codewords obtained by entropy coding are packaged into a code stream in a specified format for storage and transmission.

[0166] In traditional methods, a fixed entropy coding model is often used. However, the fixed entropy coding model cannot be flexibly adjusted according to changes in input data. In particular, when dealing with prediction errors with complex probability distribution, the entropy coding efficiency is low.

[0167] According to the embodiments of the present disclosure, adaptive entropy coding is adopted to dynamically adjust the coding strategy according to the statistical characteristics of the actual data, make more effective use of limited bit resources, reduce the overall average code length, thereby improving the compression ratio, achieving higher data compression efficiency, and reducing storage occupancy and bandwidth consumption.

[0168] The above content is described in detail below with a specific example.

[0169] Assume we are processing quantized block B k , whose normalized entropy value is E k (in the range [0, 1]), the texture complexity level is T k (The value ranges from 1 to 5, 1 means simple and 5 means complex).

[0170] According to E k and T k Statistical characteristics, select code table CodeBook k and CodingMode k .

[0171] If E k Greater than 0.9 and T k If it is greater than or equal to 4, the fine code table FineCodeBook and context adaptive coding mode CABAC are selected:

[0172] E k >0.9 and T k >0.4,CodeBook k =FineCodeBook,CodingMode k =CABAC.

[0173] If 0.3≤E k ≤0.9 and 2≤T k ≤3, then select the medium code table MediumCodeBook and adaptive variable length coding mode CAVLC:

[0174] 0.3≤E k ≤0.9 and 2≤T k ≤3, CodeBook k =MediumCodeBook,CodingMode k =CAVLC.

[0175] In other cases, select the coarse code table CoarseCodeBook and the simple adaptive coding mode SimplifiedCAVLC:

[0176] CodeBook k =CoarseCodeBook,CodingMode k =SimplifiedCAVLC.

[0177] Quantized Residual k Perform integer discrete cosine transform (IDCT) encoding:

[0178] TransformedResidual k =IDCT(QuantizedResidual k ).

[0179] Entropy coding is performed according to the selected code table and coding mode:

[0180] EncodedCoefficients k =EntropyEncode(TransformedResidual k , CodeBook k ,CodingMode k );

[0181] Among them, the EntropyEncode function uses the arithmetic coding Arithmetic Coding algorithm, which is implemented as follows:

[0182] Initialize the arithmetic encoder state, for each transform coefficient, according to the code table CodeBook k Find the corresponding probability model; in the context adaptive coding mode CABAC, update the probability model according to the adjacent coefficients; use the current probability model to arithmetically encode the coefficients; update the arithmetic encoder state to prepare to encode the next coefficient, and repeat the above steps until the encoding is completed.

[0183] The encoded codeword EncodedCoefficients k Packing bitstreams according to H.264 video coding standard: BitStream k =Pack(EncodedCoefficients k , H.264);

[0184] The Pack function performs the following steps: encapsulates the coding coefficients into one or more NAL units (Network Abstraction Layer Unit); adds necessary header information (for example, NAL type, SPS / PPS data, etc.); prefixes the start code (Start Code Prefix) in the NAL unit to ensure synchronization; and serializes the entire bitstream to ensure that it complies with the H.264 coding standard.

[0185] Through the above steps, the process of entropy coding and compressing the residual of block (B_k) is completed. For each frame in the entire video stream, this process is repeated to process all blocks in each frame, and a complete coded stream that complies with the H.264 coding standard can be obtained for storage or transmission.

[0186] In some embodiments, before performing motion estimation on the original video stream to obtain global motion information, the method further includes:

[0187] Motion compensation, quantization, and entropy coding compression processing are integrated into an end-to-end deep learning model. According to the rate-distortion optimization principle, back propagation and gradient optimization are used to find the optimal balance between compression rate and reconstruction quality, so that the model has the ability to adaptively adjust the relevant parameters in the motion compensation, quantization, and entropy coding compression processing steps.

[0188] Furthermore, according to the rate-distortion optimization principle, the optimal balance between compression rate and reconstruction quality is found through back propagation and gradient optimization, including:

[0189] According to the rate-distortion loss function, the joint optimization objectives are defined, including: reconstruction distortion v and the encoding bit rate R (C t).

[0190] The mean square error (MSE) or structural similarity index (SSIM) is usually used to measure the distortion, and the bit rate is directly calculated from the codeword length: ;

[0191] Among them, α and β are weight coefficients used to balance the relationship between distortion and bit rate, which can be adjusted according to specific application requirements and experimental results; It represents the reconstruction distortion between the original image frame and the reconstructed image frame. It can be calculated by using mean square error or structural similarity index (SSIM) according to the specific situation.

[0192] R(C t ) represents the encoded bit stream C t The bit rate can usually be calculated from the codeword length generated by the encoder.

[0193] According to the embodiments of the present disclosure, by training the end-to-end model, using back propagation and gradient optimization methods, the model can automatically learn and adjust the internal parameters of each module, including motion estimation network weights, quantization steps, probability model parameters, etc., to minimize the rate-distortion loss function. In this way, the model can adaptively find the optimal motion compensation strategy, quantization parameters, and entropy coding method under a given distortion constraint to achieve the best balance between compression rate and reconstruction quality.

[0194] The above contents will be described in more detail below with reference to a specific embodiment of an unmanned engineering machinery.

[0195] Unmanned construction machinery refers to mechanical equipment that uses advanced automation and information technology and can perform autonomous operations under preset paths or real-time planning without direct human operation. It has extremely high autonomous adaptability and safety.

[0196] This type of equipment is mostly used in complex construction environments such as disaster relief and mining, so it requires the integration of visual sensing, lidar and other technologies to achieve precise operations.

[0197] Due to the complex characteristics of the construction environment, this type of equipment will generate a large amount of continuous frame image data during operation. Especially when using high-definition / ultra-high-definition cameras, the amount of raw image data is extremely large, and the local storage demand is extremely large. The image compression and processing burden is heavy and takes a long time. Therefore, the processing and storage of raw image data has become a key factor restricting the performance of this type of equipment.

[0198] In view of the above problems, a large amount of continuous frame image data generated by unmanned engineering machinery in a complex construction environment can be solved by using the adaptive image compression method provided by the embodiment of the present disclosure. The specific steps are as follows:

[0199] 1. Hybrid Motion Estimation Stage

[0200] In this embodiment, the visual system of the unmanned engineering machinery adopts a motion estimation algorithm that combines the block matching algorithm and the optical flow method to estimate the current frame image I t and the previous frame image I t-1 Perform hybrid motion estimation and calculate the motion vector field MV of each frame in the construction site video stream in real time t The specific formula is as follows:

[0201] MV t (x,y)=MixedME(I t , I t-1 )(x, y);

[0202] Among them, I t and I t-1 They represent the high-definition images of the current frame and the previous frame respectively, and MixedME represents the hybrid motion estimation algorithm.

[0203] 2. Image segmentation and motion feature extraction stage

[0204] The current frame image I t With its motion vector field MV t Input to the image segmentation model S to obtain the segmentation mask M t :M t =S(I t , MV t ); According to the mask M t , the original high-definition image is divided into multiple blocks B related to mechanical activities k , and extract the motion features F of each block k , (e.g., excavation surface, bucket motion trajectory, etc.):

[0205] B k ={(x,y)|M t (x, y) = k};

[0206] F k =ExtractMotionFeatures(MV t , B k ).

[0207] 3. Motion compensation and residual calculation stage

[0208] For each block B related to mechanical activities k , using the optimal motion vector MV in the global motion information k Compensate the reference frame to obtain the predicted pixel value matrix :

[0209] ;

[0210] Calculate block B k The residual of , we get the residual sequence:

[0211] ;

[0212] Use 3D-CNN combined with LSTM to process the residual sequence and extract relevant residual features:

[0213] ;

[0214] Among them, H t-1 is the LSTM hidden state at the previous moment.

[0215] 4. Dynamic quantization and entropy coding stage

[0216] According to the complexity of residual features CF k 、Information complexity IC k And historical data H, update the quantization parameter QP k :QP k =f(CF k ,IC k , H);

[0217] Using QP k Quantize the residual:

[0218] .

[0219] Next, entropy coding compression is performed:

[0220] Transform coding:

[0221] ;

[0222] Entropy Coding:

[0223] ;

[0224] Select the codebook based on the statistical characteristics of the residual k and CodingMode k .

[0225] Codeword packaging:

[0226] .

[0227] At this point, the adaptive compression of the current operating frame image of the unmanned engineering machinery is completed. For the entire video stream, repeat this process to process all relevant blocks of each frame to obtain a complete encoded code stream.

[0228] The above is the image compression process. Correspondingly, for decoding and reconstructing the image, the following methods can be selected:

[0229] 1. Code stream analysis and unpacking:

[0230] Stream unpacking: Unpack the received coded stream and restore the coefficient sequence after entropy coding:

[0231] .

[0232] 2. Entropy decoding:

[0233] According to the code table CodeBook used in encoding k and CodingMode k , decode the entropy coded coefficients and restore the transform coded residual:

[0234] .

[0235] 3. Transform inverse coding:

[0236] Perform an inverse discrete cosine transform (IDCT) on the transform-coded residual to restore the quantized residual:

[0237] .

[0238] 4. Inverse quantization and motion compensation:

[0239] Inverse quantization: According to the quantization parameter QP used during encoding k , inverse quantize the quantized residual to get the original residual:

[0240] ;

[0241] Motion compensation: using the optimal motion vector MV from global motion information k Perform inverse compensation on the reference frame to obtain the decoded pixel value matrix:

[0242] .

[0243] 5. Image synthesis and display:

[0244] Image synthesis: Combine the decoded pixel value matrix with the original frame I t Add the pixel values ​​of the corresponding areas in to get the final decoded image:

[0245] ;in, Indicates that the decoded image is in block B k The pixel value on .

[0246] Display: All decoded blocks are stitched into a complete image frame and displayed in the order of the video stream for user observation or further processing.

[0247] The above is a complete embodiment, which integrates motion compensation, quantization processing, entropy coding compression, etc. through a deep learning model, and adaptively adjusts relevant parameters according to the rate-distortion optimization principle, so as to effectively reduce the amount of data while ensuring the reconstruction quality, reduce data latency, and reduce storage space requirements, thereby improving the equipment performance of related engineering machinery.

[0248] According to the embodiments of the present disclosure, the following technical effects are achieved:

[0249] 1. The hybrid motion estimation algorithm can improve the accuracy of motion estimation and reduce motion estimation errors while maintaining high coding efficiency. The reference frame is compensated based on high-precision global motion information to obtain the predicted pixel value of the current frame block, which improves the accuracy of motion compensation and reduces redundant information.

[0250] 2. Effectively mine and utilize the spatial, temporal and statistical characteristics of video data to achieve high-quality, low-latency compression of video streams while ensuring the same visual quality. At the same time, it saves storage resources and reduces transmission costs.

[0251] 3. The rate-distortion optimization principle is used to guide the automatic adjustment of model parameters, find the best balance between compression rate and reconstruction quality, and achieve a balance between overall bit rate control and local detail retention.

[0252] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0253] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0254] Figure 2 FIG. 1 is a block diagram of an adaptive image compression device according to an embodiment of the present disclosure. Figure 2 As shown, the adaptive image compression device 200 may include:

[0255] The motion estimation module 210 is used to perform motion estimation on the original video stream and obtain global motion information;

[0256] An image segmentation module 220 is used to segment each frame of the original image in the original video stream into a plurality of blocks with different motion degrees and characteristics using an image segmentation model according to the global motion information;

[0257] The motion compensation and feature extraction module 230 is used to perform motion compensation and feature extraction on each block to obtain the residual and residual feature of each block;

[0258] A quantization processing module 240, configured to perform quantization processing on the residual of each block based on the residual and residual features of each block using a dynamic quantization table;

[0259] The compression processing module 250 is used to adaptively adjust the compression parameters according to the statistical characteristics of the residual of each block after quantization; and perform entropy coding compression processing on the residual of each block according to the adjusted compression parameters.

[0260] In some embodiments, the adaptive image compression apparatus 200 may further include:

[0261] The rate-distortion optimization module is used to integrate motion compensation, quantization processing, and entropy coding compression processing into an end-to-end deep learning model. According to the rate-distortion optimization principle, through back propagation and gradient optimization, the optimal balance between compression rate and reconstruction quality is found, so that the model has the ability to adaptively adjust the relevant parameters in the motion compensation, quantization processing, and entropy coding compression processing steps.

[0262] Understandably, Figure 2 Each module / unit in the adaptive image compression device 200 shown has the function of implementing each step in the adaptive image compression method 100 provided in the embodiment of the present disclosure, and can achieve its corresponding technical effect. The specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0263] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0264] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown.

[0265] The electronic device 300 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0266] The electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a ROM 302 or a computer program loaded from a storage unit 308 into a RAM 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0267] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0268] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as method 100. For example, in some embodiments, the method 100 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the method 100 in any other appropriate manner (e.g., by means of firmware).

[0269] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0270] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0271] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0272] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0273] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0274] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0275] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0276] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An adaptive image compression method, characterized in that: The method comprises: Using a hybrid motion estimation algorithm to perform motion estimation on the original video stream to obtain global motion information; wherein the hybrid motion estimation algorithm is a motion estimation algorithm that is a mixture of a block matching algorithm and an optical flow method; According to the global motion information, each frame of the original image in the original video stream is segmented into a plurality of blocks with different motion degrees and characteristics using an image segmentation model; Perform motion compensation and feature extraction on each block to obtain the residual and residual features of each block; Based on the residual and residual features of each block, the residual of each block is quantized using a dynamic quantization table; Adaptively adjust the compression parameters according to the statistical characteristics of the residual of each block after quantization; and perform entropy coding compression processing on the residual of each block after quantization according to the adjusted compression parameters; in: The method of using a hybrid motion estimation algorithm to perform motion estimation on the original video stream to obtain global motion information includes: Divide each frame of the original video stream into multiple blocks, calculate the motion complexity of each block, and use the optical flow method to perform motion estimation when the motion complexity of the block is greater than a preset threshold; otherwise, use the block matching method to perform motion estimation and obtain global motion information; The method of segmenting each frame of the original image in the original video stream into a plurality of blocks with different motion degrees and characteristics by using an image segmentation model according to the global motion information includes: According to the global motion information, obtaining motion features of each frame of image; Each frame image and its motion features are input into the image segmentation model, and the image segmentation model segments each frame original image in the original video stream into a plurality of blocks with different motion degrees and features according to the predicted segmentation mask.

2. The method according to claim 1, characterized in that The performing motion compensation and feature extraction on each block to obtain the residual and residual features of each block includes: For each block of each frame image in the original video stream, the optimal motion vector in the global motion information is called to compensate the reference frame to obtain the predicted pixel value of each block of the current frame; Calculate the difference between the predicted pixel value and the actual pixel value of each block to obtain the residual; Based on a three-dimensional convolutional neural network combined with a long short-term memory network, feature extraction is performed on the residual of each block to obtain the spatial and temporal related features of the corresponding residual.

3. The method according to claim 1, characterized in that The method of quantizing the residual of each block by using a dynamic quantization table based on the residual and residual feature of each block includes: According to the residual and residual features of each block, the complexity of the residual features is calculated; the complexity of other information is calculated through the image analysis algorithm; the comprehensive complexity score is calculated by combining the complexity of the residual features and the complexity of other information; For blocks with high comprehensive complexity score values, the dynamic quantization table will adaptively update the first quantization parameter and quantize the residual of the block to retain detail information and prevent distortion; For blocks with low comprehensive complexity score values, the dynamic quantization table will adaptively update the second quantization parameter and quantize the residual of the block to reduce the amount of data and improve the compression rate; where: The first quantization parameter is smaller than the second quantization parameter.

4. The method according to claim 1, characterized in that: The compression parameters are adaptively adjusted according to the statistical characteristics of the residual of each block after quantization; The quantized residual of each block is subjected to entropy coding compression processing according to the adjusted compression parameters, including: According to the statistical characteristics of the residual of each block after quantization, the corresponding code table and adaptive coding mode are called; Transform coding is used to transform the residual of each block after quantization, converting the pixel value in the spatial domain into the coefficient in the frequency domain to compress the redundant information; According to the called corresponding code table and adaptive coding mode, entropy coding is performed on the residual of each block after the transformation processing; The codewords obtained by entropy coding are packaged into a code stream in a specified format for storage and transmission.

5. The method according to claim 1, characterized in that Before performing motion estimation on the original video stream to obtain global motion information, the method further includes: Motion compensation, quantization, and entropy coding compression processing are integrated into an end-to-end deep learning model. According to the rate-distortion optimization principle, back propagation and gradient optimization are used to find the optimal balance between compression rate and reconstruction quality, so that the model has the ability to adaptively adjust the relevant parameters in the motion compensation, quantization, and entropy coding compression processing steps.

6. An adaptive image compression device, characterized in that: include: A motion estimation module, used to perform motion estimation on the original video stream using a hybrid motion estimation algorithm to obtain global motion information; wherein the hybrid motion estimation algorithm is a motion estimation algorithm that is a mixture of a block matching algorithm and an optical flow method; An image segmentation module is used to segment each frame of the original image in the original video stream into a plurality of blocks with different motion degrees and characteristics using an image segmentation model according to global motion information; The motion compensation and feature extraction module is used to perform motion compensation and feature extraction on each block to obtain the residual and residual features of each block; A quantization processing module, used for quantizing the residual of each block based on the residual and residual characteristics of each block using a dynamic quantization table; A compression processing module, used to adaptively adjust the compression parameters according to the statistical characteristics of the residual of each block after quantization; and perform entropy coding compression processing on the residual of each block according to the adjusted compression parameters; in: The method of using a hybrid motion estimation algorithm to perform motion estimation on the original video stream to obtain global motion information includes: Divide each frame of the original video stream into multiple blocks, calculate the motion complexity of each block, and use the optical flow method to perform motion estimation when the motion complexity of the block is greater than a preset threshold; otherwise, use the block matching method to perform motion estimation and obtain global motion information; The method of segmenting each frame of the original image in the original video stream into a plurality of blocks with different motion degrees and characteristics by using an image segmentation model according to the global motion information includes: According to the global motion information, obtaining motion features of each frame of image; Each frame image and its motion features are input into the image segmentation model, and the image segmentation model segments each frame original image in the original video stream into a plurality of blocks with different motion degrees and features according to the predicted segmentation mask.

7. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

Images