Region-of-interest protection-oriented power image compression method and system and code stream transmission method

Abstract

The invention provides a region-of-interest protection-oriented power image compression method and system and a code stream transmission method, and relates to the technical field of power image compression. According to the method, the region of interest and the background region are distinguished, the background region is smoothed through bilinear downsampling, spatial redundant information of the background region of the power image is removed, and bit allocation of the background region is effectively reduced, so that lower compressed code streams are consumed under the condition that the fidelity of the region of interest and the same background quality are achieved. Meanwhile, through a content weighted attention module, adaptive bit distribution of different spatial positions is realized, and bits can be reasonably distributed according to the importance difference of image contents, so that the coding quality of the region of interest and the overall compression effect are improved. According to the code stream transmission method provided by the invention, the code stream required by each path of image is forwarded in a staggered and ordered manner, so that transmission bandwidth resources are better utilized.

Description

Technical Field

[0001] The present invention relates to the technical field of power image compression, and particularly relates to a power image compression method, system and bitstream transmission method for protecting regions of interest. Background Art

[0002] In the operation and management of power systems, power image monitoring plays a crucial role. With the continuous expansion of the scale of power facilities and the increasing demand for intelligent monitoring, the amount of power image data has grown exponentially. For example, a large number of cameras are deployed in widely distributed substations, transmission lines and other places to continuously collect image data. These data are crucial for real-time monitoring of the operating status of power equipment, timely detection of potential faults and ensuring the safe and stable operation of the power system. In order to effectively store, transmit and subsequently analyze and process power image data, image compression technology has become a key link. Traditional image compression technologies mainly include methods based on standard coding formats (such as H.264, H.265). Although they have been applied to a certain extent in power image processing, it is difficult to meet the requirements for high-quality restoration of the content of regions of interest (ROI) in some specific scenarios.

[0003] Several ROI-based image compression technologies have also been proposed in the prior art, such as:

[0004] ROI-based JPEG2000 scheme: For the maximum displacement method of JPEG2000 based on ROI, it is unable to control the importance of the ROI and non-ROI regions respectively. For the general scaling method of JPEG2000 based on ROI, although it can control the importance of the region, it can only process regular ROI shapes, such as rectangles and ellipses. This results in the inability to flexibly allocate bitrates according to the actual needs of power equipment and the background during power image processing, limiting its application in power image compression.

[0005] High Efficiency Video Coding (HEVC) scheme: It solves some limitations of the ROI-based JPEG2000 image compression method, but it also has the following problems. First, its ROI method is developed at the Coding Tree Unit (CTU) level and does not allow element-level ROI spatial domain compression, which is not conducive to saving bitrates. Second, this method has obvious blocking artifacts and color distortion problems at high compression ratios.

[0006] Learning-based ROI image compression method: By masking the ROI region and the background region with an ROI, it only relies on the loss function to constrain the bitstream allocation of the background region, and it is difficult to more effectively remove the spatial redundancy information in the background region.

[0007] In short, ROI-based JPEG2000 is difficult to achieve flexible bitrate allocation according to the actual needs of power equipment and the background, which greatly limits its application in power image compression. Although HEVC solves the problem of ROI bit allocation in ROI-based JPEG2000 compression, it needs to be developed at the CTU level and is difficult to effectively achieve pixel-level bit allocation. The ROI image compression algorithm based on learning is difficult to better solve the optimization problem of object-fidelity image compression. In addition, existing methods rarely comprehensively consider the bitstreams of multi-channel power images, resulting in problems such as data transmission congestion and low bandwidth utilization under a certain bandwidth.

[0008] Based on this, the present invention is proposed. Summary of the Invention

[0009] The purpose of the present invention is to provide a power image compression method, system and bitstream transmission method for region of interest protection, with high fidelity of the region of interest, lower compression bitstream consumption under the same quality, better compression effects for the region of interest and the overall image, and high transmission bandwidth utilization.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] In the first aspect, the present invention provides a power image compression method for region of interest protection, including: S1. Segment the input image x from the compression end into a region of interest x or and a background region x br such that x = x or + x br ; S2. Perform bilinear downsampling on the segmented background region to construct a background prior and obtain intermediate features of the background region; S3. Encode the segmented region of interest to obtain intermediate features of the region of interest, and merge and fuse the intermediate features of the region of interest with the intermediate features of the background region, and capture global features and weights through an internal content-weighted attention module to obtain a hidden representation y of the overall image; S4. Quantize the hidden representation y of the overall image to obtain a quantized hidden representation of the overall image S5. Perform arithmetic coding on the quantized hidden representation of the overall image to form a bitstream; S6. Perform arithmetic decoding on the bitstream to restore it to the hidden representation of the overall image S7. Decode the hidden representation of the overall image after the bitstream is restored to obtain a decoded image

[0012] Preferably, the compression method further includes the following: S8. Further encoding the implicit representation y of the overall image to introduce hyperprior information to obtain the spatial dependence of y, and obtaining the implicitly represented z of the further encoded overall image; S9. Quantizing the hyperprior implicit representation z of the overall image to obtain the quantized hyperprior implicit representation of the overall image S10. For the quantized hyperprior implicit representation of the overall image Perform arithmetic coding to form a bitstream; S11. Arithmetically decode the bitstream to restore it to the quantized hyperprior implicit representation of the overall image S12. For Perform hyperprior decoding and send it to the spatial-channel context module to predict the probability distribution of the implicitly represented of the overall image

[0013] In a second aspect, the present invention provides a power image compression system for protecting regions of interest, adopting the above compression method, including: an image segmentation module for segmenting the input image x from the compression end into a region of interest x or and a background region x br such that x = x or + x br ; a prior module for bilinearly downsampling the segmented background region to construct a background prior and obtain intermediate features of the background region; an encoder for encoding the segmented region of interest to obtain intermediate features of the region of interest, and merging and fusing the intermediate features of the region of interest with the intermediate features of the background region, and capturing global features and weights through an internal content-weighted attention module to obtain the implicitly represented y of the overall image; a first quantization module for quantizing the implicitly represented y of the overall image to obtain the quantized implicitly represented A first arithmetic coding module for arithmetically coding the quantized implicitly represented of the overall image to form a bitstream; a first arithmetic decoding module for arithmetically decoding the bitstream to restore it to the implicitly represented of the overall image; a decoder for decoding the implicitly represented of the overall image restored from the bitstream to obtain a decoded image

[0014] Preferably, the compression system further includes: a super encoder for further encoding the implicitly represented y of the overall image to introduce hyperprior information to capture the spatial dependence of y, and obtaining the hyperprior implicitly represented z of the further encoded overall image; a second quantization module for quantizing the hyperprior implicitly represented z of the overall image to obtain the quantized hyperprior implicitly represented A second arithmetic coding module for arithmetically coding the quantized implicitly represented An arithmetic coding is performed to form a bitstream; a second arithmetic decoding module is used to perform arithmetic decoding on the bitstream to restore it to the latent representation of the quantized overall image. A super decoder is used to perform super-prior decoding and send it to the spatio-channel context module to predict the latent representation of the overall image. The probability distribution of.

[0015] The present invention provides a bitstream transmission method in a third aspect. Based on the above-mentioned power image compression method for protecting regions of interest, it includes the following: S21. Decode the bitstream transmitted by each camera to obtain a decoded image; S22. Input the decoded image into the power image compression system for protecting regions of interest to perform data compression to obtain a compressed bitstream; S23. Calculate the size of the bitstream for each path, and according to the current bandwidth, perform transmission allocation through the intelligent bitstream management module to achieve an orderly transmission of the intelligent bitstream.

[0016] Compared with the prior art, the above technical solution has the following advantages:

[0017] A power image compression method for protecting regions of interest provided by the present invention distinguishes regions of interest (target regions) and background regions, smooths the background region through bilinear downsampling, removes the spatial redundant information in the background region of the power image, effectively reduces the bit allocation in the background region, so as to achieve consuming less compressed bitstream while ensuring the fidelity of the region of interest and the background quality of the same quality. At the same time, through the content-weighted attention module, an adaptive bit allocation for different spatial positions is realized, and bits can be reasonably allocated according to the importance difference of the image content to improve the coding quality of the region of interest and the overall compression effect.

[0018] A bitstream transmission method proposed by the present invention is used to forward the bitstream required for each image in an orderly manner to make better use of the transmission bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0020] Figure 1 It is a block diagram of the modules of a power image compression system for protecting regions of interest provided by a specific embodiment of the present invention;

[0021] Figure 2The specific model block diagram of each module in the power image compression system for region of interest (ROI) protection provided by a specific embodiment of the present invention;

[0022] Figure 3 The rate allocation model block diagram of the content weighted attention module in the power image compression system for region of interest (ROI) protection provided by a specific embodiment of the present invention;

[0023] Figure 4 The model block diagram of the block - based local attention module in the power image compression system for region of interest (ROI) protection provided by a specific embodiment of the present invention;

[0024] Figure 5 The flowchart of the bitstream transmission method provided by a specific embodiment of the present invention;

[0025] Figure 6 A more detailed flowchart of the bitstream transmission method provided by a specific embodiment of the present invention.

[0026] Figure 1 In the figure, reference numerals: image segmentation module 100, prior module 200, encoder 300, first quantization module 410, first arithmetic coding module 420, first arithmetic decoding module 430, decoder 500, super encoder 600, second quantization module 710, second arithmetic coding module 720, second arithmetic decoding module 730, super decoder 700, spatio - channel context module 900. Specific Embodiment

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0028] In the monitoring of power substations, the areas of concern (such as switch cabinets, transmission lines, and backgrounds) of different cameras have different monitoring requirements for the system. The present invention preferentially allocates bandwidth to the switch cabinet area through ROI protection, and moderately compresses the background area to ensure the clarity of the key target areas (also referred to as regions of interest or ROI regions in this embodiment). To achieve this purpose, in a preferred embodiment of the present invention, a power image compression system for region of interest (ROI) protection is provided, as Figure 1As shown, it mainly consists of an image segmentation module, a prior module, an encoder, a first quantization module, a first arithmetic coding module, a first arithmetic decoding module, a decoder, a super encoder, a second quantization module, a second arithmetic coding module, a super decoder, and a spatial-channel context module. Based on this compression system, in a preferred embodiment of the present invention, a power image compression method for region of interest protection (ROI-IC) is provided. Please refer to Figure 2 , and it is mainly implemented through the following steps:

[0029] S1. Segment the region of interest \(x_{ROI}\) and the background region \(x_{BG}\) of the image \(x\) input from the compression end, so that \(x = x_{ROI}+x_{BG}\). or and background region \(x_{BG}\) br such that \(x = x_{ROI}\) or + \(x_{BG}\) br .

[0030] S2. Bilinearly downsample the segmented background region to construct a background prior and obtain intermediate features of the background region.

[0031] S3. Encode the segmented region of interest to obtain intermediate features of the region of interest, merge and fuse the intermediate features of the region of interest with the intermediate features of the background region, and capture global features and weights through the internal content-weighted attention module to obtain the hidden representation \(y\) of the overall image;

[0032] S4. Quantize the hidden representation \(y\) of the overall image to obtain the quantized hidden representation \(\hat{y}\) of the overall image

[0033] S5. Perform arithmetic coding on the quantized hidden representation \(\hat{y}\) of the overall image to form a bitstream;

[0034] S6. Arithmetically decode the bitstream to restore it to the hidden representation \(\hat{y}\) of the overall image

[0035] S7. Decode the hidden representation \(\hat{y}\) of the overall image restored from the bitstream to obtain the decoded image

[0036] S8. Further encode the hidden representation \(y\) of the overall image to introduce superprior information to obtain the spatial dependence of \(y\), and obtain the superprior hidden representation \(z\) of the further encoded overall image;

[0037] S9. Quantize the superprior hidden representation \(z\) of the overall image to obtain the quantized superprior hidden representation \(\hat{z}\) of the overall image

[0038] S10. Perform arithmetic coding on the quantized superprior hidden representation \(\hat{z}\) of the overall image to form a bitstream;

[0039] S11. Arithmetically decode the bitstream to restore the hyperprior latent representation of the quantized overall image

[0040] S12. Perform hyperprior decoding on and feed it into the spatial-channel context module to predict the probability distribution;

[0041] S13. Use the information decoded by the hyperprior in S12 and the partially decoded information in S6 to predict the probability distribution for use in bitstream encoding of in S5.

[0042] A more detailed description of the above steps S1 to S7 is as follows:

[0043] At the compression end, given the input image Use an object detection algorithm, such as R-CNN (OROB), to segment the ROI region and the background region. Here, use and to represent the ROI region and the background region respectively, such that x = x or + x br . In a more preferred embodiment, to solve the problem that transmitting background information makes it difficult to achieve object-fidelity compression at low bitrates, a scale factor α is introduced to increase the proportion of the detected ROI region by object-oriented detection. Therefore, if the task requires more attention to the neighboring region of a certain object, a higher α value can be selected. Assuming the width and height of the ROI region are w and h respectively, then the width and height of the expanded ROI region will become (1 + α)w and (1 + α)h respectively. In this way, the proposed method can adaptively focus on the expanded ROI region.

[0044] Subsequently, to reduce the bit allocation for the background region, in a more preferred embodiment, a 1 / 8 bilinear downsampling smoothing operation is applied to the background region. After that, construct the background prior g d to build a background projection subspace, and then input it into the encoder g a to determine the background information essential for compression. At the decompression end, the decoded image is obtained under the condition of region difference loss optimization. Specifically, with the help of the background information contained in g d , the encoder g a aims to obtain the latent representation y in a compact manner. Subsequently, y is quantized through the quantization function Q(·) to obtain Finally, use and the decoder gs the decoded image can be obtained Figure 2 In and Figure 2 , AE and AD are the arithmetic coder and the arithmetic decoder respectively.

[0045] The above processing procedure can be described as follows:

[0046] y = g a (x or , g d (x br )); φ),

[0047]

[0048] where g a and g s represent the encoder network and the decoder network respectively, and φ and θ refer to the parameters of the encoder network and the decoder network respectively.

[0049] As Figure 2 shown, "2RB" refers to the sequential connection of two residual blocks; "Conv|k5|s2" represents a convolutional layer for downsampling with a convolutional kernel size of 5×5 and a stride of 2; "TConv" represents a transposed convolutional layer for upsampling; Attn represents a spatial attention module composed of two groups of residual blocks combined with a sigmoid function.

[0050] The intermediate features of the background region and the intermediate features of the region of interest are initially merged by element-wise addition, and then fed into Figure 3 to achieve the information fusion of the features of the background and target regions. Specifically, first, through convolutional fusion, the fused feature F is obtained; the fused feature F is then normalized, convolved with a 3×3 kernel, height-pooled, and width-pooled to obtain query, key, and value matrices; given the query, key, and value matrices, a multi-head attention calculation is performed on the query and key matrices, and the result is dot-producted with the value matrix; a model enhancement network is introduced, and the result after the dot-product operation is input into this model enhancement network to obtain the long-range dependencies global features and attention weights of the fused feature F. After reweighting, the feature vector Figure 3 The output of is while Figure 3 that is Figure 2 the CWAM in Figure 3 , that is, CWAM is Figure 2 the content-weighted attention module in the encoder in Figure 2 for encoding the input image to obtain the latent representation y. A more detailed description is as follows:

[0051] As Figure 2 shown, for the merging of the intermediate features in step S3, specifically, the intermediate features from g dand g a The intermediate features of a are initially merged. Through an addition operation, the background region and the ROI region are merged into small-scale whole-image features. In a more preferred embodiment, a content-weighted attention module (CWAM) is introduced into the encoder to reweigh the importance of the ROI region and the background region in terms of bit allocation. Generally speaking, the information content in an image varies spatially. Allocating bitrates according to the importance of different spatial contents of the image plays an important role in improving the rate-distortion performance of image compression. The CWAM is used to capture global features and dynamically allocate bits at different spatial positions. Here, the CWAM uses a global attention network structure to achieve global spatial domain feature correlation. The bitrate allocation is implicitly constrained by a loss function. Figure 2 The CWAM in Figure 2 can be regarded as a global fusion module for the background region and the target region. Guided by a loss function with regional differences, it adaptively emphasizes the target region and discards background redundancy to a certain extent, thereby ultimately realizing dynamic bit allocation in spatial positions. The loss function will be described in detail below. In a more preferred embodiment, two CWAMs are introduced into the encoder to enable bitrate allocation at different scales.

[0052] The model or process of bitrate (bitrate) allocation by CWAM is as Figure 3 shown. The key process is as follows: A convolutional layer with a 1×1 convolutional kernel is used to fuse the features of the ROI region and the background region to obtain the fused feature F. Considering the powerful modeling ability of the dilated Transformer module in capturing global dependencies, it is used as the backbone structure for the design of the content-weighted attention module. In addition, to reduce the computational complexity, average pooling operations are respectively performed on the height and width dimensions of the intermediate features generated in the Transformer module, namely "height pooling" and "width pooling". By pooling the height and width, the complexity of the matrix operations in the Transformer module can be reduced. The query, key, and value matrices (W Q ), (W K ) and (W V ) can be calculated through W Q = f Q (F), W K = f K (F), and W V = f V (F) respectively, where f Q (·), f K (·), and f V (·) represent functions based on convolutional layers. Given W Q , W K , and W VA matrix is used to perform multi-head attention calculation to achieve global attention. Then, a gated depthwise separable convolutional feed-forward network (GDFN) with dilated convolutions is further adopted to enhance the model's capabilities. In this way, the long-range dependencies of the fused features can be derived as follows:

[0053]

[0054] where W Q , W K and W V are learned parameters, d k is a scaling factor used to avoid the magnitude problem of the dot product result to stabilize the training process, and it is obtained through network learning. RC(·) represents the residual connection function, and T represents the transpose operation of the matrix.

[0055] After obtaining the global features, an attention operation is used to achieve global bit allocation, and its expression is as follows:

[0056]

[0057] where W A refers to the learned attention weights, ⊙ represents element-wise multiplication, represents that the feature vector finally output by the CWAM module is used as the input of the subsequent network.

[0058] A more detailed description of the above steps S8 to S13 is as follows:

[0059] The hyper-encoder h a is used to introduce hyper-prior information to capture the spatial dependence of the element y. After obtaining z through the quantization process, the hyper-decoder h s is used to estimate the conditional probability distribution based on . The above processing process can be described as follows:

[0060] z = h a (y; φ h ),

[0061]

[0062] where h a and h s refer to the hyper-encoder and decoder networks respectively, and φ h and θ h represent the model parameters. represents the estimated probability distribution.

[0063] The spatial-channel context module (SCCTX) is used to predict the latent variable The probability distribution estimation parameter π = (u, σ). With this estimation, the first arithmetic encoder and decoder can be compressed into a bitstream.

[0064] In a more preferred embodiment, considering the key role of the entropy model in improving the rate-distortion performance, a block-based local attention module is proposed in the design of the super encoder and decoder network to improve the estimation accuracy of the entropy model.

[0065] The super encoder, the second quantization module, the second arithmetic coding module, the second arithmetic decoding module, the super decoder, and the spatial-channel context module together constitute an entropy model based on a convolutional neural network; in the super encoder and the super decoder, a block-based local attention module (PLAM) is introduced to dynamically re-weight the local information in the entropy model based on the convolutional neural network, thereby improving the overall prediction accuracy of the data distribution. A key aspect of lossy learning-based image compression lies in the entropy model, which is responsible for predicting the probability distribution of the quantized latent representation and feeding it into the second arithmetic coding module for use. This preferred embodiment provides an integrated block-based attention mechanism to dynamically re-weight the local information in the entropy model based on the convolutional neural network, thereby improving its estimation accuracy. To this end, a block-based local attention module is proposed, and its model or process is as Figure 4 shown, and this module mainly includes:

[0066] A segmentation operation for segmenting the intermediate feature image M generated in the super encoder or the super decoder into P blocks in a non-overlapping manner according to the B×B division method;

[0067] A block attention sub-module (PAM) for performing attention processing on each block;

[0068] A splicing operation for splicing the blocks that have undergone attention processing;

[0069] A residual block sub-module (2RB) for processing the spliced features by applying two residual block layers;

[0070] A softmax activation function (S) for generating attention weights for the features processed by the two residual block layers through the softmax function.

[0071] The more detailed process is as follows:

[0072] First, the intermediate feature map M is segmented into P blocks in a non-overlapping manner according to the B×B division method. r p,i and r p,jThey are the eigenvalues of the $i$-th and $j$-th elements in the $p$-th block respectively. The output of the block attention sub-module can be expressed by the following formula:

[0073]

[0074] where the definitions of $g(\cdot)$ and $f(\cdot)$ are as follows:

[0075]

[0076] where and $\tau(r p,j ) = w τ r p $.

[0077] Therefore, the final output can be obtained in the following way:

[0078] t p = w l s p + r p

[0079] After that, the blocks obtained after attention processing are concatenated, and two residual block layers are applied, and then a softmax function is used to generate attention weights. The output of the block-based local attention module is denoted by and can be expressed as follows:

[0080]

[0081] where $T = \{t 1 , t 2 , \ldots, t p , \ldots, t P \}$ refers to the concatenation result.

[0082] According to the formula the output of the block-based local attention module can be obtained, that is, the output features of the PLAM in Figure 2 . This module is used as the main component of the encoder and decoder in the design Figure 2 to improve the prediction accuracy of the entropy model. Figure 4 In $\mu: 1\times1$, $\tau: 1\times1$, $l: 1\times1$ represent four convolutional layers with a kernel size of $1\times1$. $r p $ represents the set of all feature elements of the $p$-th block. $s p $ represents the output features of the $p$-th block calculated through the attention mechanism.

[0083] In a more preferred embodiment, a power image compression method and system for protecting regions of interest provided by the above embodiment, in which the entropy model based on a convolutional neural network needs to be trained through two training stages; the first stage: pre-train the model at a low compression rate. The second stage: use the parameters initialized from the pre-trained model for training, and adjust the weights of the region of interest and the background region to obtain a training model applicable to different compression rates; the entropy model of the convolutional neural network implicitly constrains the bit allocation of global spatial domain features through a loss function. The details are as follows:

[0084] Model training driven by a power image dataset: Use a power image dataset (the publicly available RailFOD23 dataset for detecting foreign objects on transmission lines, which contains 14,615 images and 40,541 annotated objects. The annotated objects include 4 common foreign objects: bird nests, balloons, plastic bags, and floating objects) to train the entire compression system / method (ROI-IC). To train the proposed method, the mean squared error MSE metric and the Adam optimizer (two hyperparameters of the Adam optimizer, β 1 = 0.9 and β 2 = 0.999) can be used for parameter optimization. Since it is necessary to train the model at different λ values to obtain a model applicable to different compression rates, a model is first pre-trained at a low compression rate (i.e., λ = 4.5×10 -2 ). Thereafter, other models with the target compression rate are trained using the parameters initialized from the pre-trained model. We use the default setting for the number of channels of all models, and set λ = {8, 16, 32, 150, 450}×10 -4 for each model in the first training stage. At each λ, the weights η of the target region and the background region can be further adjusted to control the compression bit rate.

[0085] In addition, the hybrid quantization technique of SCCTX is adopted to optimize the proposed model, that is, the compression system / method (ROI-IC) proposed in the present invention. Specifically, each Q(y - u) is encoded into bits instead of Q(y), and during the decoding process, Q(y - u)+u is input into the decoder g s for image decoding. However, since the input to the decoder g s is Q(y - u)+u instead of Q(u), this inconsistency makes training challenging. To solve this problem, the training is divided into two steps. In the first 80% of the total number of training epochs, by sampling noise n from the uniform distribution U = {0.5, 0.5}, generate simulated quantization values, and input them into SCCTX, and S(y) is fed into the decoder g s, where S is the straight-through estimator. In the remaining training rounds, S(y - u) + u is input into gs and SCCTX for model training.

[0086] The probability distribution estimation parameter is π = (u, σ), where u represents the mean. Q(y - u) represents quantizing the residual of y with respect to the estimated mean u. (By quantizing Q(y - u), u is used as a priori information to capture the local statistical characteristics of y, making the quantized representation more compact. In this way, the quantization error is mainly distributed in the range of smaller values of y - u, reducing the overall error magnitude.)

[0087] Q(y - u) + u represents adding the mean u subtracted at the encoding end at the decoding end to restore the signal. S(y) represents the straight-through estimator (STE). S(y) is an approximate quantization method that bypasses the non-differentiability problem of quantization through STE during backpropagation. Its role is that in the early stage of training, through S(y), the quantization process can be simulated, avoiding the high non-linear error caused by directly using hard quantization Q(y). Additionally, quantization Q(y) is non-differentiable, but the straight-through estimator S(y) can propagate gradients during backpropagation, avoiding training stagnation. S(y - u) + u means adding residual compensation in quantization. In the later stage of training, the model gradually transitions to directly using the quantized value S(y - u) + u, thus being consistent with the real input in the inference stage.

[0088] During this process, there is no explicit bit allocation. During the learning and training process of the model, bit allocation is automatically achieved through this global network because the loss function will give a greater fidelity constraint to the target region and a smaller fidelity constraint to the background. Under the limitation of the loss function, the global network will automatically allocate weights according to the loss function.

[0089] Loss function: The proposed method requires two training stages. The first stage focuses on training a pre-trained model that can achieve a low compression rate for the entire image. The second stage aims to introduce importance weights for the ROI region and the background region to fine-tune the model obtained in the first stage.

[0090] In the first stage, the loss function can be defined as follows:

[0091]

[0092] where R(·) represents the code rate obtained by calculating the entropy of the quantized latent representation and D(·, ·) represents the distortion, which is used to measure the similarity between the input image and the decoded image, and is measured by the mean square error; λ is a hyperparameter that balances the code rate R and the distortion D.

[0093] In the second stage, rate-distortion optimization is minimized by adopting a region-distinguishing loss function, which can be expressed as follows:

[0094]

[0095] where N refers to the number of images input to the proposed model in each training iteration, i is the index, and x i represents the i-th input image, and the latter is the corresponding decoded image. represents a weight map used to control the bit allocation between the ROI region and the background region of . Its value at the element position

[0096]

[0097] can be expressed as follows: bri where x i refers to the background region of the i-th input image x

[0098] Please refer to Figure 5 , in one embodiment, a bitstream transmission method is provided. Based on the above power image compression method for region of interest protection (ROI-IC), it mainly includes the following steps:

[0099] S21. Decode the bitstream transmitted by each camera first to obtain the decoded image;

[0100] S22. Input the decoded image into the power image compression system for region of interest protection to perform data compression and obtain the compressed bitstream;

[0101] S23. Calculate the size of the bitstream for each path, and through the intelligent bitstream management module, perform transmission allocation according to the current bandwidth to achieve the intelligent transmission of the bitstream in an orderly manner.

[0102] A more detailed process can be seen in Figure 6 .

[0103] Based on the above various preferred embodiments, the power image compression method, system and bitstream transmission method for region of interest protection provided by the present invention have the following beneficial technical effects

[0104] 1. The present invention proposes a power image compression method (ROI-IC) for region of interest (ROI) protection, which integrates various technical means to achieve object-fidelity remote sensing image compression at a low bit rate. Among them, the background smoothing operation reduces the bit allocation in the background area through bilinear downsampling, which is beneficial to achieving overall high rate-distortion performance. In addition, an intelligent bitstream management mechanism is incorporated into the ROI-IC method to make better use of the transmission bandwidth.

[0105] 2. The present invention adopts a unique method for dividing and processing the ROI and background regions. By smoothing the background region through bilinear downsampling, the bit allocation in the background region is effectively reduced, solving the problem that excessive background information affects object-fidelity compression at a low bit rate. At the same time, the proposed CWAM realizes adaptive bit allocation at different spatial positions, which can reasonably allocate bits according to the importance difference of the image content to improve the coding quality of the ROI region and the overall compression effect.

[0106] 3. The PLAM module provided by the present invention incorporates a block-based local attention mechanism into the traditional CNN model, reweighting the local importance of each element, and overcoming the limitation of the traditional CNN that treats local information equally when processing the entropy model. This improvement significantly improves the estimation accuracy of the entropy model, thereby further enhancing the rate-distortion performance, enabling the compressed image to better retain object details and overall quality while maintaining a low bit rate.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above embodiments of the methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0108] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0109] The above embodiments only represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. A power image compression method for protecting a region of interest, characterized in that: include: S1. Segment the image input from the compression end into a region of interest and a background region; S2. Perform bilinear downsampling on the segmented background area, construct background prior, and obtain the intermediate features of the background area; S3. Encode the segmented region of interest to obtain the intermediate features of the region of interest, and fuse the intermediate features of the region of interest with the intermediate features of the background region. Capture the global features and weights through the internal content-weighted attention module to obtain the implicit representation of the entire image. S4. quantizing the latent representation of the whole image to obtain a quantized latent representation of the whole image; S5. performing arithmetic coding on the implicit representation of the quantized overall image to form a bit stream; S6. Performing arithmetical decoding on the bit stream to restore it to an implicit representation of the overall image; S7. Decode the implicit representation of the overall image after the bit stream is restored to obtain a decoded image.

2. The power image compression method for protecting the region of interest according to claim 1, characterized in that: Also includes the following: S8. further encoding the implicit representation of the whole image to introduce super prior information to obtain the spatial dependency of the implicit representation of the whole image, thereby obtaining a further encoded super prior implicit representation of the whole image; S9. quantizing the hyper-prior latent representation of the whole image to obtain a quantized hyper-prior latent representation of the whole image; S10. performing arithmetic coding on the super-prior implicit representation of the quantized overall image to form a bit stream; S11. Performing arithmetical decoding on the bit stream to restore it to a super-prior implicit representation of the quantized overall image; S12. Perform super-prior decoding on the quantized super-prior latent representation of the whole image and send it to the spatial-channel context module to predict the probability distribution of the latent representation of the whole image.

3. The power image compression method for protecting the region of interest according to claim 2, characterized in that: Also includes: S13. Use the information decoded by super-prior in S12 and the partial information decoded in S6 to predict the probability distribution of the implicit representation of the whole image, so as to be used for bit stream encoding of the implicit representation of the whole image in S5.

4. A power image compression system for protecting a region of interest, using the compression method according to any one of claims 1 to 3, characterized in that: include: An image segmentation module is used to segment the image input from the compression end into a region of interest and a background region; The prior module is used to perform bilinear downsampling on the segmented background area, construct background prior, and obtain the intermediate features of the background area; The encoder is used to encode the segmented region of interest, obtain the intermediate features of the region of interest, merge and fuse the intermediate features of the region of interest with the intermediate features of the background area, capture the global features and weights through the internal content-weighted attention module, and obtain the implicit representation of the overall image; A first quantization module is used to quantize the latent representation of the whole image to obtain a quantized latent representation of the whole image; A first arithmetic coding module, used for performing arithmetic coding on the implicit representation of the quantized whole image to form a bit stream; A first arithmetic decoding module, used for performing arithmetic decoding on the bit stream to restore it into an implicit representation of the whole image; The decoder is used to decode the implicit representation of the overall image after the bit stream is restored to obtain a decoded image.

5. The power image compression system for protecting the region of interest according to claim 4, characterized in that: Also includes: A super encoder is used to further encode the implicit representation of the whole image to introduce super prior information to obtain the spatial dependency of the implicit representation of the whole image, and obtain a further encoded super prior implicit representation z of the whole image; A second quantization module is used to quantize the hyper-prior latent representation of the whole image to obtain a quantized hyper-prior latent representation of the whole image; A second arithmetic coding module is used to perform arithmetic coding on the quantized super-prior latent representation of the whole image to form a bit stream; A second arithmetic decoding module, used for performing arithmetic decoding on the bit stream to restore it into a super-prior implicit representation of the quantized whole image; The super decoder is used to perform super prior decoding on the quantized super prior latent representation of the whole image and feed it into the spatial-channel context module to predict the probability distribution of the latent representation of the whole image.

6. The power image compression method for protecting the region of interest according to claim 5, characterized in that: Also includes: The spatial-channel context module is used to predict the probability distribution of the implicit representation of the overall image using the information decoded by the super-decoder super-prior and the partial information decoded by the first arithmetic decoding module, so as to be used for the first arithmetic coding module to perform bit stream encoding on the implicit representation of the overall image.

7. The power image compression system for protecting the region of interest according to claim 4, characterized in that: In the encoder, the intermediate features of the region of interest are merged and fused with the intermediate features of the background region, specifically: the intermediate features of the background region and the intermediate features of the region of interest are initially merged by element-by-element addition, and then fused by convolution to obtain fused features; The fused features are then normalized, convolved, height pooled, and width pooled to obtain query, key, and value matrices. Given the query, key, and value matrices, multi-head attention calculations are performed on the query and key matrices, and the results are dot-producted with the value matrices. A model enhancement network is introduced, and the result of the dot-product operation is input into the model enhancement network to obtain the long-distance dependency, global features, and attention weights of the fused features. After re-weighting, the feature vector is obtained.

8. The power image compression system for protecting the region of interest according to any one of claims 4 to 7, characterized in that: The super encoder, the second quantization module, the second arithmetic coding module, the second arithmetic decoding module, the super decoder, and the space-channel context module together constitute an entropy model based on a convolutional neural network; the super encoder and the super decoder both introduce a block-based local attention module for dynamically re-weighting local information in the entropy model based on the convolutional neural network, specifically including: A segmentation operation for segmenting the intermediate feature image generated in the super encoder or super decoder into a plurality of blocks in a non-overlapping manner; The block attention submodule is used to perform attention processing on each block; The splicing operation is used to splice the blocks that have been processed by attention; The residual block submodule is used to process the concatenated features using two residual blocks; The softmax activation function is used to generate attention weights for the features processed by the two residual blocks through the softmax function.

9. The power image compression system for protecting the region of interest according to claim 8, characterized in that: The entropy model based on convolutional neural network needs to be trained through two training stages; Phase 1: Pre-training the model at low compression rate; The second stage: training is performed using the parameters initialized from the pre-trained model, and the weights of the region of interest and the background region are adjusted to obtain a training model suitable for different compression rates; the encoder network g implicitly constrains the bit allocation of global spatial domain features through the loss function.

10. A code stream transmission method, based on the power image compression method for protecting the region of interest as described in any one of claims 4 to 9, characterized in that: These include: S21. Decode the code stream transmitted from each camera to obtain a decoded image; S22. The decoded image is input into the power image compression system for protecting the region of interest for data compression to obtain a compressed code stream; S23. Calculate the size of each code stream, and distribute the transmission through the intelligent code stream management module according to the current bandwidth to achieve the transmission of the intelligent code stream in a staggered and orderly manner.

Images