Power inspection image feature differentiation image compression method and system based on VQ-VAE

Abstract

The invention discloses an electric power inspection image feature differentiation image compression method and system based on VQ-VAE. The electric power inspection image compression method comprises a training stage and a reasoning stage, and the training stage and the reasoning stage both depend on a system composed of a shared encoder, a differential quantization module, a decoder and an auxiliary classification / attention module. In the training stage, an image containing a target category and an ROI label is input, different areas are distinguished by using label information, and different quantization and loss weighting strategies are applied to the different areas; in the reasoning stage, a common unlabeled image is input, the region category or importance is automatically judged by a model, and a corresponding coding mode is selected. Through the multi-codebook and differential loss strategy, the model automatically pays attention to and preferentially reconstructs the key target area, the compression performance and the downstream detection effect are remarkably improved, and the method has good practical value and popularization prospect.

Description

Technical Field

[0001] This application relates to the field of power line inspection image compression and visual processing technology, specifically to a power line inspection image feature differentiation image compression method and system based on VQ-VAE. Background Technology

[0002] In the power industry, regular or real-time on-site image inspections (including manual inspection photography, drone inspections, and fixed monitoring point photography) generate massive amounts of high-resolution images. Especially in scenarios with large transmission and distribution network coverage and numerous monitoring points, the cost of image transmission and storage is extremely high. Furthermore, constraints exist in remote or field scenarios, including limited bandwidth, limited edge computing resources, and high real-time requirements. The core requirement of power inspection is not only to reduce storage and transmission burdens but also to ensure detailed fidelity in critical target areas such as conductors, insulators, joints, climbing ropes, corrosion, broken strands, arcing marks, and vegetation cover, so that subsequent automatic defect detection and assessment modules (power information systems / inspection management systems) can operate reliably.

[0003] Traditional image compression techniques (such as JPEG and JPEG2000) employ a uniform compression strategy for the entire image, ignoring the differences in importance between semantic regions such as targets and backgrounds. Although standards like JPEG2000 introduce Region of Interest (ROI) coding, such as bit-plane priority coding to improve ROI quality, these methods typically rely on manual annotation of ROI shape information, increasing coding complexity and reducing overall efficiency. Furthermore, classic ROI coding is often based on wavelet or block transforms, making it difficult to directly integrate with deep learning models and less effective when manual annotation is lacking.

[0004] In recent years, deep generative models (such as VAE and VQ-VAE) have been introduced into the field of image compression, achieving efficient encoding and reconstruction by learning the latent space of an image. VQ-VAE, proposed in 2017, maps continuous latent representations to finite discrete codebook vectors. Compared to traditional autoencoders, VQ-VAE's discrete representation has significant compression advantages: it only requires storing the codebook index corresponding to each latent element, thus greatly reducing the representation space; for example, with 1024 codebooks, each spatial location requires only 10 bits of index, far less than the storage requirements of traditional floating-point representations.

[0005] However, existing VAE / VQ-VAE compression methods typically use a single encoding strategy for the entire image, without differentiating between different semantic targets within the image. This can lead to insufficient preservation of details in important target regions; for example, pure VAE models can produce reconstruction blur due to the Gaussian assumption and probability sampling; while VQ-VAE, although capable of generating clearer image details, also struggles to maximize the visual quality of target regions within a limited bit budget if the background and target are not distinguished.

[0006] Therefore, this paper proposes an image compression method for power inspection images that can automatically identify and differentiate key areas, which has significant engineering value: reducing bandwidth / storage usage, improving the practicality of remote / edge deployment, and enhancing the effectiveness of compressed images in defect detection. Summary of the Invention

[0007] The purpose of this invention is to propose a VQ-VAE-based power inspection image feature differentiation image compression method and system, which automatically identifies different features or target regions in power inspection images, applies special processing to important regions in encoding, quantization and loss calculation, thereby achieving high-fidelity reconstruction of target regions and improving overall compression efficiency.

[0008] To achieve the objectives of this invention, the technical solution provided by this invention is as follows:

[0009] First aspect

[0010] This application provides a power inspection image feature differentiation image compression method based on VQ-VAE, including a training stage and an inference stage. Both the training stage and the inference stage rely on a system consisting of a shared encoder, a differential quantization module, a decoder, and an auxiliary classification / attention module.

[0011] The training phase involves inputting an image containing target categories and ROI annotations, using the annotation information to distinguish different regions, and applying different quantization and loss weighting strategies to different regions.

[0012] The reasoning stage involves inputting a plain, unlabeled image, which the model automatically determines the region category or importance and selects the appropriate encoding method.

[0013] Furthermore, the training phase specifically includes the following:

[0014] Step 1.1.1: Input an image containing target category and ROI annotations to the shared encoder. The shared encoder uses ResNet-50 or Visual Transformer to extract features and reduce dimensionality of the input color image with size H×W×3, and outputs a multi-channel latent feature map with size h×w×C, where h, w, and C are preset values.

[0015] Step 1.1.2: Based on the ROI annotation information in the image, segment the latent feature map output in Step 1.1.1 into target region and background region;

[0016] Step 1.1.3: Call the differential quantization module. The differential quantization module has multiple vector quantization codebooks. Each codebook corresponds to a target category or background. Each codebook contains K codebook vectors. Each vector has a dimension of D. Hard quantization is used to replace the target region feature vector with the nearest neighbor vector in the corresponding target category codebook, and to replace the background region feature vector with the nearest neighbor vector in the background codebook.

[0017] Step 1.1.4: Input the quantized feature vector into the decoder. The decoder can be an independent decoder or a conditional decoder. If it is an independent decoder, an independent decoder is configured for "person", "vehicle" and "background" respectively. Each decoder focuses on the image detail reconstruction of the corresponding category. If it is a conditional decoder, conditional decoding is achieved by inputting category condition codes, and finally the complete reconstructed image is synthesized.

[0018] Step 1.1.5: During training, a joint loss function is used, including reconstruction loss, VQ commitment loss, and classification loss. The reconstruction loss uses weighted MSE or perceptual loss, assigning weight λ_ROI to the target region and weight λ_BG to the background region. The VQ commitment loss ensures that the encoder output is close to the selected codebook vector. The classification loss uses cross-entropy loss, utilizing the target category labels in the image annotation to supervise the learning of the auxiliary classification / attention module.

[0019] Furthermore, the reasoning stage specifically includes the following:

[0020] Step 1.2.1: Input an unlabeled image to be compressed to a shared encoder. The shared encoder outputs a multi-channel latent feature map and simultaneously inputs the feature map into an auxiliary classification / attention module. The auxiliary classification / attention module is a lightweight CNN classifier or a module based on a self-attention mechanism, which outputs the class probability of each spatial location in the latent feature map.

[0021] Step 1.2.2: Based on the category probabilities output by the auxiliary classification / attention module, determine the category corresponding to each spatial location, call the codebook of the corresponding category in the differential quantization module, quantize and encode the latent feature map to obtain the quantization index sequence;

[0022] Step 1.2.3: Perform entropy encoding on the quantization index sequence to generate the final compressed bitstream;

[0023] Step 1.2.4: During decompression, entropy decoding is performed on the compressed bitstream to recover the quantization index sequence. The quantization vector is obtained by looking up the corresponding codebook according to the index. The quantization vector is then input into the decoder for reconstruction to generate a reconstructed image of the same size as the original image.

[0024] Furthermore, in step 1.1.3, the number of codebooks in the differential quantization module can be adjusted according to actual needs. If the target categories that need to be focused on in the image to be compressed increase, corresponding codebooks can be added accordingly. The K and D values ​​of different codebooks can be designed independently. For codebooks corresponding to target categories with complex textures, larger K and D values ​​can be set to improve the reconstruction accuracy of the target region of that category.

[0025] Furthermore, in step 1.1.5, when the target region is a texture-rich category, the weight of the perceptual loss is increased in the reconstruction loss. The perceptual loss obtains the high-level features of the image through a pre-trained image feature extraction network and calculates the difference between the reconstructed image and the original image in the high-level features to enhance the restoration effect of the structural details of the target region.

[0026] Furthermore, in step 1.2.1, if the auxiliary classification / attention module adopts a self-attention mechanism, it determines the salient regions in the image by calculating the attention weights at different spatial locations in the feature map, identifies the salient regions as high-priority target regions, and calls the corresponding high-priority codebook for quantization.

[0027] Furthermore, the system is deployed on an AI chip, the encoder is implemented using a standard deep learning library, the lookup operation of the quantization module is incorporated into the backpropagation process of the encoder, and the straight-through estimation is used to solve the non-differentiability problem of the quantization operation, ensuring gradient transfer during the training process.

[0028] Second aspect

[0029] This application provides a power line inspection image feature differentiation image compression system based on VQ-VAE, including a shared encoder, a differentiation quantization module, a decoder, an auxiliary classification / attention module, and an entropy encoder;

[0030] The system is used to perform the following steps:

[0031] Training phase: Input images containing target categories and ROI annotations, use the annotation information to distinguish different regions, and apply different quantization and loss weighting strategies to different regions;

[0032] Inference phase: Input a plain, unlabeled image, and the model automatically determines the region category or importance and selects it.

[0033] Furthermore, the shared encoder employs ResNet-50 or a visual Transformer to extract features and reduce dimensionality of the input image, outputting a multi-channel latent feature map.

[0034] The differential quantization module has multiple vector quantization codebooks, each codebook corresponding to a target category or background, used to quantize and encode the latent feature map according to the category information;

[0035] The decoder is either an independent decoder or a conditional decoder, used to receive quantization vectors and reconstruct image details, and output a fully reconstructed image;

[0036] The auxiliary classification / attention module is a lightweight CNN classifier or a module based on a self-attention mechanism, used to determine the category or importance of each spatial location in the latent feature map during the inference stage;

[0037] The entropy encoding / decoding module uses arithmetic encoding to encode the quantization index sequence to generate a compressed bit stream, and to decode the compressed bit stream to recover the quantization index sequence.

[0038] Furthermore, the codebook of the differentiated quantization module can be dynamically updated during the training process. In the early stage of training, the codebook vector is randomly initialized. During the training process, the value of the codebook vector is adjusted according to the distance between the latent feature vector output by the encoder and the codebook vector, so that the codebook better fits the image feature distribution.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] This invention proposes an innovative VQ-VAE-based differential image compression algorithm. Through multiple codebooks and differential loss strategies, the model automatically focuses on and prioritizes the reconstruction of key target regions, significantly improving the compression performance and downstream detection effect of power line inspection images. It has good practical value and promising prospects for promotion. Attached Figure Description

[0041] Figure 1 A schematic diagram of the process for a power inspection image feature differentiation image compression method based on VQ-VAE provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the network module structure provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the differentiated compression strategy in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0045] This invention is based on artificial intelligence and deep learning technology. It automatically identifies and preserves the details of key target areas (ROI) during image compression, thereby improving compression efficiency and fidelity.

[0046] like Figure 1 As shown, this application provides a power inspection image feature differentiation image compression method based on VQ-VAE, including a training stage and an inference stage. Both the training stage and the inference stage rely on a system consisting of a shared encoder, a differentiation quantization module, a decoder, and an auxiliary classification / attention module.

[0047] The training phase involves inputting an image containing target categories and ROI annotations, using the annotation information to distinguish different regions, and applying different quantization and loss weighting strategies to different regions.

[0048] The reasoning stage involves inputting a plain, unlabeled image, which the model automatically determines the region category or importance and selects the appropriate encoding method.

[0049] Furthermore, the training phase specifically includes the following:

[0050] Step 1.1.1: Input an image containing target category and ROI annotations to the shared encoder. The shared encoder uses ResNet-50 or Visual Transformer to extract features and reduce dimensionality of the input color image with size H×W×3, and outputs a multi-channel latent feature map with size h×w×C, where h, w, and C are preset values.

[0051] Step 1.1.2: Based on the ROI annotation information in the image, segment the latent feature map output in Step 1.1.1 into target region and background region;

[0052] Step 1.1.3: Call the differential quantization module. The differential quantization module has multiple vector quantization codebooks. Each codebook corresponds to a target category or background. Each codebook contains K codebook vectors. Each vector has a dimension of D. Hard quantization is used to replace the target region feature vector with the nearest neighbor vector in the corresponding target category codebook, and to replace the background region feature vector with the nearest neighbor vector in the background codebook.

[0053] Step 1.1.4: Input the quantized feature vector into the decoder. The decoder can be an independent decoder or a conditional decoder. If it is an independent decoder, an independent decoder is configured for "person", "vehicle" and "background" respectively. Each decoder focuses on the image detail reconstruction of the corresponding category. If it is a conditional decoder, conditional decoding is achieved by inputting category condition codes, and finally the complete reconstructed image is synthesized.

[0054] Step 1.1.5: During training, a joint loss function is used, including reconstruction loss, VQ commitment loss, and classification loss. The reconstruction loss uses weighted MSE or perceptual loss, assigning weight λ_ROI to the target region and weight λ_BG to the background region. The VQ commitment loss ensures that the encoder output is close to the selected codebook vector. The classification loss uses cross-entropy loss, utilizing the target category labels in the image annotation to supervise the learning of the auxiliary classification / attention module.

[0055] Furthermore, the reasoning stage specifically includes the following:

[0056] Step 1.2.1: Input an unlabeled image to be compressed to a shared encoder. The shared encoder outputs a multi-channel latent feature map and simultaneously inputs the feature map into an auxiliary classification / attention module. The auxiliary classification / attention module is a lightweight CNN classifier or a module based on a self-attention mechanism, which outputs the class probability of each spatial location in the latent feature map.

[0057] Step 1.2.2: Based on the category probabilities output by the auxiliary classification / attention module, determine the category corresponding to each spatial location, call the codebook of the corresponding category in the differential quantization module, quantize and encode the latent feature map to obtain the quantization index sequence;

[0058] Step 1.2.3: Perform entropy encoding on the quantization index sequence to generate the final compressed bitstream;

[0059] Step 1.2.4: During decompression, entropy decoding is performed on the compressed bitstream to recover the quantization index sequence. The quantization vector is obtained by looking up the corresponding codebook according to the index. The quantization vector is then input into the decoder for reconstruction to generate a reconstructed image of the same size as the original image.

[0060] Furthermore, in step 1.1.3, the number of codebooks in the differential quantization module can be adjusted according to actual needs. If the target categories that need to be focused on in the image to be compressed increase, corresponding codebooks can be added accordingly. The K and D values ​​of different codebooks can be designed independently. For codebooks corresponding to target categories with complex textures, larger K and D values ​​can be set to improve the reconstruction accuracy of the target region of that category.

[0061] Furthermore, in step 1.1.5, when the target region is a texture-rich category, the weight of the perceptual loss is increased in the reconstruction loss. The perceptual loss obtains the high-level features of the image through a pre-trained image feature extraction network and calculates the difference between the reconstructed image and the original image in the high-level features to enhance the restoration effect of the structural details of the target region.

[0062] Furthermore, in step 1.2.1, if the auxiliary classification / attention module adopts a self-attention mechanism, it determines the salient regions in the image by calculating the attention weights at different spatial locations in the feature map, identifies the salient regions as high-priority target regions, and calls the corresponding high-priority codebook for quantization.

[0063] Furthermore, the system is deployed on an AI chip, the encoder is implemented using a standard deep learning library, the lookup operation of the quantization module is incorporated into the backpropagation process of the encoder, and the straight-through estimation is used to solve the non-differentiability problem of the quantization operation, ensuring gradient transfer during the training process.

[0064] It should be noted that the shared encoder employs a deep convolutional neural network (such as the ResNet series) or a visual Transformer to extract features and reduce the dimensionality of the input image, outputting a multi-channel latent feature map. The encoder network is shared across different categories, ensuring consistency in the overall feature space. The features output by the encoder are used by both the quantization and decoding modules, and can also be fed into subsequent classification / attention assistance modules.

[0065] The aforementioned vector quantization module sets up multiple vector quantization codebooks, each corresponding to a target category or background. For example, dedicated codebooks can be configured for major target categories such as people and vehicles, while a general codebook can be configured for the background. During the training phase, the latent feature map is segmented into target and background regions based on the annotation information, and the corresponding codebooks are called for quantization encoding. During the inference phase, an auxiliary module automatically determines the category or importance of each spatial location in the feature map and selects the appropriate quantization codebook accordingly. Quantization can be performed using hard quantization (directly replacing with nearest neighbor vectors) or soft quantization. The dimension and capacity of different codebooks can be designed individually according to the complexity of the categories.

[0066] The decoder has two optional designs: one is to configure independent decoder networks for different categories or features, with each decoder focusing on the reconstruction of its corresponding category; the other is to use a unified decoder, achieving conditional decoding through input category condition codes (such as category embeddings or attention weights). The independent decoder approach is structurally simple but computationally expensive; the conditional decoding approach is more flexible and scalable. After receiving the quantized vector, the decoder reconstructs the image details of the target and background regions through structures such as convolutional layers, ultimately synthesizing a complete image.

[0067] The aforementioned auxiliary classification / attention module, during the inference phase, needs to automatically identify the target category or importance of each region in the image to replace the labeled information. A lightweight network can be added after the encoder: for example, global pooling of the encoder features before entering a multi-classification branch, or using a self-attention mechanism to detect salient regions. This module uses existing labels for supervised learning during training and outputs the category probability or importance of each feature space location during inference to guide the selection of the quantization codebook.

[0068] In the compression process, such as Figures 2-3 As shown, when working with unlabeled images, the image to be compressed is first input into a shared encoder and an auxiliary module. After the encoder generates a latent feature map, the auxiliary module predicts the category / importance of each region or pixel block. Then, based on the prediction results, the corresponding quantization codebook is called at each position of the feature map for discrete encoding, resulting in a quantization index sequence. All indices, along with the necessary category information, are entropy encoded to output the final compressed bitstream.

[0069] In the decompression and decoding process

[0070] The decoder first performs entropy decoding on the compressed stream to recover the discrete index sequence and retrieves the quantization vector from the corresponding codebook based on the category information. Then, these vectors are input into the corresponding decoder pathway for fusion reconstruction, or sent to a single decoder (with category conditions) for decoding, ultimately generating a reconstructed image of the same size as the original image. Because the differential weights and dedicated decoding modes have been learned during training, the details of the target region generated by inference are clearer, while the background achieves a higher compression ratio.

[0071] Corresponding to the above method, this application also provides a VQ-VAE-based power inspection image feature differentiation image compression system, including a shared encoder, a differentiation quantization module, a decoder, an auxiliary classification / attention module, and an entropy encoder;

[0072] The system is used to perform the following steps:

[0073] Training phase: Input images containing target categories and ROI annotations, use the annotation information to distinguish different regions, and apply different quantization and loss weighting strategies to different regions;

[0074] Inference phase: Input a plain, unlabeled image, and the model automatically determines the region category or importance and selects it.

[0075] Furthermore, the shared encoder employs ResNet-50 or a visual Transformer to extract features and reduce dimensionality of the input image, outputting a multi-channel latent feature map.

[0076] The differential quantization module has multiple vector quantization codebooks, each codebook corresponding to a target category or background, used to quantize and encode the latent feature map according to the category information;

[0077] The decoder is either an independent decoder or a conditional decoder, used to receive quantization vectors and reconstruct image details, and output a fully reconstructed image;

[0078] The auxiliary classification / attention module is a lightweight CNN classifier or a module based on a self-attention mechanism, used to determine the category or importance of each spatial location in the latent feature map during the inference stage;

[0079] The entropy encoding / decoding module uses arithmetic encoding to encode the quantization index sequence to generate a compressed bit stream, and to decode the compressed bit stream to recover the quantization index sequence.

[0080] Furthermore, the codebook of the differentiated quantization module can be dynamically updated during the training process. In the early stage of training, the codebook vector is randomly initialized. During the training process, the value of the codebook vector is adjusted according to the distance between the latent feature vector output by the encoder and the codebook vector, so that the codebook better fits the image feature distribution.

[0081] The present invention will be further described below with reference to specific embodiments. It should be understood that the following embodiments are only used to explain the present invention and do not constitute any limitation on the scope of protection of the present invention.

[0082] (1) Network parameter example

[0083] A ResNet-50 encoder can be used as the shared encoder, with an input color image size of H×W×3; the encoder outputs a feature map h×w×C (e.g., 64×64×512). Three vector quantization codebooks are set up: corresponding to the categories "person", "vehicle", and "background" respectively. Each codebook contains K=1024 codebook vectors, and each vector has a dimension D=64. During training, image data with target category and bounding box annotations are used. The encoder feature map is divided into target region and background region using ROI masks, and the corresponding codebook is called for quantization. Quantization uses a hard replacement method, that is, each latent feature vector is replaced with the nearest neighbor vector in the codebook of its corresponding category.

[0084] (2) Loss Function

[0085] During training, joint losses are used, including reconstruction loss, VQ commitment loss, and classification loss. The reconstruction loss can employ weighted MSE or perceptual loss, assigning higher weights λ to the target region. ROI The background region is assigned a lower weight λ. BG For example, λ can be set. ROI =1.0, λ BG =0.5; For targets with rich textures, perceptual loss can be increased to enhance the restoration of structural details. VQ commitment loss ensures that the encoder output is close to the selected codebook vector. The classification module uses cross-entropy loss during training, and the region classifier is trained with known labels.

[0086] (3) Reasoning process

[0087] After training, the network model is fixed as a combination of encoder, quantization codebook, and decoder. During the inference phase, for any input image, the encoder and auxiliary modules (e.g., a lightweight CNN classifier) ​​first automatically determine the class probability of each spatial location; then, the corresponding codebook with the highest probability is selected to quantize and encode the feature map; the remaining steps are similar to those during training. Finally, the decoder reconstructs the target and background vectors in parallel (or conditionally), synthesizing them into the final reconstructed image.

[0088] (4) Model Deployment

[0089] This invention can be deployed on AI chips such as GPUs. The encoder is implemented using a standard deep learning library, and the quantization module is a lookup operation incorporated into the encoder's backpropagation (straight-through estimation can be used). The decoder structure can be a multi-branch or conditional network depending on computational resources. The encoded output is processed by an entropy encoder (such as arithmetic encoding) to generate a compressed stream. During table decoding, the same entropy decoder and codebook can be used to completely reconstruct the feature vector. The above embodiments are merely illustrative examples of the invention's application; in practical applications, parameters such as network structure, codebook size, and loss weights can be adjusted as needed. For example, the encoder can be replaced with a Transformer model, and the auxiliary module can employ a self-attention mechanism. Through these configurations, this invention achieves an effective balance between target area fidelity and overall compression efficiency, making it suitable for scenarios with high visual quality requirements for target areas, such as video surveillance and autonomous driving.

[0090] This invention introduces a differentiated compression strategy to achieve on-demand encoding of key regions and background, significantly improving compression efficiency while maintaining high fidelity in the target region. Specific beneficial effects include:

[0091] By introducing target category labeling and a multi-codebook structure during the training phase, the model learns to use dedicated encoding for different targets. This allows for automatic differentiation of feature regions and targeted quantization during the inference phase, ensuring that the reconstruction quality of target regions is significantly better than that of background regions. Experiments show that, at the same bit rate, the target regions generated by this method are richer in detail and clearer (eliminating reconstruction blur compared to traditional VAE methods); while traditional compression based on a single model cannot capture such subtle differences.

[0092] Leveraging the discrete representation characteristics of VQ-VAE, this invention only requires storing codebook indices instead of all floating-point parameters, significantly reducing data redundancy. For example, when using a 1024-dimensional codebook, each potential spatial location requires only 10 bits for index representation, which is easier to compress compared to continuous representations. Combined with target weight differentiation, this method improves the overall compression ratio while maintaining PSNR / SSIM without reducing it.

[0093] Compared to traditional ROI-based methods such as JPEG2000, this invention eliminates the need for explicit encoding of ROI shapes or additional masking information during transmission, avoiding the complexity of manual annotation and achieving a fully automated compression process. The model can adaptively select important regions for high-quality encoding in various scenarios without user intervention, offering greater adaptability and flexibility.

[0094] Improved performance in downstream visual tasks: Because this invention prioritizes the preservation of target region details, the semantic information of the compressed image is more completely retained. In tasks such as object detection or semantic segmentation, the detection accuracy on the reconstructed image is significantly improved (test reports show that the mAP index is superior to traditional methods at the same compression ratio). Overall…

[0095] As can be seen from the provided test report data, under the same compression ratio, the PSNR and SSIM of the target region obtained by this method are significantly better than those of the traditional unified compression scheme; and because it automatically focuses on key targets, the target detection success rate is improved accordingly. By utilizing multiple codebooks and target weights, this invention achieves the optimal balance between target region reconstruction and overall compression ratio.

[0096] Finally, it should be noted that the above embodiments are merely illustrative and explanatory of the present invention, and are not intended to limit the present invention to the scope of the described embodiments. Furthermore, those skilled in the art will understand that the present invention is not limited to the above embodiments, and many more variations and modifications can be made based on the teachings of the present invention, all of which fall within the scope of protection claimed by the present invention.

Claims

1. A power line inspection image feature differentiation image compression method based on VQ-VAE, characterized in that, It includes a training phase and an inference phase, both of which rely on a system consisting of a shared encoder, a differential quantization module, a decoder, and an auxiliary classification / attention module; The training phase involves inputting an image containing target categories and ROI annotations, using the annotation information to distinguish different regions, and applying different quantization and loss weighting strategies to different regions. The reasoning stage involves inputting a plain, unlabeled image, which the model automatically determines the region category or importance and selects the appropriate encoding method.

2. The power line inspection image feature differentiation image compression method based on VQ-VAE according to claim 1, characterized in that, The training phase specifically includes the following: Step 1.1.1: Input an image containing target category and ROI annotations to the shared encoder. The shared encoder uses ResNet-50 or Visual Transformer to extract features and reduce dimensionality of the input color image with size H×W×3, and outputs a multi-channel latent feature map with size h×w×C, where h, w, and C are preset values. Step 1.1.2: Based on the ROI annotation information in the image, segment the latent feature map output in Step 1.1.1 into target region and background region; Step 1.1.3: Call the differential quantization module. The differential quantization module has multiple vector quantization codebooks. Each codebook corresponds to a target category or background. Each codebook contains K codebook vectors. Each vector has a dimension of D. Hard quantization is used to replace the target region feature vector with the nearest neighbor vector in the corresponding target category codebook, and to replace the background region feature vector with the nearest neighbor vector in the background codebook. Step 1.1.4: Input the quantized feature vector into the decoder. The decoder can be an independent decoder or a conditional decoder. If it is an independent decoder, an independent decoder is configured for "person", "vehicle" and "background" respectively. Each decoder focuses on the image detail reconstruction of the corresponding category. If it is a conditional decoder, conditional decoding is achieved by inputting the category condition code, and finally the complete reconstructed image is synthesized. Step 1.1.5: During training, a joint loss function is used, including reconstruction loss, VQ commitment loss, and classification loss. The reconstruction loss uses weighted MSE or perceptual loss, assigning weight λ_ROI to the target region and weight λ_BG to the background region. The VQ commitment loss ensures that the encoder output is close to the selected codebook vector. The classification loss uses cross-entropy loss, utilizing the target category labels in the image annotation to supervise the learning of the auxiliary classification / attention module.

3. The power line inspection image feature differentiation image compression method based on VQ-VAE according to claim 2, characterized in that, The reasoning stage specifically includes the following: Step 1.2.1: Input an unlabeled image to be compressed to a shared encoder. The shared encoder outputs a multi-channel latent feature map and simultaneously inputs the feature map into an auxiliary classification / attention module. The auxiliary classification / attention module is a lightweight CNN classifier or a module based on a self-attention mechanism, which outputs the class probability of each spatial location in the latent feature map. Step 1.2.2: Based on the category probabilities output by the auxiliary classification / attention module, determine the category corresponding to each spatial location, call the codebook of the corresponding category in the differential quantization module, quantize and encode the latent feature map to obtain the quantization index sequence; Step 1.2.3: Perform entropy encoding on the quantization index sequence to generate the final compressed bitstream; Step 1.2.4: During decompression, entropy decoding is performed on the compressed bitstream to recover the quantization index sequence. The quantization vector is obtained by looking up the corresponding codebook according to the index. The quantization vector is then input into the decoder for reconstruction to generate a reconstructed image of the same size as the original image.

4. The power inspection image feature differentiation image compression method based on VQ-VAE according to claim 3, characterized in that, In step 1.1.3, the number of codebooks in the differential quantization module can be adjusted according to actual needs. If the target categories that need to be focused on in the image to be compressed increase, corresponding codebooks can be added accordingly. The K and D values ​​of different codebooks can be designed independently. For codebooks corresponding to target categories with complex textures, larger K and D values ​​can be set to improve the reconstruction accuracy of the target region of that category.

5. The power inspection image feature differentiation image compression method based on VQ-VAE according to claim 4, characterized in that, In step 1.1.5, when the target region is a texture-rich category, the weight of the perceptual loss is added to the reconstruction loss. The perceptual loss obtains the high-level features of the image through a pre-trained image feature extraction network and calculates the difference between the reconstructed image and the original image in the high-level features to enhance the restoration effect of the structural details of the target region.

6. The power line inspection image feature differentiation image compression method based on VQ-VAE according to claim 5, characterized in that, In step 1.2.1, if the auxiliary classification / attention module adopts a self-attention mechanism, it determines the salient regions in the image by calculating the attention weights at different spatial locations in the feature map, identifies the salient regions as high-priority target regions, and calls the corresponding high-priority codebook for quantization.

7. The power line inspection image feature differentiation image compression method based on VQ-VAE according to claim 6, characterized in that, The system is deployed on an AI chip, and the encoder is implemented using a standard deep learning library. The lookup operation of the quantization module is incorporated into the backpropagation process of the encoder, and the straight-through estimation is used to solve the problem of non-differentiability of the quantization operation, ensuring gradient transfer during the training process.

8. A power line inspection image feature differentiation image compression system based on VQ-VAE, characterized in that, This includes a shared encoder, a differential quantization module, a decoder, an auxiliary classification / attention module, and an entropy encoder; The system is used to perform the following steps: Training phase: Input images containing target categories and ROI annotations, use the annotation information to distinguish different regions, and apply different quantization and loss weighting strategies to different regions; Inference phase: Input a plain, unlabeled image, and the model automatically determines the region category or importance and selects it.

9. A power line inspection image feature differentiation image compression system based on VQ-VAE according to claim 8, characterized in that, The shared encoder uses ResNet-50 or a visual Transformer to extract features and reduce dimensionality of the input image, and outputs a multi-channel latent feature map. The differential quantization module has multiple vector quantization codebooks, each codebook corresponding to a target category or background, used to quantize and encode the latent feature map according to the category information; The decoder is either an independent decoder or a conditional decoder, used to receive quantization vectors and reconstruct image details, and output a fully reconstructed image; The auxiliary classification / attention module is a lightweight CNN classifier or a module based on a self-attention mechanism, used to determine the category or importance of each spatial location in the latent feature map during the inference stage; The entropy encoding / decoding module uses arithmetic encoding to encode the quantization index sequence to generate a compressed bit stream, and to decode the compressed bit stream to recover the quantization index sequence.

10. The power line inspection image feature differentiation image compression system based on VQ-VAE according to claim 9, characterized in that, The codebook of the differential quantization module can be dynamically updated during the training process. In the early stage of training, the codebook vector is randomly initialized. During the training process, the value of the codebook vector is adjusted according to the distance between the latent feature vector output by the encoder and the codebook vector, so that the codebook better fits the image feature distribution.

Images