Privacy protection image compression method and device, equipment and storage medium

Abstract

The invention discloses a privacy protection image compression method and device, equipment and a storage medium, and relates to the technical field of image privacy protection, and the method comprises the steps: obtaining an original image, and determining a retrieval reference image according to the original image; performing image compression quantization potential characterization on the retrieval reference image, and determining a first quantization feature and a second quantization feature; performing entropy loss modeling on the probability distribution of the second quantitative feature by using a factorization density model to determine entropy loss; embedding the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image; calculating a signal retention loss and a semantic embedding loss based on the original image, the retrieval reference image, and the reconstructed image; and restraining the potential representation of the reconstructed image based on the entropy loss, the signal retention loss and the semantic embedding loss, and generating a target reconstructed image. Privacy is effectively protected, and high-efficiency and high-quality image compression is realized.

Description

Technical Field

[0001] The present application relates to the field of image privacy protection technology, and in particular to a privacy-preserving image compression method, apparatus, device, and storage medium. Background Art

[0002] With the widespread development of IoT infrastructure and the growing demand for machine-oriented applications, image compression is increasingly focused on machine vision analysis, making machine-oriented image coding (ICM) a promising direction. ICM aims to create a unified compression framework that balances human perception and machine analysis. In recent years, the rise of learning-based image compression (LIC) has enabled the integration of machine-oriented codec modules and joint optimization with downstream analysis models to improve performance under the rate-perception-analysis criterion. A transfer-based ICM approach has been developed to seamlessly adapt human-oriented LIC for machine analysis in a plug-and-play manner. This significantly improves the efficiency of edge-cloud collaboration through a more compact bitstream, further expanding the applicability of compression to a wider range of machine vision analysis scenarios. However, despite improvements in compression efficiency and application scope, when edge devices upload data to the cloud, such as surveillance systems or smartphones, the image content is directly exposed, which poses significant challenges and potential risks for image analysis in the context of intelligent analytics and large models.

[0003] Currently, the existing approach is to directly apply image steganography to address image privacy issues. Image steganography is an information hiding technique that aims to embed secret data, such as binary code, text, or images, into images, ensuring that the modification is visually imperceptible. However, direct application of image steganography still cannot solve the privacy issues in cloud-edge collaboration scenarios.

[0004] However, image steganography typically assumes a lossless transmission channel. In reality, lossy image processing operations such as compression, scaling, and cropping can make it difficult to accurately extract the embedded secret information. While steganography aims to transmit secret information, it doesn't consider the comprehensive nature of information utilization in conjunction with cloud-based models. Furthermore, image steganography doesn't consider the image and secret information together, resulting in poor performance in collaborative scenarios. Therefore, ensuring the covert transmission of secret information under lossy compression while flexibly controlling bit rate distortion and transmission performance has become a pressing issue.

[0005] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a privacy-preserving image compression method, apparatus, device and storage medium, aiming to solve the technical problem of how to ensure the secret transmission of secret information under lossy compression and flexibly control bit rate distortion and transmission performance.

[0007] To achieve the above objectives, this application proposes a privacy-preserving image compression method, which includes:

[0008] Acquire an original image, and determine a retrieval reference image based on the original image;

[0009] Performing image compression quantization potential representation on the retrieved reference image to determine a first quantization feature and a second quantization feature;

[0010] Performing entropy loss modeling on the probability distribution of the second quantized feature using a factor decomposition density model to determine the entropy loss;

[0011] Embedding the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image;

[0012] Calculating a signal preservation loss and a semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image;

[0013] The reconstructed image latent representation is constrained based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

[0014] In one embodiment, the step of determining a retrieval reference image based on the original image includes:

[0015] According to the original image and the pre-built vector database, a corresponding target search sample image is matched to determine a database sample image set;

[0016] Extracting feature vectors of the original image and the database sample image set using an image pre-training model to calculate cosine similarity and determine image similarity;

[0017] Sort the database sample image set according to the image similarity to determine a preceding sample image set;

[0018] A target image is selected based on the preceding sample image set to obtain a retrieval reference image.

[0019] In one embodiment, the step of performing image compression quantization potential representation on the retrieved reference image and determining the first quantitative feature and the second quantitative feature includes:

[0020] Extracting semantic features of the retrieval reference image and inputting them into a deep learning model for image compression coding to determine potential representations;

[0021] Using a super-prior model to capture the spatial dependencies between the corresponding elements of the latent representation, and determining a super-prior representation;

[0022] quantizing the potential representation using a quantization operator to obtain a first quantized feature;

[0023] The super-prior representation is quantized using a quantization operator to obtain a second quantized feature.

[0024] In one embodiment, the step of embedding the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image includes:

[0025] embedding the online semantic features of the original image into the first quantized features in the form of variational prediction according to the entropy loss, and determining a joint optimized coding;

[0026] The joint optimized code is decoded and reconstructed to obtain a reconstructed image.

[0027] In one embodiment, the step of calculating the signal preservation loss and the semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image includes:

[0028] Extracting semantic features of the retrieved reference image and the reconstructed image to calculate signal preservation loss;

[0029] Semantic features of the original image and the reconstructed image are extracted to calculate the cosine similarity between the target images and determine the semantic embedding loss.

[0030] In one embodiment, the step of constraining the reconstructed image potential representation based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image includes:

[0031] Get the compactness trade-off factor;

[0032] determining an overall objective function based on the compactness tradeoff factor, the entropy loss, the signal preservation loss, and the semantic embedding loss;

[0033] The potential representation of the reconstructed image is constrained based on the overall objective function to obtain a target reconstructed image.

[0034] In one embodiment, the step of constraining the potential representation of the reconstructed image based on the overall objective function to obtain a target reconstructed image includes:

[0035] Determining entropy model parameters, encoder parameters, decoder parameters, and feature extraction network parameters based on the overall objective function;

[0036] The entropy model parameters, the decoder parameters, and the feature extraction network parameters are fixed, and the encoder parameters are adjusted to constrain the potential representation of the reconstructed image to obtain a target reconstructed image.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a privacy-preserving image compression device, which includes:

[0038] An acquisition module, configured to acquire an original image and determine a retrieval reference image based on the original image;

[0039] The acquisition module is further configured to perform image compression quantization potential representation on the retrieved reference image to determine a first quantization feature and a second quantization feature;

[0040] A processing module, configured to perform entropy loss modeling on the probability distribution of the second quantized feature using a factor decomposition density model to determine the entropy loss;

[0041] The processing module is further configured to embed the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image;

[0042] An execution module, configured to calculate a signal preservation loss and a semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image;

[0043] The execution module is further configured to constrain the potential representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes a privacy-preserving image compression device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the privacy-preserving image compression method described above.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the privacy-preserving image compression method described above are implemented.

[0046] One or more technical solutions proposed in this application have at least the following technical effects:

[0047] This embodiment proposes a privacy-preserving image compression method, which includes obtaining an original image, determining a retrieval reference image based on the original image, performing image compression and quantization on the retrieval reference image to determine a first quantization feature and a second quantization feature, performing entropy loss modeling on the probability distribution of the second quantization feature using a factor decomposition density model to determine an entropy loss, embedding the online semantic features of the original image into the first quantization feature based on the entropy loss to generate a reconstructed image, calculating a signal preservation loss and a semantic embedding loss based on the original image, the retrieval reference image, and the reconstructed image, and constraining the latent representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image. This application obtains the original image and retrieves a semantically similar reference image, compresses and quantizes the reference image to obtain a potential representation, uses a factor decomposition density model to model entropy loss, and embeds the semantic features of the original image into the quantized features to generate a reconstructed image. The potential representation of the reconstructed image is constrained by calculating the signal preservation loss and the semantic embedding loss, and finally the target reconstructed image is generated, which effectively protects the privacy of the image and achieves efficient image compression at a low bit rate. At the same time, it ensures that the performance of the reconstructed image in human visual perception and machine vision analysis tasks is close to that of the original image, significantly improving the performance and application scope of the image in machine vision analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A flowchart of the first embodiment of the privacy protection image compression method of this application is provided;

[0051] Figure 2 Schematic diagram of obtaining album picture data for the privacy protection image compression method of this application;

[0052] Figure 3 A flowchart of the second embodiment of the privacy protection image compression method of this application is provided;

[0053] Figure 4 A brief flowchart of the privacy-preserving image compression method provided in an embodiment of the present application;

[0054] Figure 5Schematic diagram of the experimental results of the privacy-preserving image compression method in this application on image classification, image-text retrieval, and image description;

[0055] Figure 6 Schematic diagram of the experimental results of the privacy-preserving image compression method in this application on human perception tasks;

[0056] Figure 7 This is a schematic diagram of the module structure of the privacy protection image compression device according to an embodiment of the present application;

[0057] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the privacy-preserving image compression method in the embodiment of the present application.

[0058] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0060] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0061] The main solution of the embodiment of the present application is: obtaining an original image, determining a retrieval reference image based on the original image; performing image compression and quantization on the retrieval reference image to represent the potential, and determining a first quantization feature and a second quantization feature; performing entropy loss modeling on the probability distribution of the second quantization feature using a factor decomposition density model to determine the entropy loss; embedding the online semantic features of the original image into the first quantization feature based on the entropy loss to generate a reconstructed image; calculating the signal preservation loss and the semantic embedding loss based on the original image, the retrieval reference image and the reconstructed image; constraining the potential representation of the reconstructed image based on the entropy loss, the signal preservation loss and the semantic embedding loss to generate a target reconstructed image.

[0062] In this embodiment, for ease of description, the following description is made by taking the identification of a privacy-preserving image compression device as the execution subject.

[0063] Since image steganography in existing technologies usually assumes a lossless transmission channel, in reality, lossy image processing operations such as compression, scaling, and cropping may make it impossible to accurately extract the embedded secret information. The goal of steganography is to transmit secret information, but it does not consider the comprehensiveness of information utilization in conjunction with cloud models. In addition, in image steganography, the image and secret information are not considered jointly, resulting in poor performance in collaborative scenarios.

[0064] The present application provides a solution, which includes obtaining an original image, determining a retrieval reference image based on the original image; performing image compression and quantization on the retrieval reference image to represent a potential, and determining a first quantization feature and a second quantization feature; performing entropy loss modeling on the probability distribution of the second quantization feature using a factor decomposition density model to determine the entropy loss; embedding the online semantic features of the original image into the first quantization feature based on the entropy loss to generate a reconstructed image; calculating a signal preservation loss and a semantic embedding loss based on the original image, the retrieval reference image, and the reconstructed image; and constraining the potential representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

[0065] It can be seen from the above embodiments that the present application obtains the original image and retrieves a semantically similar reference image, compresses and quantizes the reference image to obtain a potential representation, uses a factor decomposition density model to model the entropy loss, and embeds the semantic features of the original image into the quantized features to generate a reconstructed image. The potential representation of the reconstructed image is constrained by calculating the signal preservation loss and the semantic embedding loss, and finally the target reconstructed image is generated, which effectively protects the privacy of the image and achieves efficient image compression at a low bit rate. At the same time, it ensures that the performance of the reconstructed image in human visual perception and machine vision analysis tasks is close to that of the original image, significantly improving the performance and application scope of the image in machine vision analysis.

[0066] Based on this, the embodiment of the present application provides a privacy protection image compression method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the privacy protection image compression method of this application.

[0067] In this embodiment, the privacy-preserving image compression method includes steps S10 to S60:

[0068] Step S10, obtaining an original image, and determining a retrieval reference image based on the original image;

[0069] It should be noted that the original image is an image that needs to be privacy protected and compressed, and the user-related data involved, such as the user's sensitive information or personal privacy content, are all obtained after obtaining the user's permission or consent. That is to say, when this application is applied to specific products or technologies, it is necessary to obtain user permission to obtain and process the relevant data, and the processing of the relevant data needs to comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions. For example, Figure 2 As shown, Figure 2This is a schematic diagram of obtaining album picture data using the privacy protection image compression method of this application. When it is necessary to obtain the user's album picture data, a prompt for obtaining the album picture data can be displayed in the user's terminal. After receiving the user's confirmation operation for the prompt for obtaining the album picture data, the terminal can obtain the user's album picture data.

[0070] It can be understood that the retrieval reference image is obtained by retrieving the original image from a large-scale data set based on semantic similarity. The retrieval reference image obtained is semantically similar to the original image, but there are differences in pixel-level details. Using a retrieval reference image that is semantically similar to the original image to replace the original image for compression processing can effectively protect image privacy without exposing the fine-grained details of the original image.

[0071] In a feasible implementation, step S10 may include steps A11 to A14:

[0072] Step A11, matching the corresponding target retrieval sample image with the original image and the pre-built vector database to determine a database sample image set;

[0073] It should be noted that the target retrieval sample image is a single image that is semantically similar to the original image and is matched from a pre-constructed vector database, while the database sample image set is all image samples that are successfully matched in the pre-constructed vector database. After obtaining the original image, a reference image with semantic similarity to the original image is found from the pre-constructed vector database through semantic similarity retrieval. The result will be multiple images, and these images can be integrated to obtain the database sample image set.

[0074] It is understandable that in order to facilitate the matching of semantically similar images, it is necessary to pre-build a large-scale vector database, which can contain image feature vectors extracted from large-scale data sets. For example, a comprehensive vector database can be built using CLIP on a CC3M data set containing three million image-text pairs. That is, the CLIP model is used to extract features from each image in the CC3M data set, and the high-dimensional feature vectors are stored in the vector database, forming a comprehensive image feature library for similarity retrieval.

[0075] In addition, it should be noted that using the pre-built vector database for matching can select multiple target retrieval sample images with similar semantics but different pixel-level details, effectively hiding the fine-grained details of the original image and reducing the risk of privacy leakage. Even if the target retrieval sample image is leaked, the specific content of the original image cannot be directly restored. Therefore, the resulting database sample image set is obviously more secure and significantly enhances the effect of image privacy protection.

[0076] Step A12, using an image pre-training model to extract feature vectors of the original image and the database sample image set, calculate cosine similarity, and determine image similarity;

[0077] It should be noted that the feature vector is a vector extracted from the corresponding image through a pre-trained image model, which is used to represent the semantic information of the image. It can capture the detailed features of the image and quantify the semantic similarity of the image. The image similarity is the cosine similarity calculated between the feature vectors of the two images, where the value range of the cosine similarity is between [-1, 1]. The closer the value is to 1, the more similar the two images are semantically.

[0078] It is understandable that the CLIP model can be used to extract the semantic features of the image and calculate its cosine similarity with the images in the database, thereby quantifying the semantic similarity between the query image and each image in the database, and selecting a semantically close reference image. While protecting the privacy of the image, it ensures that the reference image has a high semantic similarity with the original image. When an original image x is given, o When , first perform similarity search in the database to match the corresponding target retrieval sample image, and then calculate x o The similarity between the feature vector of and the samples in the vector database: I = Clip(x o ,x), where Clip(·) means using the CLIP model to extract the feature vectors of two input samples respectively and calculate the cosine similarity between them, and x represents the sample in the vector database.

[0079] In addition, it should be noted that by calculating cosine similarity to determine image similarity, the semantic association between images can be accurately measured. Specifically, cosine similarity focuses on the vector direction and is highly sensitive to the semantic content of the image. It can accurately characterize the degree of semantic matching, that is, obtain image similarity. Image similarity can be used to select multiple reference images with similar semantics but different pixel-level details, thereby effectively hiding the details of the original image and reducing the risk of privacy leakage. Pixel-level information will not be exposed during the retrieval process, significantly enhancing privacy protection.

[0080] Step A13, sorting the database sample image set according to the image similarity to determine a preceding sample image set;

[0081] It should be noted that the preceding sample image set is a set consisting of the top k sample images with the highest similarity selected after sorting all sample images in the database sample image set based on the calculated image similarity, which are semantically close to the original image. The sorting is to sort all sample images in the database sample image set from high to low according to the similarity with the original image, with the image with the highest similarity placed in front and the image with the lowest similarity placed in the back, and the top k sample images are selected from the sorted sample images to form the preceding sample image set.

[0082] It can be understood that sorting by image similarity can quickly locate images that are semantically similar to the original image, thereby improving retrieval efficiency. In large-scale data sets, selecting the first k sample images to obtain a set of preceding sample images for processing can significantly reduce the amount of computation, ignore unusable dissimilar images, and significantly improve the response speed of the system. The resulting set of preceding sample images is highly semantically consistent with the original image, ensuring that the performance of the reconstructed image in machine vision analysis tasks is close to that of the original image, significantly improving the performance and application scope of the image in machine vision analysis.

[0083] Step A14: selecting a target image based on the preceding sample image set to obtain a retrieval reference image.

[0084] It should be noted that after obtaining the previous sample image set, in order to avoid the intuitive similarity between the original image and the retrieved reference image, the mth sample in the previous sample image set is selected as the retrieval reference image x r ,Because the first sample has a high semantic and appearance similarity with the ,original image, which may cause hidden information leakage, ,the first ranked sample is not selected to enhance the ,effect of privacy protection.

[0085] It is understandable that in order to avoid the original image and the retrieved reference image being too visually similar, which may leak the details of the original image, free sample selection can be achieved by controlling the size of m. Specifically, m here is an integer greater than 1, and the specific value can be adjusted according to actual needs and the strictness of privacy protection. For example, if the requirement for privacy protection is high, a larger m value can be selected to ensure that the visual difference between the reference image and the original image is large enough. Conversely, if the requirement for image quality is high, a smaller m value can be selected to ensure a high degree of semantic consistency between the reference image and the original image.

[0086] Step S20, performing image compression quantization potential representation on the retrieved reference image to determine a first quantization feature and a second quantization feature;

[0087] It should be noted that the first quantitative feature is a feature obtained by quantizing the latent representation obtained after image compression encoding of the retrieval reference image, which is used for image compression and reconstruction and is an intermediate feature vector generated during the compression process. Specifically, the encoder network of the deep learning model can be used to realize the quantization of the extracted semantic features of the retrieval reference image. The encoder converts the pixel-level information of the image into a low-dimensional latent representation, which contains the main information of the image but is represented in a more compact form.

[0088] It can be understood that the second quantized feature is a feature obtained by quantizing the super-prior representation in the potential representation of the retrieved reference image, which can more accurately estimate the bit rate and optimize the compression efficiency. Specifically, the encoder network of the deep learning model can be used to realize the quantization of the super-prior representation, which is used to learn the spatial dependency in the potential representation, thereby further optimizing the compression efficiency.

[0089] In addition, it should be noted that through quantization processing, continuous potential representations can be converted into discrete representations, significantly reducing the storage and transmission costs of data. The combined use of the first quantization feature and the second quantization feature can achieve efficient image compression at low bit rates, significantly optimizing compression efficiency.

[0090] In a feasible implementation, step S20 may include steps B11 to B14:

[0091] Step B11, extracting semantic features of the retrieval reference image and inputting them into a deep learning model for image compression coding to determine a potential representation;

[0092] It should be noted that the latent representation is a low-dimensional feature representation obtained after the extracted semantic features of the retrieval reference image are processed by the encoder. It is a compact feature vector that represents the main information of the image and removes redundant information to facilitate transmission and processing. Specifically, the CLIP model is used to extract the semantic features of the retrieval reference image and input them into a pre-trained deep learning model. The model can be a convolutional neural network CNN or other network structure suitable for image processing, including an encoder network and a decoder network, wherein the encoder network is part of the deep learning model, which converts the input semantic features of the retrieval reference image into a low-dimensional latent representation, that is, the encoder network gradually reduces the dimension of the features through a series of convolutional layers and nonlinear activation functions while retaining the main information of the image. For example, assuming that the size of the input image is 256×256×3, where the height of the image is 256 and the unit is 1. is a pixel, the width of the image is 256, and the unit is pixel. At this time, it indicates that the image has 256 rows, each row has 256 pixels, and the number of channels of the image is 3, indicating that the color image has 3 channels, corresponding to red R, green G and blue B respectively. After processing by the encoder, a potential representation of size 32×32×128 will be obtained. The potential representation is a low-dimensional feature representation, which significantly reduces the storage and transmission cost of data. For example, the original image may contain millions of pixels, while the potential representation has only a few thousand eigenvalues. Moreover, despite the reduction in dimensionality, the potential representation can still retain the main semantic information of the image. Therefore, even at low bit rates, the reconstructed image can maintain semantic content similar to the original image. At the same time, the potential representation can be reconstructed into an output close to the original image through the decoder network, where the role of the decoder network is to restore the low-dimensional potential representation to high-dimensional image data.

[0093] It can be understood that the use of the latent representation can achieve efficient image compression and privacy protection, significantly improve the performance of machine vision analysis tasks, and maintain the consistency of human visual perception. Moreover, as a low-dimensional feature representation, the latent representation can significantly reduce the storage and transmission costs of data while retaining the main semantic information of the image, thereby achieving efficient compression at a low bit rate.

[0094] In a specific embodiment, an end-to-end image compression network using the deep learning model as a backbone for image compression may include an encoder and a decoder, for example, to retrieve a reference image x r It can be encoded into a latent representation y by the encoder E(·) and can be expressed as:

[0095] y=E(x r )

[0096] At this time, the potential representation y will contain redundant information. Directly entropy encoding of y can only obtain a suboptimal solution that minimizes the bit rate. Therefore, it is still necessary to use the entropy model to introduce a super prior to solve the spatial dependence of y, so as to more accurately estimate the entropy model and reduce the bit rate.

[0097] Step B12, using a super prior model to capture the spatial dependency between the corresponding elements of the potential representation, and determining a super prior representation;

[0098] It should be noted that the super-prior representation is obtained by capturing the spatial dependency between elements in the latent representation through a super-prior model, and is used to accurately estimate the probability distribution of the latent representation, thereby effectively improving the compression efficiency. The super-prior model is a network structure for capturing the spatial dependency between elements in the latent representation, and generates a super-prior representation by analyzing the contextual information of the latent representation. Specifically, the super-prior model can also be a convolutional neural network CNN or other network structure suitable for processing spatial dependencies. Through multi-layer convolution operations, the spatial dependency between elements in the latent representation is gradually extracted. The dependency can be expressed as the interaction between elements in the latent representation, for example, the correlation between adjacent elements or the dependency between distant elements, thereby receiving the latent representation as input and outputting a super-prior representation.

[0099] It can be understood that the super-prior representation can be used as auxiliary information to help the encoder encode the potential representation more efficiently. By utilizing the super-prior representation, the encoder can more accurately estimate the probability distribution of the potential representation, thereby achieving more efficient entropy coding. The entropy coding is a method in image compression that further reduces the storage and transmission costs of data by reducing redundant information, significantly improving compression efficiency. Especially at low bit rates (BPP), by more accurately estimating the probability distribution of the potential representation, the encoder can allocate bits more efficiently, thereby retaining more image information within a limited number of bits.

[0100] In a specific embodiment, because the potential representation y contains redundant information, directly entropy coding y can only obtain a suboptimal solution that minimizes the bit rate. Therefore, the entropy model is introduced into the super prior to obtain a super prior model to solve the spatial dependence of y, thereby more accurately estimating the entropy model and reducing the bit rate. The super prior model can be expressed as:

[0101] z=H(y)

[0102] Here, H(·) is a super-prior encoder that is used to capture the spatial dependencies between each element of y, thereby obtaining a super-prior representation z.

[0103] Step B13, quantizing the potential representation using a quantization operator to obtain a first quantized feature;

[0104] It is understandable that because continuous potential representations are mapped to discrete values, a certain amount of information loss will occur during the quantization process. However, by optimizing the quantization process, this information loss can be minimized while achieving efficient compression. Although the quantization process is irreversible, the quantized features can be subsequently reconstructed into outputs close to the original image through the decoder network. The role of the decoder network is to restore discrete quantized features to high-dimensional image data, ensuring that the performance of the reconstructed image in human visual perception and machine vision analysis tasks is close to the original image.

[0105] In a specific embodiment, the potential representation is quantized using a quantization operator to obtain a first quantized feature, and the potential representation y is quantized into the first quantized feature using a quantization operator Q(·) It can be expressed as:

[0106]

[0107] Among them, μ is the mean, and quantization converts the continuous potential representation into a discrete representation, which significantly reduces the storage and transmission cost of data, reduces the amount of data, and maps the continuous eigenvalues ​​to a finite discrete set, thereby reducing redundant information, improving compression efficiency, and retaining more image information within a limited number of bits.

[0108] Step B14: quantize the super-prior representation using a quantization operator to obtain a second quantized feature.

[0109] It should be noted that the quantization operator can be a mathematical function of a neural network layer, which optimizes the quantization process through training and is used to map each element in the continuous potential representation to a finite discrete set. It can be regarded as rounding or truncating the potential representation, thereby reducing the accuracy and storage requirements of the data, such as rounding or taking the nearest integer value. For example, assuming that the size of the potential representation is 32×32×128, each element is a floating point number representing the feature value of the image.

[0110] It can be understood that the quantization operator maps each floating-point number to a discrete value, rounds each floating-point number to the nearest integer value, or maps them to a predefined discrete set. After quantization, the obtained quantized feature is a discrete feature representation. Assume that the feature size after quantization is still 32×32×128, but each element is now a discrete value instead of a floating-point number.

[0111] In a specific embodiment, the super-prior representation is quantized using a quantization operator to obtain a second quantized feature, which can be expressed as:

[0112]

[0113] At this time, the super prior representation z is quantized by the quantization operator Q(·) into the second quantized feature At this time, the quantized second quantized features retain the key information in the original latent representation, improve compression efficiency, and enhance privacy protection capabilities, ensuring the quality and performance of the reconstructed image at a low bit rate.

[0114] Step S30, performing entropy loss modeling on the probability distribution of the second quantized feature using a factor decomposition density model to determine the entropy loss;

[0115] It should be noted that the entropy loss is the degree of information loss during the compression process. It should be understood that entropy represents the amount of information of each eigenvalue and is used to measure the uncertainty of information. The higher the entropy, the more uncertain the information is and more bits are required to encode it. The entropy loss is the information loss caused by quantization and encoding. Specifically, the entropy loss measures the information difference between the compressed eigenvalue and the original eigenvalue. By minimizing the entropy loss, redundant information can be reduced and compression efficiency can be improved.

[0116] It can be understood that the entropy loss can be determined by performing entropy loss modeling on the probability distribution of the second quantized feature through a factor decomposition density model, wherein the purpose of entropy loss modeling is to estimate the code stream of the quantized feature more accurately by estimating the probability distribution of the quantized feature, optimize the compression efficiency, and ensure the quality of the reconstructed image. The factor decomposition density model is a probability model used to estimate the probability distribution of the quantized feature. By decomposing the complex joint probability distribution into multiple simple conditional probability distributions, the probability of each eigenvalue can be estimated more efficiently. For example, assuming that the size of the second quantized feature is 32×32×128, the factor decomposition density model can decompose the probability distribution of each eigenvalue into a conditional probability distribution, that is, the factor decomposition density model assigns a probability value to each eigenvalue by learning the probability distribution of the quantized feature, which is used to characterize the frequency of occurrence of the eigenvalue in the data.

[0117] In a specific embodiment, the second quantized feature is quantized using the factorized density model Ψ. Model the probability distribution of to estimate more accurately The code stream is jointly optimized in the form of variational estimation. The factorized density model decomposes the complex probability distribution into multiple simple conditional probability distributions, making the encoding of each eigenvalue more accurate and reducing redundant information. The variational estimation is a statistical method used to approximate complex probability distributions and optimize the model parameters by minimizing the difference between the model distribution and the true distribution. This joint optimization method makes the entire compression and decoding process more efficient and accurate, thereby achieving efficient image compression at low bit rates while maintaining high image quality and privacy protection. At this time, the entropy loss L is calculated. enIt can be expressed as:

[0118]

[0119] Among them, E represents expectation, express under conditions A learnable probability distribution, such as a Gaussian distribution, Indicates Ψ condition The learnable probability distribution of , where the first term is used to accurately estimate The probability distribution of the coding symbols is realized The second term is used to accurately estimate The probability distribution of the coding symbols is realized Highly efficient compression, in practical applications, and Based on the learned probability distribution model, range coding is used for lossless compression for efficient transmission, and the decoding end decodes the data using the same probability distribution model. and

[0120] Step S40, embedding the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image;

[0121] It should be noted that the reconstructed image is an image reconstructed at the decoder end based on the compressed feature information. The goal of entropy loss is to minimize the entropy of the probability distribution of the eigenvalues, thereby reducing redundant information and improving compression efficiency. By minimizing the entropy loss, it can ensure that the probability distribution of the quantized features is more compact, thereby reducing the number of bits required for encoding. Therefore, entropy loss needs to be considered during embedding for efficient encoding and decoding.

[0122] It can be understood that, based on the entropy loss, the online semantic features of the original image are embedded into the first quantized features in the form of variational prediction. This method has a certain flexibility to better adapt to different image contents and compression requirements. At this time, the encoding obtained through the embedding process is a joint optimization encoding, which contains the semantic information of the original image and takes into account compression efficiency and decoding quality.

[0123] In a specific embodiment, the online semantic features of the original image are embedded into the first quantitative features based on the entropy loss, and the generated quantitative features are optimized. After this, the Input decoder D(·), and you can get the reconstructed image x t , which can be expressed as:

[0124]

[0125] The use of the reconstructed image can ensure that the performance of downstream tasks is close to the original image without exposing the fine-grained details of the original image, more securely protect the privacy of the image, and significantly improve the performance and application scope of the image in machine vision analysis while protecting privacy.

[0126] In a feasible implementation, step S40 may include steps C11 to C12:

[0127] Step C11, embedding the online semantic features of the original image into the first quantized features in the form of variational prediction according to the entropy loss, and determining a joint optimized coding;

[0128] It should be noted that the joint optimization coding is an optimized coding representation generated by optimizing the image compression coding based on entropy loss and embedding it in the form of variational prediction. The coding representation not only contains the compression features of the image, but also embeds the semantic information of the original image, so that high-quality reconstructed images can be generated during decoding.

[0129] It can be understood that by minimizing the entropy loss, joint optimization coding can make the probability distribution of eigenvalues ​​as close to the true distribution as possible, thereby achieving more efficient entropy coding, and the decoder can reconstruct the image more accurately, thereby improving the quality and performance of the reconstructed image and effectively protecting the privacy of the image.

[0130] Step C12: Decode and reconstruct the joint optimized code to obtain a reconstructed image.

[0131] It should be noted that the reconstructed image is also visually as close as possible to the original image, and high visual quality can be maintained even at a low bit rate by minimizing signal retention loss, ensuring that the visual difference between the reconstructed image and the reference image is minimal.

[0132] It can be understood that the joint optimization code is input into the decoder for decoding and reconstruction, wherein the task of the decoder is to reconstruct a reconstructed image that is as close as possible to the original image based on the encoded feature information, thereby generating a reconstructed image through the processing of the decoder. At this time, the reconstructed image will be as close as possible to the original image both visually and semantically, and ensures efficient encoding and decoding of information during the compression process.

[0133] Step S50, calculating a signal preservation loss and a semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image;

[0134] It should be noted that the signal preservation loss is a measure of the pixel-level similarity between the reconstructed image and the retrieved reference image, which is used to maintain image quality. It can be calculated using the mean square error or other distance functions. By minimizing the signal preservation loss, it is ensured that the reconstructed image is visually consistent with the reference image, thereby achieving efficient image compression at a low bit rate while maintaining high visual quality.

[0135] It can be understood that the semantic embedding loss is a measure of the semantic similarity between the reconstructed image and the original image, which is used to maintain the semantic information of the image during image compression and decompression. It can be measured by calculating the cosine similarity between feature vectors. By minimizing the semantic embedding loss, the reconstructed image can be semantically consistent with the original image, so that the performance of the reconstructed image in the machine vision analysis task is close to that of the original image, and by embedding the semantic features of the original image into the latent representation of the retrieved reference image, the performance of the downstream task is close to that of the original image without exposing the fine-grained details of the original image, thereby effectively protecting the privacy of the image and achieving efficient image compression at a low bit rate.

[0136] In a specific embodiment, in order to embed semantic features while minimizing the degradation of reconstructed image quality, an online optimized encoding process is used to convert the original image x o The semantic embedding of the retrieved reference image x r In

[15] , the rate-distortion / performance can be flexibly controlled. Specifically, from the perspective of information theory, embedding semantic features under the same bit rate constraint will inevitably lead to a decrease in the quality of the reconstructed image. In order to improve the compression efficiency and reduce the optimization complexity, a sample-by-sample online optimization mechanism is adopted, in which the parameters of the decoder D(·) are fixed and only the parameters of the encoder E(·) are trainable. Therefore, the signal preservation loss L can be used. sp , semantic embedding loss L se and entropy loss L en Three constraints are optimized, and the entropy loss L en The above calculation method can be used for calculation. Here, the signal holding loss L needs to be calculated. sp and semantic embedding loss L se .

[0137] Signal retention loss L sp The purpose is to keep the retrieved reference image x r and the reconstructed image x t The fidelity between the signals is maintained at a loss of L sp It can be expressed as:

[0138] L sp =d(x r ,x t )

[0139] Where d(·) is the distance function, expressed as x r and x t The mean square error MSE between them.

[0140] Semantic embedding loss L se It is mainly used to meet the growing demand for machine-oriented image coding and increasingly prominent information security. It can embed the high-level semantics of the original image into the retrieval reference image to ensure that the original image remains secret. Therefore, the semantic embedding loss L se The optimization goal is to minimize the cosine similarity between the original image and the reconstructed image, and the semantic embedding loss L se It can be expressed as:

[0141] L se =cos(f(x o ),f(x t ))

[0142] Among them, f(x o ) and f(x t ) represent the original image x extracted using the CLIP image encoder o and the reconstructed image x t The feature vector of , i.e., the semantic feature, embeds the semantics of the original image into the latent representation of the retrieved reference image by minimizing this objective function.

[0143] In a feasible implementation, step S50 may include steps D11 to D12:

[0144] Step D11, extracting semantic features of the retrieved reference image and the reconstructed image to calculate signal preservation loss;

[0145] It should be noted that the pre-trained CLIP model can be used to extract the semantic features of the reference image and the reconstructed image, and the signal preservation loss of the extracted semantic features can be calculated using the distance function. The distance function is a mathematical function that measures the difference between two images and can have various forms, such as x r and x t The mean square error is a measure of the pixel-level difference between two images.

[0146] It can be understood that the mean square error calculates the average value of the square of the corresponding pixel value differences between the retrieved reference image and the reconstructed image. The smaller the mean square error, the smaller the difference between the two images at the pixel level, that is, the reconstructed image is visually closer to the retrieved reference image. In the context of image compression, minimizing the mean square error can maintain the visual quality of the compressed image.

[0147] Step D12: extracting semantic features of the original image and the reconstructed image, calculating the cosine similarity between the target images, and determining the semantic embedding loss.

[0148] It should be noted that the cosine similarity between target images is an indicator for measuring the semantic similarity between two images, and the degree of semantic closeness is determined by calculating the cosine angle between the feature vectors of the two images.

[0149] It can be understood that the pre-trained CLIP model can be used to extract the semantic features of the original image and the reconstructed image, and then the cosine similarity between these feature vectors is calculated. Specifically, the semantic embedding loss is defined by minimizing the cosine similarity between the original image and the reconstructed image, which means that through this optimization process, the reconstructed image can be adjusted to be as semantically close to the original image as possible. In this way, even if the pixel-level details of the image change during the compression process, the reconstructed image can still retain the semantic information of the original image, thereby maintaining high performance in machine vision analysis tasks. By minimizing the semantic embedding loss, it can be ensured that the performance of downstream tasks is close to the original image without exposing the fine-grained details of the original image, effectively protecting the privacy of the image.

[0150] Step S60 , constraining the latent representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

[0151] It should be noted that the target reconstructed image is generated without exposing the fine-grained details of the original image. Since the semantic information of the original image is embedded in the reference image instead of directly transmitting the pixel data of the original image, the privacy of the original image is protected. Even if the target reconstructed image is leaked, the specific content of the original image cannot be directly restored, thereby reducing the risk of privacy leakage. Therefore, it can be used for tasks such as machine vision analysis.

[0152] It can be understood that the target reconstructed image is the final image generated after optimizing by balancing multiple encoder losses. During the optimization process, the decoder parameters are fixed and only the encoder parameters are adjusted to achieve constraints on the potential representation of the reconstructed image. Therefore, the target reconstructed image achieves efficient compression and privacy protection while maintaining semantic and visual quality. For example, by adjusting the encoder parameters, the relationship between compression efficiency, visual quality and privacy protection can be flexibly balanced in different application scenarios.

[0153] This embodiment proposes a privacy-preserving image compression method, which includes obtaining an original image, determining a retrieval reference image based on the original image, performing image compression and quantization on the retrieval reference image to determine a first quantization feature and a second quantization feature, performing entropy loss modeling on the probability distribution of the second quantization feature using a factor decomposition density model to determine an entropy loss, embedding the online semantic features of the original image into the first quantization feature based on the entropy loss to generate a reconstructed image, calculating a signal preservation loss and a semantic embedding loss based on the original image, the retrieval reference image, and the reconstructed image, and constraining the latent representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image. The invention solves the technical problem of how to ensure the secret transmission of secret information under lossy compression and flexibly control the bit rate distortion and transmission performance. Compared with the existing technology, the invention obtains the original image and retrieves the semantically similar reference image, compresses and quantizes the reference image to obtain the potential representation, uses the factor decomposition density model to model the entropy loss, and embeds the semantic features of the original image into the quantized features to generate a reconstructed image. The potential representation of the reconstructed image is constrained by calculating the signal preservation loss and the semantic embedding loss, and finally the target reconstructed image is generated. It should be understood that at present, the image content will be directly exposed when uploading to the cloud, and there is a risk of privacy leakage. The invention embeds the semantic information of the original image into the lossy code of the reference image. In the coding form, the contrastive language-image pre-training model is used to align the feature maps of the original image and the reconstructed image, and the semantic embedding loss is used to ensure that the reconstructed image is highly semantically consistent with the original image. A two-stage optimization scheme is adopted to jointly minimize the signal preservation loss and the semantic embedding loss, and the compactness of the potential representation is optimized through entropy loss to ensure efficient compression at low bit rates. At the same time, an online optimization mechanism is introduced to fix the decoder parameters and only update the encoder parameters, which further improves the compression efficiency and the performance of machine vision analysis tasks. Combined with the super-prior encoder and the factorized density model, the coding efficiency of the potential representation is optimized, so that the performance of the reconstructed image in downstream tasks is close to that of the original image, effectively protecting image privacy.

[0154] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction and will not be repeated later.

[0155] In this embodiment, refer to Figure 3 , Figure 3 This is a flowchart of the second embodiment of the privacy protection image compression method of this application. Step S60 specifically includes steps S61 to S63:

[0156] Step S61, obtaining a compactness trade-off factor;

[0157] It should be noted that the compactness trade-off factor is a non-negative scalar value, denoted as γ and λ, which is used to balance the trade-off factors of human visual perception, machine vision analysis and potential representation compactness. During the optimization process, the weights between entropy loss, signal preservation loss and semantic embedding loss can be balanced.

[0158] It is understandable that the selection of the compactness trade-off factor needs to be adjusted according to specific application requirements and scenarios. For example, different combinations of γ and λ values ​​can be tested on different data sets through cross-validation, so as to select the combination that performs best on the validation set, achieve a better balance between visual quality and compression efficiency, and achieve the best optimization effect.

[0159] Step S62, determining an overall objective function based on the compactness trade-off factor, the entropy loss, the signal preservation loss, and the semantic embedding loss;

[0160] It should be noted that the overall objective function is used to comprehensively measure and optimize the quality of the reconstructed image to calculate a weighted sum, which combines three key loss terms, entropy loss, signal preservation loss and semantic embedding loss, and is balanced by a compactness trade-off factor.

[0161] It can be understood that the overall objective function can be used to reversely derive the optimal entropy loss, signal preservation loss, and semantic embedding loss. Specifically, adjusting the entropy loss can ensure that the compressed features have a compact representation, thereby improving compression efficiency. By minimizing the entropy loss, redundant information can be reduced and coding efficiency can be optimized. Adjusting the signal preservation loss can ensure that the reconstructed image is visually consistent with the reference image, thereby maintaining high visual quality, and reducing pixel-level differences to ensure visual similarity of the image. Adjusting the semantic embedding loss can ensure that the reconstructed image is semantically consistent with the original image, thereby maintaining high performance in machine vision analysis tasks and effectively retaining the semantic information of the image. By adjusting the compactness trade-off factor, compression efficiency, visual quality, and semantic preservation can be flexibly weighed in different application scenarios. By jointly optimizing multiple loss terms, a target reconstructed image with excellent performance in multiple aspects can be generated to better meet the diverse needs in practical applications and significantly enhance image privacy protection.

[0162] In a specific embodiment, an overall objective function is determined based on the compactness trade-off factor, the entropy loss, the signal preservation loss, and the semantic embedding loss. The overall objective function can be expressed as:

[0163] L=L en +λL sp +γL se

[0164] Among them, γ and λ are trade-off factors used to balance human visual perception, machine visual analysis, and the compactness of the potential representation, namely the compactness trade-off factor. By minimizing this objective function, the compactness of the potential representation can be constrained, the consistency of human visual perception of the reconstructed image and the original image can be maintained, and the machine visual analysis performance of the original image and the retrieved image can be made as close as possible.

[0165] Step S63 : constraining the potential representation of the reconstructed image based on the overall objective function to obtain a target reconstructed image.

[0166] It should be noted that the target reconstructed image is the image that is finally generated after optimization processing by comprehensively considering multiple constraints such as entropy loss, signal preservation loss and semantic embedding loss without exposing the fine-grained details of the original image.

[0167] It can be understood that the target reconstructed image is highly consistent with the original image in semantics, close to the retrieved reference image in visual quality, and has a compact latent representation, thereby achieving efficient compression and privacy protection at low bit rate.

[0168] In addition, it should be noted that since the target reconstructed image is highly similar to the original image at the semantic level, even if the pixel-level details of the image change during the compression process, the target reconstructed image can still retain the semantic information of the original image, thereby maintaining high performance in machine vision analysis tasks. For example, in an image classification task, if the original image is a cat, the target reconstructed image can also be correctly identified as a cat, and its semantic information is not lost. This makes the reconstructed image have similar performance to the original image in various machine vision analysis tasks, and the target reconstructed image can maintain a visual effect similar to the retrieved reference image, although there are certain differences at the pixel level from the original image. , but still has high visual fidelity and will not cause obvious visual distortion. In the actual image compression and transmission process, the target reconstructed image can achieve efficient compression at a lower bit rate, significantly reducing data storage and transmission costs while ensuring the quality and performance of the reconstructed image. It is particularly suitable for scenarios where edge devices collaborate with the cloud, such as monitoring systems or smartphones uploading image data to the cloud. At the same time, in image processing and analysis tasks that need to protect user privacy, such as remote diagnosis of medical images, cloud storage of personal photo albums, etc., the target reconstructed image can meet the needs of tasks such as machine vision analysis without exposing user sensitive information.

[0169] In a specific embodiment, in order to clearly express the correlation between the network structure parameters and the input and output variables, the overall objective function is expanded in detail, and its expansion is expressed as follows:

[0170]

[0171] in, Represents the entropy model, the input is the original image x o , the encoder encodes it into a latent representation y = E φ (x o ), then a series of entropy coding is performed on y, and the main optimization target is the encoder parameter φ; φ、 θ represents the entropy model The parameters of the encoder E(·), decoder D(·) and feature extraction network f(·), the objective function is fixed and θ, only optimize φ, specifically, given the model input x o and x r , in the forward propagation process, the model is based on the pre-trained parameters of each network module { φ, θ}, the above expansion can be used to calculate the difference between the predicted value and the true value, and the back propagation process calculates the gradient of each parameter in φ based on the difference and updates φ. Finally, after multiple iterative training, the encoder E φ (·) will learn how to convert x o Mapping to the optimal latent space ensures the compactness of the latent representation while taking into account the reconstruction quality and semantic preservation.

[0172] In a feasible implementation, step S63 may include steps E11 to E12:

[0173] Step E11, determining entropy model parameters, encoder parameters, decoder parameters and feature extraction network parameters based on the overall objective function;

[0174] It should be noted that the entropy model parameters are parameters used to estimate the probability distribution of quantized features. By accurately estimating the probability distribution of features, optimizing the entropy coding process, and reducing redundant information, more efficient compression can be achieved. Specifically, a neural network can be used to construct an entropy model, such as a convolutional neural network (CNN). The weights and bias parameters of the neural network are the entropy model parameters.

[0175] It can be understood that the encoder parameters are parameters used to convert the input image into a low-dimensional potential representation, which is responsible for converting the pixel-level information of the image into a more compact feature representation, converting the semantic features of the input image into a low-dimensional potential representation, and removing redundant information. Specifically, the encoder can be composed of a multi-layer convolutional neural network CNN, and the weight and bias parameters of each layer are the encoder parameters.

[0176] It should be noted that the decoder parameters are parameters used to reconstruct the low-dimensional latent representation into a high-dimensional image. They are responsible for restoring the compressed feature information to an output close to the original image, and restoring the low-dimensional latent representation to high-dimensional image data, ensuring that the reconstructed image is consistent with the original image in terms of vision and semantics. Specifically, the decoder can be composed of a multi-layer deconvolutional neural network, and the weight and bias parameters of each layer are the decoder parameters.

[0177] It can be understood that the feature extraction network parameters are parameters used to extract the semantic features of the image, calculate the similarity between images and embed semantic information, and use a pre-trained deep learning model to extract the semantic features of the image. The weight and bias parameters of the model are the feature extraction network parameters.

[0178] Step E12: fixing the entropy model parameters, the decoder parameters, and the feature extraction network parameters, and adjusting the encoder parameters to constrain the potential representation of the reconstructed image to obtain a target reconstructed image.

[0179] It can be understood that fixing the entropy model parameters can ensure the stability and efficiency of entropy coding, fixing the decoder parameters can stably reconstruct the compressed feature information into a high-quality image, and fixing the feature extraction network parameters can keep the semantic features of the image stable and consistent. When the entropy model parameters, the decoder parameters, and the feature extraction network parameters are all fixed, the encoder parameters can be adjusted to optimize the overall objective function, thereby constraining the potential representation of the reconstructed image. First, the initial encoder parameters are loaded from the pre-trained model, and the loss value under the current encoder parameters is calculated according to the overall objective function. The gradient of the loss function with respect to the encoder parameters is calculated by the backpropagation algorithm. The encoder parameters are updated according to the calculated gradient using the optimization algorithm, and the iterative optimization is repeated until the loss function converges or the preset number of iterations is reached. At this point, the encoder parameters are adjusted to the optimal state, which can effectively constrain the potential representation of the reconstructed image, thereby generating a target reconstructed image with efficient compression and privacy protection at a low bit rate. At this point, the generated target reconstructed image is highly consistent with the original image in semantics, and is close to the retrieval reference image in visual quality. It has a compact potential representation, effectively protects the privacy of the original image, and maintains high performance in various machine vision analysis tasks.

[0180] This embodiment proposes a privacy-preserving image compression method, which obtains a compactness trade-off factor; determines an overall objective function based on the compactness trade-off factor, the entropy loss, the signal preservation loss, and the semantic embedding loss; and constrains the latent representation of the reconstructed image based on the overall objective function to obtain a target reconstructed image. This method addresses the technical problem of ensuring the covert transmission of secret information under lossy compression while flexibly controlling bit rate distortion and transmission performance. Compared to the prior art, this method determines an overall objective function using the compactness trade-off factor, entropy loss, signal preservation loss, and semantic embedding loss, and uses this overall objective function to constrain the latent representation of the reconstructed image, ultimately generating the target reconstructed image. By introducing a compactness trade-off factor to balance the weights between different losses, a comprehensive objective function is constructed to comprehensively optimize the quality of the reconstructed image. The encoder parameters are then adjusted by optimizing this objective function to constrain the latent representation. This method not only improves compression efficiency and reduces redundant information, but also ensures that the reconstructed image is highly similar to the original image in terms of visual quality and semantic consistency. It also effectively protects image privacy and reduces the risk of privacy leakage. The method possesses excellent flexibility and adaptability, meeting the diverse needs of different application scenarios.

[0181] For example, to help understand the implementation process of the privacy protection image compression method obtained by combining this embodiment with the above embodiment 1, please refer to Figure 4 , Figure 4 A brief flowchart of a privacy-preserving image compression method is provided. Specifically:

[0182] Referring to Example 1, an original image is obtained, and a retrieval reference image is determined based on the original image; an image compression quantization latent representation is performed on the retrieval reference image to determine a first quantization feature and a second quantization feature; an entropy loss is modeled on the probability distribution of the second quantization feature using a factor decomposition density model to determine an entropy loss; based on the entropy loss, the online semantic features of the original image are embedded into the first quantization feature to generate a reconstructed image; a signal preservation loss and a semantic embedding loss are calculated based on the original image, the retrieval reference image, and the reconstructed image; the latent representation of the reconstructed image is constrained based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image. Referring to Example 2, a compactness trade-off factor is obtained; an overall objective function is determined based on the compactness trade-off factor, the entropy loss, the signal preservation loss, and the semantic embedding loss; the latent representation of the reconstructed image is constrained based on the overall objective function to obtain a target reconstructed image.

[0183] Original image x o Retrieve the top k samples based on semantic similarity and select the mth sample as the reference image x r . Reconstructed graph x t Use xo The semantics and x r By using the cloud model in x t Perform downstream tasks and effectively analyze x o The semantics of , without exposing its fine-grained pixel-level information. Specifically, given an original image x o , first perform a similarity search in the database to retrieve the top k most similar samples. From these samples, select one that is relatively far away from x o The sample of is used as the retrieved reference image x r . Then use an encoder E(·) to compress x r , obtain a potential representation y=E(x r ). The binary codes of y are then transmitted to the cloud, and the decoder D(·) takes them as input to generate the reconstructed image x t =D(E(x r )), x t It is then used in various downstream tasks T, ensuring that the performance of the reconstructed image in the downstream tasks is very close to that of the original image, that is, T(x o )≈T(x t ) and minimize the bit rate while maintaining human visual perception.

[0184] like Figure 5 As shown, Figure 5This is a schematic diagram of the experimental results of the privacy-preserving image compression method of this application in image classification, image-text retrieval, and image description. This is the experimental result of three machine vision tasks with different granularities: image classification (ACC), image-text retrieval (mAP@50), and image description (Bleu_4, CIDEr, ROUGE_L). Among them, "LIC-TCM (retrieved reference image)" means using the image compression method LIC-TCM as the backbone network and compressing the reference image, and "LIC-TCM (original image)" means using LIC-TCM to compress the original image. According to the experimental results, it can be clearly analyzed that the present application performs well in tasks such as classification, image-text retrieval, and image description, indicating that the semantic information of the original image can be effectively embedded into the reconstructed image, and it is suitable for various tasks. Moreover, even with a lower BPP, that is, a low bit rate, the present application has achieved a strong effect, while "LIC-TCM (retrieved reference image)" requires a higher BPP. This shows that this application improves the privacy protection effect and improves the coding efficiency. At the same time, as the BPP value increases, both "LIC-TCM (original image)" and "LIC-TCM (retrieved reference image)" perform better, confirming that image coding does cause certain information loss, and only sending the retrieved reference image is not enough for machine vision tasks. Figure 6 As shown, Figure 6 This is a schematic diagram of the experimental results of the privacy-preserving image compression method of the present application on human perception tasks. This is the experimental result on human perception (PSNR, MS-SSIM) tasks. The comparison of multiple groups of images can show the human perception performance of LIC-TCM and the present application, as well as the perceptual difference between the original image and the reconstructed image. As shown in the experimental results, the performance of the present application is only slightly worse than that of LIC-TCM, indicating that the present application still maintains competitive perceptual quality for the reference image while protecting the privacy of the image. A significant perceptual gap can be observed between the original image and the reconstructed image, confirming that the present application can effectively prevent the exposure of the original image information.

[0185] It should be noted that the above examples are only used to understand this application and do not constitute a limitation on the privacy-preserving image compression method of this application. More simple transformations based on this technical concept are all within the scope of protection of this application.

[0186] This application also provides a privacy protection image compression device, please refer to Figure 7 , the privacy protection image compression device includes:

[0187] An acquisition module 10 is configured to acquire an original image and determine a retrieval reference image based on the original image;

[0188] The acquisition module 10 is further configured to perform image compression quantization potential representation on the retrieved reference image to determine a first quantization feature and a second quantization feature;

[0189] A processing module 20 is configured to perform entropy loss modeling on the probability distribution of the second quantized feature using a factor decomposition density model to determine the entropy loss;

[0190] The processing module 20 is further configured to embed the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image;

[0191] An execution module 30 is configured to calculate a signal preservation loss and a semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image;

[0192] The execution module 30 is further configured to constrain the potential representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

[0193] The acquisition module 10 is further configured to match the original image with a pre-built vector database to a corresponding target search sample image, and determine a database sample image set;

[0194] Extracting feature vectors of the original image and the database sample image set using an image pre-training model to calculate cosine similarity and determine image similarity;

[0195] Sort the database sample image set according to the image similarity to determine a preceding sample image set;

[0196] A target image is selected based on the preceding sample image set to obtain a retrieval reference image.

[0197] The acquisition module 10 is further configured to extract semantic features of the retrieval reference image and input them into a deep learning model for image compression coding to determine a potential representation;

[0198] Using a super-prior model to capture the spatial dependencies between the corresponding elements of the latent representation, and determining a super-prior representation;

[0199] quantizing the potential representation using a quantization operator to obtain a first quantized feature;

[0200] The super-prior representation is quantized using a quantization operator to obtain a second quantized feature.

[0201] The processing module 20 is further configured to embed the online semantic features of the original image into the first quantized features in the form of variational prediction according to the entropy loss, and determine a joint optimized coding;

[0202] The joint optimized code is decoded and reconstructed to obtain a reconstructed image.

[0203] The execution module 30 is further configured to extract semantic features of the retrieval reference image and the reconstructed image and calculate a mean square error;

[0204] Calculating signal preservation loss based on the distance function and the mean square error;

[0205] Semantic features of the original image and the reconstructed image are extracted to calculate the cosine similarity between the target images and determine the semantic embedding loss.

[0206] The execution module 30 is further configured to obtain a compactness trade-off factor;

[0207] determining an overall objective function based on the compactness tradeoff factor, the entropy loss, the signal preservation loss, and the semantic embedding loss;

[0208] The potential representation of the reconstructed image is constrained based on the overall objective function to obtain a target reconstructed image.

[0209] The execution module 30 is further configured to determine encoder parameters, decoder parameters, and feature extraction network parameters based on the overall objective function;

[0210] The decoder parameters and feature extraction network parameters are fixed, and the encoder parameters are adjusted to constrain the potential representation of the reconstructed image to obtain a target reconstructed image.

[0211] The privacy-preserving image compression device provided in this application, employing the privacy-preserving image compression method of the aforementioned embodiment, can address the technical problem of ensuring the covert transmission of secret information under lossy compression while flexibly controlling bit rate distortion and transmission performance. Compared to the prior art, the privacy-preserving image compression device provided in this application achieves the same beneficial effects as the privacy-preserving image compression method of the aforementioned embodiment. Other technical features of the privacy-preserving image compression device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0212] The present application provides a privacy-preserving image compression device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the privacy-preserving image compression method of the above-mentioned embodiment 1.

[0213] Reference below Figure 8 , which shows a schematic structural diagram of a privacy-preserving image compression device suitable for implementing embodiments of the present application. The privacy-preserving image compression device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The privacy-preserving image compression device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0214] like Figure 8 As shown, the privacy-preserving image compression device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in ROM (Read Only Memory) 1002 or programs loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the privacy-preserving image compression device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication device 1009. Communication device 1009 can allow the privacy-preserving image compression device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a privacy-preserving image compression device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or provided instead.

[0215] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0216] The privacy-preserving image compression device provided in this application, employing the privacy-preserving image compression method of the aforementioned embodiment, can address the technical problem of ensuring the covert transmission of secret information under lossy compression while flexibly controlling bit rate distortion and transmission performance. Compared to the prior art, the privacy-preserving image compression device provided in this application achieves the same beneficial effects as the privacy-preserving image compression method of the aforementioned embodiment. Other technical features of this privacy-preserving image compression device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0217] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0218] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0219] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the privacy-preserving image compression method in the above-mentioned embodiment.

[0220] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0221] The computer-readable storage medium may be included in the privacy-preserving image compression device, or may exist independently without being incorporated into the privacy-preserving image compression device.

[0222] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the privacy-preserving image compression device, the privacy-preserving image compression device: obtains an original image, determines a retrieval reference image based on the original image; performs image compression and quantization latent representation on the retrieval reference image to determine a first quantization feature and a second quantization feature; uses a factor decomposition density model to perform entropy loss modeling on the probability distribution of the second quantization feature to determine the entropy loss; embeds the online semantic features of the original image into the first quantization feature based on the entropy loss to generate a reconstructed image; calculates signal preservation loss and semantic embedding loss based on the original image, the retrieval reference image and the reconstructed image; constrains the latent representation of the reconstructed image based on the entropy loss, the signal preservation loss and the semantic embedding loss to generate a target reconstructed image.

[0223] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0224] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0225] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0226] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned privacy-preserving image compression method. This computer-readable storage medium addresses the technical challenges of ensuring the confidential transmission of secret information under lossy compression while flexibly controlling bit rate distortion and transmission performance. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the privacy-preserving image compression method provided in the aforementioned embodiments and are not further elaborated here.

[0227] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A privacy-preserving image compression method, characterized in that: The method includes: Acquire an original image, and determine a retrieval reference image based on the original image; Performing image compression quantization potential representation on the retrieved reference image to determine a first quantization feature and a second quantization feature; Performing entropy loss modeling on the probability distribution of the second quantized feature using a factor decomposition density model to determine the entropy loss; Embedding the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image; Calculating a signal preservation loss and a semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image; The reconstructed image latent representation is constrained based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

2. The method according to claim 1, wherein The step of determining a retrieval reference image based on the original image comprises: According to the original image and the pre-built vector database, a corresponding target search sample image is matched to determine a database sample image set; Extracting feature vectors of the original image and the database sample image set using an image pre-training model to calculate cosine similarity and determine image similarity; Sort the database sample image set according to the image similarity to determine a preceding sample image set; A target image is selected based on the preceding sample image set to obtain a retrieval reference image.

3. The method according to claim 1, wherein The step of performing image compression quantization potential representation on the retrieved reference image to determine the first quantization feature and the second quantization feature comprises: Extracting semantic features of the retrieval reference image and inputting them into a deep learning model for image compression coding to determine potential representations; Using a super-prior model to capture the spatial dependencies between the corresponding elements of the latent representation, and determining a super-prior representation; quantizing the potential representation using a quantization operator to obtain a first quantized feature; The super-prior representation is quantized using a quantization operator to obtain a second quantized feature.

4. The method according to claim 1, wherein The step of embedding the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image includes: embedding the online semantic features of the original image into the first quantized features in the form of variational prediction according to the entropy loss, and determining a joint optimized coding; The joint optimized code is decoded and reconstructed to obtain a reconstructed image.

5. The method according to claim 1, wherein The step of calculating the signal preservation loss and the semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image comprises: Extracting semantic features of the retrieved reference image and the reconstructed image to calculate signal preservation loss; Semantic features of the original image and the reconstructed image are extracted to calculate the cosine similarity between the target images and determine the semantic embedding loss.

6. The method according to claim 1, wherein The step of constraining the potential representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image comprises: Get the compactness trade-off factor; determining an overall objective function based on the compactness tradeoff factor, the entropy loss, the signal preservation loss, and the semantic embedding loss; The potential representation of the reconstructed image is constrained based on the overall objective function to obtain a target reconstructed image.

7. The method according to claim 6, wherein The step of constraining the potential representation of the reconstructed image based on the overall objective function to obtain a target reconstructed image comprises: Determining entropy model parameters, encoder parameters, decoder parameters, and feature extraction network parameters based on the overall objective function; The entropy model parameters, the decoder parameters, and the feature extraction network parameters are fixed, and the encoder parameters are adjusted to constrain the potential representation of the reconstructed image to obtain a target reconstructed image.

8. A privacy-preserving image compression device, characterized in that: The device comprises: An acquisition module, configured to acquire an original image and determine a retrieval reference image based on the original image; The acquisition module is further configured to perform image compression quantization potential representation on the retrieved reference image to determine a first quantization feature and a second quantization feature; A processing module, configured to perform entropy loss modeling on the probability distribution of the second quantized feature using a factor decomposition density model to determine the entropy loss; The processing module is further configured to embed the online semantic features of the original image into the first quantitative features based on the entropy loss to generate a reconstructed image; An execution module, configured to calculate a signal preservation loss and a semantic embedding loss based on the original image, the retrieved reference image, and the reconstructed image; The execution module is further configured to constrain the potential representation of the reconstructed image based on the entropy loss, the signal preservation loss, and the semantic embedding loss to generate a target reconstructed image.

9. A privacy-preserving image compression device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the privacy-preserving image compression method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the privacy-preserving image compression method according to any one of claims 1 to 7 are implemented.

Images