A privacy protection image feature compression method and system for edge cloud collaborative inference

Abstract

The application discloses a privacy protection image feature compression method and system for edge cloud collaborative reasoning, which is based on a new privacy protection feature compression framework, combines an attention mechanism, and guides the compression of intermediate features through an adversarial collaborative training strategy in a feature domain set. The adversarial collaborative training strategy of feature domain focus introduces selective aggregation of feature domains and feature disturbance suppression, enhances the security and attack resistance of data in the transmission process, and effectively reduces the risk of data privacy being leaked on a cloud honest but curious server. To improve the generalization ability and adaptability of the algorithm, the application conducts system evaluation on multiple image data sets and different deep learning models. The experimental results show that the application is significantly better than existing feature compression methods in intermediate feature transmission efficiency, analysis accuracy and privacy protection level, and provides strong technical support for the application of large-scale deep learning models in resource-constrained edge device environments.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image feature coding, and particularly relates to a privacy protection image feature compression method and system for edge cloud collaborative inference. BACKGROUND

[0002] With the continuous expansion of the capacity of deep neural networks, the difficulty of advanced models in performing image machine vision analysis on devices with limited computing and storage capacity has significantly increased. Traditional deep learning models usually require a large amount of computing resources, which makes it a major challenge to effectively deploy these models on mobile devices or edge devices. In order to solve this problem, collaborative intelligence (CI) systems based on mobile cloud deployment have gradually become an important solution. In CI systems, neural models are divided into front-end and back-end sub-models to realize distributed processing of tasks. The front-end device is responsible for data acquisition and preliminary processing, while the back-end server performs more complex calculations. This architecture can reduce overall application latency and improve system performance. However, the intermediate features extracted by the front-end must be transmitted to the back-end, which often becomes a bottleneck, significantly affecting the accuracy of inference and the processing time of the system.

[0003] In order to solve the problem of intermediate feature transmission, researchers have proposed various feature compression methods, but these methods generally face three major challenges of efficiency, portability and security. Deep learning models usually ignore compression performance when analyzing, resulting in a large amount of redundant information in the extracted features, thereby increasing transmission time and reducing the efficiency of the overall framework. Therefore, it is crucial to explore effective feature compression methods. When designing a feature compression method, ensuring the universality and portability of the module is a basic requirement. Some research introduces learning-based feature compression methods, which use end-to-end optimization to fine-tune the model parameters of the original deep neural network and jointly train the compression module. Although these methods have good compression performance, modifying the parameters of the pre-trained deep neural network (DNN) model in practical applications faces challenges, requiring the participation of all mobile devices and relying on a large number of training samples. In addition, combining the compression module with the modified DNN model also introduces portability restrictions. In terms of privacy protection, it is particularly important to protect personal privacy when users want to perform server data analysis. By transmitting intermediate features rather than raw physical data, user privacy can be protected to some extent. However, these intermediate features can still leak potential information, posing a risk of model inversion attacks. Although there have been studies on differential privacy in mobile cloud applications, these methods also face their own risks and challenges. SUMMARY

[0004] The application provides a privacy protection image feature compression method and system for edge cloud collaborative reasoning, aiming to optimize the feature compression and privacy protection performance in edge cloud collaborative reasoning. The core goal of the method is to achieve efficient transmission of intermediate features while effectively removing privacy sensitive information by constructing a feature compression framework to meet the security needs of collaborative intelligent systems.

[0005] The application is implemented by adopting the following technical solutions:

[0006] A privacy protection image feature compression method for edge cloud collaborative reasoning, comprising the following steps:

[0007] 1) The pre-trained deep learning network model is divided from the middle and the network parameters are frozen, and is respectively deployed at the edge as a front-end feature extraction network and at the cloud as a back-end feature analysis network;

[0008] 2) Based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided, convolution layers are used for feature fusion, and full connection layers are used for compression to realize an encoder network structure; based on the non-local feature domain concentration module and convolution operation, the features are reconstructed to realize a decoder network structure;

[0009] 3) Based on the encoder network and the decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain an image analysis result;

[0010] 4) Based on the front-end feature extraction network and the encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck feature;

[0011] 5) Based on the image analysis result, the analysis task loss is calculated, the compression code rate loss is calculated based on the output of the encoder network, and the reconstructed image loss is calculated based on the inversion attack network, and the directional gradient propagation is performed according to the loss function to optimize the network parameters to obtain different transmission code rate compression models;

[0012] 6) Under the cloud edge collaborative reasoning, different transmission code rate compression models are used to realize privacy protection image feature compression under different image data sets and deep learning networks.

[0013] Further improvement of the application is that in step 1), the pre-trained deep learning network model is divided from the middle and the network parameters are frozen, and is respectively deployed at the edge as a front-end feature extraction network and at the cloud as a back-end feature analysis network, comprising:

[0014] In the image analysis task based on the cloud edge collaborative system, the neural network model is divided into two sub-models and distributed between the front-end device and the back-end server; in the edge cloud collaborative inference system, the DNN is divided into two parts, represented as ; in this structure, is deployed on the edge device, responsible for collecting input images X from the environment and generating intermediate values , which are then transmitted to the cloud server; while is executed on the cloud server, receiving intermediate features and calculating the final output ; however, in the mobile cloud deployment of the cloud edge collaborative intelligent system, the intermediate features extracted by the deep network are transmitted from the front-end to the back-end.

[0015] The further improvement of the application is that in step 2), based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided, feature fusion is performed using a convolution layer, compression is performed using a fully connected layer, and an encoder network structure is realized; based on the non-local feature domain concentration module and the convolution operation, the features are reconstructed, and a decoder network structure is realized, including:

[0016] In order to remove the privacy sensitive information in the intermediate features and improve the compression efficiency, first, the reason why the intermediate features are compressible and their privacy is easily leaked is analyzed from the perspective of information theory; in the deep learning model, the entropy H(T) of the intermediate feature T is represented as:

[0017]

[0018] where represents the probability distribution of the feature ; by optimizing the encoding method of the feature, the entropy value can be reduced, thereby improving the compression efficiency; according to information theory, the goal of privacy protection is to minimize the mutual information between the compressed data and the original input , by designing a feature compression strategy, to reduce while retaining information related to the analysis task ; the calculation of mutual information is represented as:

[0019]

[0020] where, is the entropy of the input data, is the uncertainty of the input data given the bottleneck feature.

[0021] The further improvement of the present application is that in step 3), the features extracted by the front-end feature extraction network are input into the encoder for compression based on the encoder network and the decoder network, and the compressed features are input into the back-end feature analysis network for analysis after being decoded by the decoder to obtain image analysis results, including:

[0022] The SET-NET framework includes an encoder and a decoder; in the implementation of the encoder, a feature domain concentration module is designed, which combines a non-local attention mechanism with a residual network; the module generates an implicit importance mask to guide the adaptive compression of latent features; to further reduce communication overhead, an entropy coding technique is used to eliminate statistical redundancy in compressed data; the compressed data is transmitted wirelessly and uploaded to a cloud server; after receiving the data, the DNN model is decompressed, and the subsequent processing steps generate the final analysis results.

[0023] The further improvement of the present application is that in step 4), based on the front-end feature extraction network and the encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck features, including:

[0024] An auxiliary DNN model called reconstruction network RecNet is constructed, and the goal is to reconstruct the input image from the bottleneck features; RecNet is a mirror network of the front-end network and the encoder; the encoder and RecNet are trained in an adversarial manner; RecNet tries to recover the input image from the latent features, while the encoder tries to destroy the performance of RecNet by manipulating the generated latent features; at the same time, both the encoder and the decoder try to maintain the accuracy of machine vision analysis.

[0025] The further improvement of the present application is that in step 5), based on the image analysis results, the analysis task loss is calculated, the compression code rate loss is calculated based on the output of the encoder network, and the reconstruction image loss is calculated based on the inversion attack network, and the directional gradient propagation is performed according to the loss function to optimize the network parameters, and different transmission code rate compression models are obtained, including:

[0026] The goal of combining machine analysis tasks is to maximize the mutual information between intermediate features and true labels, and the basic optimization goal of privacy protection is to concentrate the feature space on the analysis task while suppressing as much irrelevant information as possible, and the loss function is established as follows:

[0027]

[0028] where, is the compression weight parameter, which can be adjusted to obtain different bit rate models; the parameter represents the safety factor, and by changing different privacy protection strength models can be obtained; and respectively represent the intermediate features between the input image and the real label ; the mutual information between the reconstructed image and the input; the mutual information reflects the similarity between the reconstructed image and the input, and is calculated by the reconstruction loss is specifically represented, and is defined as follows:

[0029]

[0030] wherein, is the reconstructed input, and are horizontal and vertical Sobel filters respectively, * represents convolution operation, and n is the total number of tensor elements in a batch; the Sobel filter is incorporated into the reconstruction loss to emphasize edge information; represents a gradient inversion operation, which is a custom calculation layer in the deep network, and the operation rule is defined as follows:

[0031]

[0032] .

[0033] Further improvements of the present application are that in step 5), different image data sets and privacy-protected image feature compression under deep learning network are realized by using different transmission code rate compression models under cloud edge collaborative reasoning, including:

[0034] The SET-NET architecture can be trained in an end-to-end manner; a step-by-step training method is adopted, and the training process of each batch of data is summarized as the following steps: in the first step, the encoder network is initialized and trained, the input X passes through the front-end network, the encoder and the decoder, and the back-end network calculates and analyzes the precision loss and the code rate loss; in the second step, the RecNet network is initialized and trained, the encoder network is frozen, and the reconstruction loss between the reconstructed image and the input image is calculated to obtain an attackable RecNet; in the third step, the encoder network and the RecNet are trained in an antagonistic manner, the RecNet tries to recover the input image from the latent features as much as possible under the condition of ensuring the analysis precision, and the encoder simultaneously tries to destroy the performance of the RecNet by manipulating the generated latent features, so as to achieve the purpose of privacy protection.

[0035] The further improvement of the present application further comprises: performance analysis and verification of the efficient privacy protection image feature compression algorithm for cloud edge collaborative inference are performed using different image data sets and deep learning networks; the performance of the proposed method in multiple image analysis tasks is evaluated, including image classification using ResNet34 on CIFAR-10, classification using ResNet50 on ImageNet, and multi-label classification using ResNet34 on CelebA; during the performance analysis, the channel dimension c in SET-NET and the RecNet network structure during training are adjusted for different DNN models; the trade-off between accuracy and reconstructed image quality is analyzed by using Privacy-Preserving Characteristics evaluation.

[0036] A privacy protection image feature compression system for edge cloud collaborative inference, comprising:

[0037] A network parameter acquisition module, which divides and freezes the network parameters of a pre-trained deep learning network model, and respectively deploys the network parameters as a front-end feature extraction network at the edge and a back-end feature analysis network at the cloud;

[0038] A network structure confirmation module, which adopts a non-local feature domain concentration module and provides an attention mechanism based on the front-end feature extraction network and the back-end feature analysis network, uses a convolution layer for feature fusion, uses a fully connected layer for compression, and realizes an encoder network structure; and reconstructs features based on the non-local feature domain concentration module and convolution operation, and realizes a decoder network structure;

[0039] An image analysis module, which inputs the features extracted by the front-end feature extraction network into the encoder for compression, inputs the compressed features into the back-end feature analysis network for analysis after decoding by the decoder, and obtains image analysis results based on the encoder network and the decoder network;

[0040] An image reconstruction module, which constructs an inversion attack network to reconstruct input images from bottleneck features based on the front-end feature extraction network and the encoder network structure;

[0041] A compression model construction module, which calculates an analysis task loss based on the image analysis results, calculates a compression code rate loss based on the output of the encoder network, and calculates a reconstructed image loss based on the inversion attack network, and propagates the direction gradient according to the loss function to optimize the network parameters and obtain different transmission code rate compression models;

[0042] An image feature compression module, which realizes privacy protection image feature compression under different transmission code rate compression models for different image data sets and deep learning networks under cloud edge collaborative inference.

[0043] A computer-readable storage medium stores a computer program which, when executed by a processor, implements the steps of the method for privacy-preserving image feature compression for edge-cloud collaborative inference.

[0044] The present application has at least the following beneficial technical effects:

[0045] The present application provides a method and system for privacy-preserving image feature compression for edge-cloud collaborative inference. First, a pre-trained high-performance image analysis deep learning network is divided into a front-end network and a back-end network, which are deployed on the edge and cloud respectively, and the network parameters are frozen. Second, the intermediate features output by the front-end network are input into the SET-NET network designed by the present application. Through convolution layers, residual blocks and non-local feature domain concentration modules, feature fusion is performed. The feature map is compressed by a fully connected layer, and the feature vector is rate-converted by entropy encoding. The RecNet is used to simulate the attacker to reconstruct the image, calculate the loss between the reconstructed image and the input image, and conduct adversarial collaborative training on the SET-NET to guide the feature domain concentration, remove the privacy information while retaining the relevant information for the analysis task to the greatest extent. The analysis task loss, compression rate loss and reconstructed image loss are calculated respectively, and the directional gradient propagation is performed according to the loss function to optimize the network parameters. Finally, different image datasets and deep learning networks are used to analyze and verify the performance of the efficient privacy-preserving image feature compression algorithm for cloud-edge collaborative inference.

[0046] Further, the SET-NET framework proposed by the present application realizes double optimization in privacy protection and feature compression. The framework is flexible in design and can adapt to various deep learning backbone networks without the need for modification or parameter fine-tuning of the network structure. This adaptability greatly improves the flexibility of the system, enabling it to be quickly applied to different image analysis tasks, thereby enhancing the user experience in various scenarios.

[0047] Further, SET-NET realizes high compression ratio of intermediate features by optimizing the portable module after feature extraction. This capability significantly reduces the required storage space and data transmission bandwidth, while effectively protecting local privacy information. Running complex deep learning models on edge devices becomes more feasible, providing a solution for resource-constrained environments and promoting the widespread application of intelligent devices.

[0048] Further, the present application adopts an adversarial training strategy to reduce edge details and privacy-sensitive information in the input image by introducing an attack network. This method not only reduces the risk of model inversion attacks, but also ensures the accuracy of machine vision tasks. At the same time, the design of the feature-aware concentration compression network strengthens the task-related features, further improving the performance of the model in practical applications.

[0049] In summary, the SET-NET framework of the present application provides an efficient and secure solution for cloud-edge collaborative inference tasks by innovatively combining privacy protection and feature compression techniques. This framework not only demonstrates excellent adaptability in different deep learning models and application scenarios, but also effectively prevents the leakage of private information through the adversarial training mechanism. At the same time, the introduction of the feature-aware centralized compression network enables the system to extract and analyze information related to the task, thereby improving overall performance. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 Figure 1 is a general structure diagram of a privacy protection image feature compression method for edge-cloud collaborative inference according to the present application;

[0051] Figure 2 Figure 2 is a schematic diagram of removing redundancy and privacy protection targets according to the present application;

[0052] Figure 3 Figure 3 is a basic structure diagram of an encoder network and a decoder network according to the present application;

[0053] Figure 4 Figure 4 is a structure block diagram of a privacy protection image feature compression system for edge-cloud collaborative inference according to the present application. DETAILED DESCRIPTION

[0054] Exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0055] Embodiment 1

[0056] The privacy protection image feature compression method for edge-cloud collaborative inference provided by the present application comprises the following steps:

[0057] 1) The pre-trained deep learning network model is divided and the network parameters are frozen in the middle, and is respectively deployed at the edge as a front-end feature extraction network and at the cloud as a back-end feature analysis network;

[0058] 2) Based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided, convolution layers are used for feature fusion, and full connection layers are used for compression to realize the encoder network structure; based on the non-local feature domain concentration module and the convolution operation to reconstruct the features, the decoder network structure is realized;

[0059] 3) Based on the encoder network and the decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain the image analysis result;

[0060] 4) Based on the front-end feature extraction network and the encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck feature;

[0061] 5) Based on the image analysis result, the analysis task loss is calculated, the compression code rate loss is calculated based on the output of the encoder network, and the reconstructed image loss is calculated based on the inversion attack network, and the directional gradient propagation is performed according to the loss function, the network parameters are optimized, and different transmission code rate compression models are obtained;

[0062] 6) Under the cloud edge collaborative reasoning, different transmission code rate compression models are used to realize the privacy protection image feature compression under different image data sets and deep learning networks.

[0063] Embodiment 2

[0064] As Figure 1 shown, the application provides a privacy protection image feature compression method for edge cloud collaborative reasoning, which includes the following parts:

[0065] 1) In the edge cloud collaborative reasoning system, the DNN is divided into two parts, represented as In this structure, is deployed on the edge device, responsible for collecting input images X from the environment and generating intermediate values , which are then transmitted to the cloud server; while is executed on the cloud server, receiving intermediate features and calculating the final output

[0066] 2) The main components of the SET-NET framework include an encoder and a decoder. The design of the framework allows it to be transplanted without modifying the architecture and parameters of a given deep neural network (DNN) model, and it is adaptable and can be replaced with a general DNN model at a specified segmentation point. In the implementation of the encoder, a feature domain concentration module is designed, which combines a non-local attention mechanism with a residual network. This module generates an implicit importance mask to guide the adaptive compression of latent features. To further reduce communication overhead, an entropy coding technique is used to eliminate statistical redundancy in compressed data. The compressed data is transmitted wirelessly and uploaded to a cloud server. After receiving the data, the DNN model decompresses it, and the subsequent processing steps generate the final analysis results.

[0067] 3) A auxiliary DNN model called reconstruction network (RecNet) is constructed, which aims to reconstruct the input image from the bottleneck features. RecNet is a mirror network of the front-end network and the encoder. We train the encoder and RecNet in an adversarial manner, with RecNet trying to recover the input image from the latent features as much as possible, while the encoder tries to sabotage the performance of RecNet by manipulating the generated latent features. At the same time, both the encoder and the decoder try to maintain high machine vision analysis accuracy.

[0068] As Figure 2 shown, in order to remove privacy sensitive information in intermediate features and improve compression efficiency, the present application first analyzes the compressibility of intermediate features and the reason why privacy is easily leaked from the perspective of information theory. From the perspective of information theory, the compressibility of intermediate features is due to the redundant information they contain. In deep learning models, the representation of features often has a high degree of correlation, and some features exhibit similarity between multiple samples. According to Shannon's information theory, this redundancy allows us to reduce the size of the data representation through encoding methods. For example, using entropy coding techniques, statistical redundancy can be effectively removed, significantly reducing the storage space and transmission bandwidth required for intermediate features. Specifically, the entropy H(T) of the intermediate feature T can be represented as:

[0069]

[0070] where represents the probability distribution of feature . By optimizing the encoding method of the feature, the entropy value can be reduced, thereby improving the compression efficiency. The privacy vulnerability of intermediate features mainly lies in the fact that these features may still contain sensitive information related to the input data. During the transmission of intermediate features, an attacker can recover the original input data from them using model inversion techniques, leading to user privacy leakage. According to information theory, the goal of privacy protection is to minimize the mutual information between compressed data and the original input The mutual information reflects the degree of association between the input data and the intermediate features , and the higher mutual information means stronger privacy risk. In order to effectively remove the privacy-sensitive information, it is necessary to reduce by designing a reasonable feature compression strategy, while retaining the information related to the analysis task. The calculation of mutual information can be expressed as:

[0071]

[0072] wherein, is the entropy of the input data, is the uncertainty of the input data given the bottleneck feature. By minimizing , the present application realizes effective protection of privacy information. The present application lays a theoretical foundation for efficient feature compression and privacy protection by analyzing the compressibility and privacy vulnerability of the intermediate features. This analysis will guide the subsequent feature compression strategy design and adversarial training process, ensuring the effectiveness and security of SET-NET in practical applications.

[0073] As Figure 3As shown, the structure of the encoder network aims to efficiently extract and compress the feature information of the input image. First, the network performs preliminary feature extraction on the input image through a convolutional layer with a size of 5x5. The convolutional operation can capture local features and enhance image details. Subsequently, the output of the convolutional layer is normalized by a generalized division normalization (GDN) layer to improve the expression ability of nonlinear features, enabling the model to adapt to more complex feature distributions. Next, the features are further processed through multiple residual blocks (ResBlock), each consisting of two 3x3 convolutional layers and a ReLU activation function. The introduction of residual blocks aims to learn residual mappings, ensuring effective information propagation between layers and promoting network convergence. After completing the preliminary feature extraction, the network introduces a non-local domain aggregation module (NLM). This module focuses on feature regions related to specific analysis tasks by generating an importance mask. This design improves information efficiency, enabling the network to optimize feature representation during processing and thus improve compression performance. Next, the output after feature extraction and processing is quantized (Q) to convert continuous feature values into discrete values for storage and transmission. The quantized features are then compressed using entropy encoding techniques to eliminate statistical redundancy, generating an efficient bitstream for transmission to the decoder. In the decoder, the bitstream is first restored to the quantized features through an entropy decoder, and then the continuous features are recovered through the inverse quantization process. This process ensures the integrity of the compressed information, enabling the decoder to effectively reconstruct the image. The recovered features are then processed through an inverse non-local module to recapture global and local information, enhancing the completeness and effectiveness of the features. Next, the features are again processed through residual blocks to optimize feature representation. Finally, the features are normalized through an inverse generalized division normalization (IGDN) layer and mapped back to the original image dimension through a convolutional layer to generate the final decoded image. The entire SET-NET architecture effectively integrates the encoder and decoder, not only achieving efficient image feature compression and transmission but also ensuring effective protection of privacy information. The design of this network provides an innovative solution for cloud-edge collaborative reasoning, adapting to the dual demands of high performance and high security in modern applications.

[0074] The goal of combining machine analysis tasks is to maximize the mutual information between intermediate features and true labels. The basic optimization goal of privacy protection is to concentrate the feature space on the analysis task while suppressing as much irrelevant information as possible. The loss function is established as follows:

[0075] where is the compression weight parameter, which can be adjusted to obtain models with different bit rates. The parameter represents the security coefficient, and by changing different models with different privacy protection strengths can be obtained. and respectively represent mutual information between intermediate features and input image and and true label For multi-dimensional tensor data, the exact value of mutual information is usually difficult to calculate. Therefore, in the implementation of the present application, by the objective function tailored for machine vision tasks, such as cross-entropy loss used in classification tasks. Mutual information reflects the similarity between the reconstructed image and the input, which can be calculated by the reconstruction loss is specifically represented, defined as follows:

[0076]

[0077] wherein, is the reconstructed input, and are horizontal and vertical Sobel filters, respectively, * represents convolution operation, and n is the total number of tensor elements in the batch. Empirically, the value of is set to 0.5. Since privacy-sensitive information is usually related to image edges, the Sobel filter is incorporated into the reconstruction loss to emphasize edge information. This method enhances the ability of RecNet to reconstruct edges, thereby forcing the encoder network to remove edge information from the bottleneck features. represents a gradient reversal operation, which is understood as a custom calculation layer in a deep network rather than a standard mathematical operation. Its operation rules are defined as follows:

[0078]

[0079]

[0080] The present application adopts a step-by-step training method, and the training process of each batch of data can be summarized as follows: the first step is to initialize the training of the encoder network, the input X passes through the front-end network, the encoder and the decoder, and the back-end network calculates and analyzes the precision loss and the code rate loss; the second step is to initialize the training of the RecNet network, freeze the encoder network, calculate the reconstruction loss between the reconstructed image and the input image, and obtain an attack-resistant RecNet; the third step is to perform adversarial collaborative training on the encoder network and the RecNet, and under the condition of ensuring the analysis accuracy, the RecNet tries to recover the input image from the latent features as much as possible, while the encoder simultaneously tries to destroy the performance of the RecNet by manipulating the generated latent features, thereby achieving the purpose of privacy protection.

[0081] Example 3

[0082] Experimental setup and analysis of experimental results are given below.

[0083] Experimental setup. The present invention is used for image classification tasks on CIFAR-10 using ResNet34, on ImageNet using ResNet50, and on CelebA using ResNet34 for multi-label classification. All implementations are based on the PyTorch framework, and the DNN model is split at the end of the second residual block. To fairly evaluate the effectiveness of the present invention, we employ distributed data parallelism with 8 NVIDIA GeForce RTX 4090 GPUs for training, each equipped with 24 GB of RAM. First, we comprehensively evaluate the method from the perspectives of compression performance and privacy protection. The encoder network is initially trained using the stochastic gradient descent (SGD) optimizer with momentum, with a momentum setting of 0.9 and a batch size of 128. In this stage, the weight decay is set to 0.0001, the learning rate is initialized to 0.1, and cosine decay scheduling is used for adjustment over 300 epochs. After the preliminary training of the encoder network, its parameters are frozen, and then the inversion network is initialized and trained based on the loss function Finally, the encoder and inversion network are trained in an adversarial and collaborative manner, with privacy weights p set to 0.01, 0.005, and 0.001 to generate models with different levels of privacy protection. By adjusting the compression weight, different compression rates are obtained. The weight values used include {2.0, 1.0, 0.5, 0.3, 0.1, 0.075, 0.05, 0.025, 0.01} to achieve the desired compression effect.

[0084] In the experimental results, the comparative algorithms are traditional image encoding methods including JPEG, JPEG2000, BPG, and VVC. End-to-end image compression methods include bmshj2018_factorized, bmshj2018_hyperprior, and cheng2020_anchor. These image compression methods are evaluated in the same application scenario. These image compression methods are evaluated in cloud-assisted machine vision analysis tasks, where images are first encoded at the front end, and the compressed bitstream is then transmitted to the back end for image reconstruction. The reconstructed lossy images are analyzed using pre-trained inference models, and the performance of each model is evaluated by setting different QP values. In addition, we also compare the performance of the supervised feature compression algorithm Sufc based on the official code. Given the scarcity of algorithms that consider both privacy protection and feature compression, we also evaluate the latest privacy protection feature compression algorithm privacyFC, which is based on the HEVC video encoder. All comparative algorithms are tested under the same experimental conditions.

[0085] The compression capability is first evaluated, especially at low compression bitrates, to demonstrate its effectiveness in a co-intelligence (CI) system. The compression ratio is measured in bits per pixel (bpp) by dividing the total output bitstream by the total number of pixels in the input image, which provides an indicator of the compression efficiency of the feature compression algorithm. Next, the privacy protection performance is analyzed, showing the effectiveness of the framework in protecting the reconstructed image and label information. To simulate the image reconstruction quality and image attributes that an actual attacker can obtain, and to avoid achieving privacy protection by only inverting the network parameters of the co-trained encoder, the encoder network parameters are frozen during the testing phase. At the same time, the inversion network and attribute attack network are reinitialized and retrained. In addition, the trade-off between accuracy and reconstructed image quality is analyzed through the privacy protection characteristic (PPC) analysis, which calculates the product of the final machine analysis accuracy and the distortion of the reconstructed image, so that a higher PPC value indicates better privacy protection performance and minimal loss of analysis accuracy. Table 1 shows the comparison results of image classification accuracy and reconstructed image quality of different compression methods on the ImageNet dataset at different code rates. The classification accuracy of our method at the same code rate is consistent with other comparison methods, the reconstructed image quality is lower than other comparison methods, and the PPC value is higher than other comparison algorithms. Therefore, the present application performs well in terms of compression efficiency and comprehensive performance of privacy protection.

[0086] Table 1. Index results of different compression methods on the ImageNet dataset

[0087]

[0088] Embodiment 4

[0089] As Figure 4 shown, the present application provides a privacy protection image feature compression system for edge cloud collaborative inference, which comprises:

[0090] A network parameter acquisition module, which divides and freezes the network parameters of a pre-trained deep learning network model from the middle, and respectively deploys them as a front-end feature extraction network at the edge and a back-end feature analysis network in the cloud;

[0091] A network structure confirmation module, which adopts a non-local feature domain concentration module and provides an attention mechanism based on the front-end feature extraction network and the back-end feature analysis network, uses a convolution layer for feature fusion, uses a fully connected layer for compression, and realizes an encoder network structure; and reconstructs features based on the non-local feature domain concentration module and convolution operation, and realizes a decoder network structure;

[0092] An image analysis module, based on the encoder network and the decoder network, inputs the features extracted by the front-end feature extraction network into the encoder for compression, and inputs the compressed features decoded by the decoder into the back-end feature analysis network for analysis to obtain an image analysis result;

[0093] An image reconstruction module, based on the front-end feature extraction network and the encoder network structure, constructs an inversion attack network to reconstruct the input image from the bottleneck features;

[0094] A compression model construction module calculates an analysis task loss based on the image analysis result, calculates a compression code rate loss based on the output of the encoder network, and calculates a reconstructed image loss based on the inversion attack network, and performs directional gradient propagation according to a loss function to optimize network parameters to obtain different transmission code rate compression models;

[0095] An image feature compression module realizes private image feature compression under different transmission code rate compression models for different image data sets and deep learning networks in cloud-edge collaborative inference.

[0096] Embodiment 5

[0097] The application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program realizes the steps of the private image feature compression method for cloud-edge collaborative inference when executed by a processor.

[0098] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system, or a computer program product. Therefore, the application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0099] The application is described with reference to flowcharts and / or block diagrams according to the methods, systems and computer program products of the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The system specified in one flow or multiple flows and / or blocks Figure 1 The system specified in one flow or multiple flows and / or blocks

[0100] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0102] While the present application has been described in detail with respect to particular embodiments thereof, it will be apparent to those skilled in the art that various modifications or alterations can be made without departing from the spirit and scope of the present application. Accordingly, it is not intended that the application be limited, except as by the appended claims.

Claims

1. A privacy-preserving image feature compression method for edge-cloud collaborative inference, characterized in that, Includes the following steps: 1) Divide the pre-trained deep learning network model in the middle and freeze the network parameters, and deploy them on the edge as the front-end feature extraction network and in the cloud as the back-end feature analysis network. 2) Based on the aforementioned front-end feature extraction network and back-end feature analysis network, a non-local feature domain aggregation module is adopted and an attention mechanism is provided. Convolutional layers are used for feature fusion, and fully connected layers are used for compression to realize the encoder network structure. Based on the non-local feature domain aggregation module and convolutional operations, features are reconstructed to realize the decoder network structure. 3) Based on the encoder network and decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression. The compressed features are then decoded by the decoder and input into the back-end feature analysis network for analysis to obtain image analysis results, including: The SET-NET framework includes an encoder and a decoder. In the encoder implementation, a feature domain scalar module is designed, which combines a non-local attention mechanism with a residual network. This module generates a hidden importance mask to guide the adaptive compression of latent features. To further reduce communication overhead, entropy coding is used to eliminate statistical redundancy in the compressed data. The compressed data is transmitted wirelessly and uploaded to a cloud server. After receiving the data, the DNN model decompresses it, and subsequent processing steps generate the final analysis results. 4) Based on the aforementioned front-end feature extraction network and encoder network structure, an inversion attack network is constructed to reconstruct the input image from bottleneck features, including: An auxiliary DNN model called RecNet is constructed, with the goal of reconstructing the input image from bottleneck features. RecNet is a mirror network of the front-end network and the encoder. The encoder and RecNet are trained adversarially, with RecNet attempting to recover the input image from latent features, while the encoder simultaneously attempts to compromise RecNet's performance by manipulating the generated latent features. At the same time, both the encoder and decoder try to maintain the accuracy of machine vision analysis. 5) Calculate the analysis task loss based on the image analysis results, calculate the compression bit rate loss based on the encoder network output and calculate the reconstructed image loss based on the inversion attack network, and perform directional gradient propagation according to the loss function to optimize the network parameters and obtain compression models with different transmission bit rates. 6) Under cloud-edge collaborative inference, use compression models with different transmission bit rates to achieve privacy-preserving image feature compression for different image datasets and deep learning networks.

2. The privacy-preserving image feature compression method for edge-cloud collaborative inference according to claim 1, characterized in that, In step 1), the pre-trained deep learning network model is divided and its parameters are frozen, and then deployed at the edge as a front-end feature extraction network and in the cloud as a back-end feature analysis network, including: In image analysis tasks based on cloud-edge collaborative systems, the neural network model is divided into two sub-models, distributed between the front-end device and the back-end server; in edge-cloud collaborative inference systems, the DNN is divided into two parts, represented as follows: In this structure, Deployed on edge devices, it is responsible for collecting input image X from the environment and generating intermediate values. The value is then transmitted to the cloud server; and It then executes on the cloud server, receiving intermediate features. And calculate the final output. However, in cloud-edge collaborative intelligent systems deployed in mobile cloud environments, the intermediate features extracted by deep networks are transmitted from the front end to the back end.

3. The privacy-preserving image feature compression method for edge-cloud collaborative inference according to claim 2, characterized in that, In step 2), based on the aforementioned front-end feature extraction network and back-end feature analysis network, a non-local feature domain aggregation module is adopted and an attention mechanism is provided. Convolutional layers are used for feature fusion, and fully connected layers are used for compression to realize the encoder network structure. Based on the non-local feature domain aggregation module and convolutional operations, features are reconstructed to realize the decoder network structure, including: To remove privacy-sensitive information from intermediate features and improve compression efficiency, we first analyze the compressibility of intermediate features and the reasons for their vulnerability to privacy leakage from an information theory perspective. In the deep learning model, the entropy H(T) of the intermediate feature T is expressed as: in Representation of features The probability distribution; by optimizing the feature encoding method, the entropy value can be reduced, thereby improving compression efficiency; according to information theory, the goal of privacy protection is to minimize the mutual information between compressed data and the original input. By designing feature compression strategies, the reduction At the same time, retain information relevant to the analysis task. Mutual information is calculated as follows: in, The entropy of the input data, This refers to the uncertainty of input data under given bottleneck characteristics.

4. The privacy-preserving image feature compression method for edge-cloud collaborative inference according to claim 1, characterized in that, The SET-NET architecture described above can be trained in an end-to-end manner. It employs a step-by-step training method, and the training process for each batch of data can be summarized as follows: First, the encoder network is initialized and trained. The input X passes through the front-end network, encoder, and decoder, and the back-end network calculates and analyzes the accuracy loss and bitrate loss. Second, the RecNet network is initialized and trained. The encoder network is frozen, and the reconstruction loss between the reconstructed image and the input image is calculated. The first step is to obtain an offensive RecNet. The third step is to perform adversarial co-training on the encoder network and RecNet. While ensuring the accuracy of the analysis, RecNet tries to recover the input image from the latent features as much as possible, while the encoder tries to destroy the performance of RecNet by manipulating the generated latent features, thereby achieving the purpose of privacy protection.

5. A privacy-preserving image feature compression method for edge-cloud collaborative inference according to claim 4, characterized in that, Also includes: Performance analysis and verification of an efficient privacy-preserving image feature compression algorithm for cloud-edge collaborative inference were conducted using different image datasets and deep learning networks. The proposed method was systematically evaluated on multiple image analysis tasks, including image classification using ResNet34 on CIFAR-10, classification using ResNet50 on ImageNet, and multi-label classification using ResNet34 on CelebA. In the performance analysis, the channel dimension c in SET-NET and the RecNet network structure during training were adjusted for different DNN models. The trade-off between analytical accuracy and reconstructed image quality was evaluated using Privacy-Preserving Characteristics.

6. A privacy-preserving image feature compression system for edge-cloud collaborative inference, characterized in that, include: The network parameter acquisition module divides the pre-trained deep learning network model from the middle and freezes the network parameters, which are then deployed at the edge as the front-end feature extraction network and in the cloud as the back-end feature analysis network. The network structure confirmation module, based on the aforementioned front-end feature extraction network and back-end feature analysis network, employs a non-local feature domain concentration module and provides an attention mechanism, utilizes convolutional layers for feature fusion, and uses fully connected layers for compression to realize the encoder network structure; and reconstructs features based on the non-local feature domain concentration module and convolutional operations to realize the decoder network structure. The image analysis module, based on the encoder and decoder networks, inputs the features extracted by the front-end feature extraction network into the encoder for compression. The compressed features are then decoded by the decoder and input into the back-end feature analysis network for analysis to obtain image analysis results, including: The SET-NET framework includes an encoder and a decoder. In the encoder implementation, a feature domain scalar module is designed, which combines a non-local attention mechanism with a residual network. This module generates a hidden importance mask to guide the adaptive compression of latent features. To further reduce communication overhead, entropy coding is used to eliminate statistical redundancy in the compressed data. The compressed data is transmitted wirelessly and uploaded to a cloud server. After receiving the data, the DNN model decompresses it, and subsequent processing steps generate the final analysis results. The image reconstruction module, based on the aforementioned front-end feature extraction network and encoder network structure, constructs an inversion attack network to reconstruct the input image from bottleneck features, including: An auxiliary DNN model called RecNet is constructed, with the goal of reconstructing the input image from bottleneck features. RecNet is a mirror network of the front-end network and the encoder. The encoder and RecNet are trained adversarially, with RecNet attempting to recover the input image from latent features, while the encoder simultaneously attempts to compromise RecNet's performance by manipulating the generated latent features. At the same time, both the encoder and decoder try to maintain the accuracy of machine vision analysis. The compression model construction module calculates the analysis task loss based on the image analysis results, calculates the compression bit rate loss based on the encoder network output, and calculates the reconstructed image loss based on the inversion attack network. It also performs directional gradient propagation based on the loss function to optimize the network parameters and obtain compression models with different transmission bit rates. The image feature compression module uses different transmission bitrate compression models to achieve privacy-preserving image feature compression for different image datasets and deep learning networks under cloud-edge collaborative inference.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the privacy-preserving image feature compression method for edge-cloud collaborative inference according to any one of claims 1-5.

Images