Privacy protection image feature compression method and system for edge cloud collaborative reasoning

Abstract

The invention discloses a privacy protection image feature compression method and a privacy protection image feature compression system for edge cloud collaborative reasoning, which are based on a novel privacy protection feature compression framework, are combined with an attention mechanism, and guide the compression of intermediate features through an adversarial collaborative training strategy in a feature domain set. Selective aggregation and feature disturbance suppression of a feature domain are introduced into a feature domain focused confrontation cooperative training strategy, the security and anti-attack ability of data in the transmission process are enhanced, and the risk that data privacy is leaked on a cloud honest but curious server is effectively reduced. In order to improve the generalization ability and adaptability of the algorithm, system evaluation is carried out on a plurality of image data sets and different deep learning models. Experimental results show that the method is remarkably superior to an existing feature compression method in the aspects of intermediate feature transmission efficiency, analysis precision and privacy protection level, and powerful technical support is provided for application of a large-scale deep learning model in a resource-limited edge device environment.

Description

Technical Field

[0001] The present invention belongs to the field of image feature coding, and in particular relates to a privacy-preserving image feature compression method and system for edge-cloud collaborative reasoning. Background Art

[0002] As the capacity of deep neural networks continues to expand, the difficulty of advanced models for image machine vision analysis on devices with limited computing and storage capabilities has increased significantly. Traditional deep learning models usually require a lot of computing resources, which makes it a major challenge to effectively deploy these models on mobile devices or edge devices. To address this problem, collaborative intelligence (CI) systems based on mobile cloud deployment have gradually become an important solution. In CI systems, neural models are divided into front-end and back-end sub-models to achieve distributed processing of tasks. The front-end device is responsible for data acquisition and preliminary processing, while the back-end server performs more complex calculations. This architecture can reduce overall application latency and improve system performance. However, the intermediate features extracted by the front-end must be transmitted to the back-end, which often becomes a bottleneck, significantly affecting the accuracy of reasoning and the processing time of the system.

[0003] In order to solve the problem of intermediate feature transmission, researchers have proposed a variety of feature compression methods, but these methods generally face three major challenges: efficiency, portability, and security. Deep learning models usually ignore compression performance during analysis, resulting in a large amount of redundant information in the extracted features, which increases the transmission time and reduces the efficiency of the overall framework. Therefore, it is crucial to explore effective feature compression methods. When designing feature compression methods, ensuring the universality and portability of the module is a basic requirement. Some studies have introduced learning-based feature compression methods, using end-to-end optimization to fine-tune the model parameters of the original deep neural network and jointly train the compression module. Although these methods have good compression performance, in practical applications, modifying the parameters of the pre-trained deep neural network (DNN) model is challenging, requiring the participation of all mobile devices and relying on a large number of training samples. In addition, combining the compression module with the modified DNN model also introduces portability limitations. In terms of privacy protection, it is particularly important to protect personal privacy when users want to perform server data analysis. By transmitting intermediate features instead of original physical data, user privacy can be protected to a certain extent. However, these intermediate features may still leak potential information and there is a risk of model inversion attacks. Although previous studies have explored differential privacy in mobile cloud applications, these approaches also face their own risks and challenges. Summary of the invention

[0004] This paper provides a privacy-preserving image feature compression method and system for edge-cloud collaborative reasoning, aiming to optimize the feature compression and privacy protection performance in edge-cloud collaborative reasoning. The core goal of this method is to achieve efficient transmission of intermediate features by building a feature compression framework, while effectively removing privacy-sensitive information to meet the security requirements of collaborative intelligent systems.

[0005] The present invention is achieved by adopting the following technical solutions: A privacy-preserving image feature compression method for edge-cloud collaborative reasoning includes the following steps: 1) Split the pre-trained deep learning network model in the middle and freeze the network parameters, and deploy them on the edge as the front-end feature extraction network and on the cloud as the back-end feature analysis network; 2) Based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided. The convolution layer is used for feature fusion and the fully connected layer is used for compression to realize the encoder network structure. The features are reconstructed based on the non-local feature domain concentration module and the convolution operation to realize the decoder network structure. 3) Based on the encoder network and decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain image analysis results; 4) Based on the front-end feature extraction network and encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck features; 5) Calculate the analysis task loss based on the image analysis results, calculate the compression rate loss based on the output of the encoder network, and calculate the reconstruction image loss based on the inversion attack network. Perform directional gradient propagation based on the loss function, optimize network parameters, and obtain compression models with different transmission bit rates. 6) Use different transmission bit rate compression models to achieve privacy-preserving image feature compression under different image data sets and deep learning networks under cloud-edge collaborative reasoning.

[0006] A further improvement of the present invention is that in step 1), the pre-trained deep learning network model is divided from the middle and the network parameters are frozen, and deployed on the edge as a front-end feature extraction network and on the cloud as a back-end feature analysis network, respectively, including: In the image analysis task based on the cloud-edge collaborative system, the neural network model is divided into two sub-models and distributed between the front-end device and the back-end server; in the edge-cloud collaborative reasoning system, the DNN is divided into two parts, represented as ; In this structure, Deployed on edge devices, responsible for collecting input images X from the environment and generating intermediate values , which is then transmitted to the cloud server; It is executed on the cloud server and receives the intermediate features And calculate the final output ;However, in the cloud-edge collaborative intelligent system deployed in mobile cloud, the intermediate features extracted by the deep network are transmitted from the front-end to the back-end.

[0007] A further improvement of the present invention is that, in step 2), based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided, a convolution layer is used for feature fusion, and a fully connected layer is used for compression to realize an encoder network structure; based on the non-local feature domain concentration module and convolution operation, features are reconstructed to realize a decoder network structure, including: In order to remove privacy-sensitive information in intermediate features and improve compression efficiency, we first analyze the reasons why intermediate features are compressible and their privacy is easily leaked from the perspective of information theory. In the deep learning model, the entropy H(T) of the intermediate feature T is expressed as:

[0008] in Representation characteristics The probability distribution of the compressed data; By optimizing the encoding method of the features, the entropy value can be reduced, thereby improving the compression efficiency; According to information theory, the goal of privacy protection is to minimize the mutual information between the compressed data and the original input. , by designing feature compression strategies, , while retaining information relevant to the analysis task ; The calculation of mutual information is expressed as:

[0009] in, is the entropy of the input data, is the uncertainty of the input data under given bottleneck characteristics.

[0010] A further improvement of the present invention is that in step 3), based on the encoder network and the decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain the image analysis results, including: The SET-NET framework includes an encoder and a decoder. In the implementation of the encoder, a feature domain concentration module is designed, which combines the non-local attention mechanism with the residual network. This module generates an implicit importance mask to guide the adaptive compression of latent features. To further reduce communication overhead, entropy coding technology is used to eliminate statistical redundancy in compressed data. The compressed data is transmitted wirelessly and uploaded to the cloud server. After receiving the data, the DNN model decompresses it, and subsequent processing steps generate the final analysis results.

[0011] A further improvement of the present invention is that in step 4), based on the front-end feature extraction network and the encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck features, including: An auxiliary DNN model called the reconstruction network RecNet is built, whose goal is to reconstruct the input image from the bottleneck features; RecNet is a mirror network of the front-end network and the encoder; the encoder and RecNet are trained in an adversarial manner, RecNet tries to recover the input image from the latent features, while the encoder simultaneously tries to destroy the performance of RecNet by manipulating the generated latent features; at the same time, both the encoder and the decoder try to maintain the accuracy of machine vision analysis.

[0012] A further improvement of the present invention is that in step 5), the analysis task loss is calculated based on the image analysis result, the compression code rate loss is calculated based on the output of the encoder network, and the reconstruction image loss is calculated based on the inversion attack network, and directional gradient propagation is performed according to the loss function to optimize the network parameters to obtain different transmission code rate compression models, including: The goal of the machine analysis task is to maximize the mutual information between the intermediate features and the true labels. The basic optimization goal of privacy protection is to focus the feature space on the analysis task while suppressing redundant information that is irrelevant to the input data as much as possible. The loss function is established as follows:

[0013] in, is the compression weight parameter, which can be adjusted to obtain models with different bit rates; parameter Represents the safety factor, by changing Models with different privacy protection strengths can be obtained; and Represents the intermediate features With the input image The mutual information between With the true label mutual information between Reflects the similarity between the reconstructed image and the input, through the reconstruction loss Specifically, the definition is as follows:

[0014] in, is the reconstruction input, and are horizontal and vertical Sobel filters respectively, ∗ represents the convolution operation, and n is the total number of tensor elements in the batch; Sobel filter is incorporated into the reconstruction loss to emphasize edge information; Represents the gradient reversal operation, which is a custom calculation layer in the deep network. Its operation rules are defined as follows:

[0015] .

[0016] A further improvement of the present invention is that, in step 5), different transmission code rate compression models are used under cloud-edge collaborative reasoning to realize privacy-preserving image feature compression under different image data sets and deep learning networks, including: The SET-NET architecture can be trained in an end-to-end manner; using a step-by-step training method, the training process of each batch of data is summarized as follows: the first step is to initialize the encoder network for training, the input X passes through the front-end network, encoder and decoder, and the back-end network calculates and analyzes the accuracy loss and bit rate loss; the second step is to initialize the RecNet network for training, freeze the encoder network, and calculate the reconstruction loss between the reconstructed image and the input image , and obtain an attackable RecNet; the third step is to perform adversarial collaborative training on the encoder network and RecNet. While ensuring the analysis accuracy, RecNet attempts to restore the input image as much as possible from the latent features, while the encoder attempts to destroy the performance of RecNet by manipulating the generated latent features, thereby achieving the purpose of privacy protection.

[0017] A further improvement of the present invention is that it also includes: using different image datasets and deep learning networks to perform performance analysis and verification on an efficient privacy-preserving image feature compression algorithm for cloud-edge collaborative reasoning; systematically evaluating the performance of the proposed method in multiple image analysis tasks, including image classification using ResNet34 on CIFAR-10, classification using ResNet50 on ImageNet, and multi-label classification using ResNet34 on CelebA; during the performance analysis process, adjusting the channel dimension c in SET-NET and the RecNet network structure during training for different DNN models; and evaluating the trade-off between analysis accuracy and reconstructed image quality by using Privacy-Preserving Characteristics.

[0018] A privacy-preserving image feature compression system for edge-cloud collaborative reasoning, comprising: The network parameter acquisition module divides the pre-trained deep learning network model in the middle and freezes the network parameters, which are deployed on the edge as the front-end feature extraction network and on the cloud as the back-end feature analysis network. The network structure confirmation module is based on the front-end feature extraction network and the back-end feature analysis network, adopts a non-local feature domain concentration module and provides an attention mechanism, uses a convolution layer for feature fusion, and uses a fully connected layer for compression to realize an encoder network structure; reconstructs features based on the non-local feature domain concentration module and convolution operations to realize a decoder network structure; The image analysis module, based on the encoder network and the decoder network, inputs the features extracted by the front-end feature extraction network into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain the image analysis results; An image reconstruction module, based on the front-end feature extraction network and the encoder network structure, constructs an inversion attack network to reconstruct the input image from the bottleneck features; The compression model building module calculates the analysis task loss based on the image analysis results, the compression code rate loss based on the output of the encoder network, and the reconstruction image loss based on the inversion attack network. It also performs directional gradient propagation according to the loss function, optimizes the network parameters, and obtains compression models with different transmission code rates. The image feature compression module uses different transmission bit rate compression models under cloud-edge collaborative reasoning to achieve privacy-preserving image feature compression under different image data sets and deep learning networks.

[0019] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a privacy-preserving image feature compression method for edge-cloud collaborative reasoning.

[0020] The present invention has at least the following beneficial technical effects: The present invention provides a privacy-preserving image feature compression method and system for edge-cloud collaborative reasoning. First, a pre-trained high-performance image analysis deep learning network is divided into a front-end network and a back-end network, which are deployed on the edge and the cloud respectively, and their network parameters are frozen; secondly, the intermediate features output by the front-end network are input into the SET-NET network designed by the present invention, and feature fusion is performed through convolutional layers, residual blocks and non-local feature domain concentration modules, the feature map is compressed by a fully connected layer, and the feature vector is converted by entropy coding; RecNet is used to simulate an attacker to reconstruct an image, the loss between the reconstructed image and the input image is calculated, and SET-NET is subjected to adversarial collaborative training to guide feature domain concentration, remove privacy information while retaining information related to the analysis task to the greatest extent; the analysis task loss, compression code rate loss and reconstruction image loss are calculated respectively, and directional gradient propagation is performed according to the loss function to optimize the network parameters; finally, different image data sets and deep learning networks are used to perform performance analysis and verification on an efficient privacy-preserving image feature compression algorithm for cloud-edge collaborative reasoning.

[0021] Furthermore, the SET-NET framework proposed in the present invention achieves dual optimization in terms of privacy protection and feature compression. The framework is flexible in design and can adapt to a variety of deep learning backbone networks without modifying the network structure or fine-tuning parameters. This adaptability greatly improves the flexibility of the system, enabling it to be quickly applied to different image analysis tasks, thereby enhancing the user experience in various scenarios.

[0022] Furthermore, SET-NET achieves a high compression ratio of intermediate features by optimizing portable modules after feature extraction. This capability significantly reduces the required storage space and data transmission bandwidth while effectively protecting local privacy information. Running complex deep learning models on edge devices becomes more feasible, which provides a solution for resource-constrained environments, thereby promoting the widespread application of smart devices.

[0023] Furthermore, the present invention adopts an adversarial training strategy to reduce edge details and privacy-sensitive information in the input image by introducing an attack network. This method not only reduces the risk of model inversion attacks, but also ensures the accuracy of machine vision tasks. At the same time, the design of the feature-aware centralized compression network strengthens the features related to the task, further improving the performance of the model in practical applications.

[0024] In summary, the SET-NET framework of the present invention provides an efficient and secure solution for cloud-edge collaborative reasoning tasks by innovatively combining privacy protection and feature compression technologies. The framework not only demonstrates excellent adaptability in different deep learning models and application scenarios, but also effectively prevents the leakage of privacy information through adversarial training mechanisms. At the same time, the introduction of feature-aware centralized compression networks enables the system to centrally extract information related to analysis tasks, improving overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is an overall structural diagram of a privacy-preserving image feature compression method for edge-cloud collaborative reasoning of the present invention; Figure 2 A schematic diagram of the redundancy removal and privacy protection objectives of the present invention; Figure 3 It is a basic structural diagram of the encoder network and the decoder network of the present invention; Figure 4 This is a structural block diagram of a privacy-preserving image feature compression system for edge-cloud collaborative reasoning according to the present invention. DETAILED DESCRIPTION

[0026] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Example 1 The present invention provides a privacy-preserving image feature compression method for edge-cloud collaborative reasoning, comprising the following steps: 1) Split the pre-trained deep learning network model in the middle and freeze the network parameters, and deploy them on the edge as the front-end feature extraction network and on the cloud as the back-end feature analysis network; 2) Based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided. The convolution layer is used for feature fusion and the fully connected layer is used for compression to realize the encoder network structure. The features are reconstructed based on the non-local feature domain concentration module and the convolution operation to realize the decoder network structure. 3) Based on the encoder network and decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain image analysis results; 4) Based on the front-end feature extraction network and encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck features; 5) Calculate the analysis task loss based on the image analysis results, calculate the compression rate loss based on the output of the encoder network, and calculate the reconstruction image loss based on the inversion attack network. Perform directional gradient propagation based on the loss function, optimize network parameters, and obtain compression models with different transmission bit rates. 6) Use different transmission bit rate compression models to achieve privacy-preserving image feature compression under different image data sets and deep learning networks under cloud-edge collaborative reasoning.

[0028] Example 2 like Figure 1 As shown, the privacy-preserving image feature compression method for edge-cloud collaborative reasoning provided by the present invention includes the following parts: 1) In the edge-cloud collaborative reasoning system, DNN is divided into two parts, represented as In this structure, Deployed on edge devices, responsible for collecting input images X from the environment and generating intermediate values , which is then transmitted to the cloud server; It is executed on the cloud server and receives the intermediate features And calculate the final output

[0029] 2) The main components of the SET-NET framework include an encoder and a decoder. The design of the framework allows it to be ported without modifying the architecture and parameters of a given deep neural network (DNN) model. It is highly adaptable and can be replaced with a general DNN model at a specified split point. In the implementation of the encoder, a feature domain concentration module is designed, which combines a non-local attention mechanism with a residual network. This module generates an implicit importance mask to guide the adaptive compression of latent features. To further reduce communication overhead, entropy coding technology is used to eliminate statistical redundancy in the compressed data. The compressed data is transmitted wirelessly and uploaded to a cloud server. After receiving the data, the DNN model is decompressed, and subsequent processing steps generate the final analysis results.

[0030] 3) An auxiliary DNN model called Reconstruction Network (RecNet) is constructed, whose goal is to reconstruct the input image from the bottleneck features. RecNet is a mirror network of the front-end network and the encoder. We train the encoder and RecNet in an adversarial manner, where RecNet tries to restore the input image as much as possible from the latent features, while the encoder simultaneously tries to destroy the performance of RecNet by manipulating the generated latent features. At the same time, both the encoder and the decoder try to maintain a high accuracy of machine vision analysis.

[0031] like Figure 2 As shown, in order to remove privacy-sensitive information in intermediate features and improve compression efficiency, the present invention first analyzes the reasons why intermediate features are compressible and their privacy is easily leaked from the perspective of information theory. From the perspective of information theory, the compressibility of intermediate features stems from the redundant information they contain. In deep learning models, the representation of features is often highly correlated, and some features show similarities between multiple samples. According to Shannon's information theory, this redundancy allows us to reduce the representation size of data through encoding methods. For example, using entropy coding technology, statistical redundancy can be effectively removed, significantly reducing the storage space and transmission bandwidth required for intermediate features. Specifically, the entropy H(T) of the intermediate feature T can be expressed as:

[0032] in Representation characteristics By optimizing the encoding method of features, the entropy value can be reduced, thereby improving compression efficiency. The privacy vulnerability of intermediate features is mainly reflected in the fact that these features may still contain sensitive information related to the input data. In the process of transmitting intermediate features, attackers can use model inversion technology to recover the original input data from them, resulting in user privacy leakage. According to information theory, the goal of privacy protection is to minimize the mutual information between compressed data and original input. This mutual information reflects the input data With intermediate features The higher the mutual information, the stronger the privacy risk. In order to effectively remove privacy-sensitive information, it is necessary to design a reasonable feature compression strategy to reduce , while retaining information relevant to the analysis task The calculation of mutual information can be expressed as:

[0033] in, is the entropy of the input data, is the uncertainty of the input data under given bottleneck characteristics. By minimizing , the present invention realizes effective protection of privacy information. By analyzing the compressibility and privacy vulnerability of intermediate features, the present invention lays a theoretical foundation for achieving efficient feature compression and privacy protection. This analysis will guide the subsequent feature compression strategy design and adversarial training process to ensure the effectiveness and security of SET-NET in practical applications.

[0034] like Figure 3 As shown in the figure, the structure of the encoder network is designed to efficiently extract and compress the feature information of the input image. First, the network performs preliminary feature extraction on the input image through a 5×5 convolutional layer. The convolution operation can capture the features of the local area and enhance the details of the image. Subsequently, the output of the convolutional layer passes through a generalized division normalization (GDN) layer to improve the expressiveness of nonlinear features, so that the model can adapt to more complex feature distributions. Next, the features are further processed by multiple residual blocks (ResBlocks), each of which consists of two 3×3 convolutional layers and a ReLU activation function. The introduction of residual blocks is to learn residual mapping, ensure the effective propagation of information between layers, and promote the convergence of the network. After completing the preliminary feature extraction, the network introduces a non-local domain aggregation module (NLM). This module focuses on the feature areas related to specific analysis tasks by generating importance masks. This design improves the efficiency of information use, allowing the network to optimize feature representation during processing, thereby improving compression performance. Next, the output after feature extraction and processing is quantized (Q) to convert continuous feature values ​​into discrete values ​​for easy storage and transmission. The quantized features are then compressed using entropy coding techniques to eliminate statistical redundancy, thereby generating an efficient bitstream ready for transmission to the decoder. In the decoder, the bitstream is first restored to the quantized features through an entropy decoder, and then the continuous features are restored through the dequantization process. This process ensures the integrity of the compressed information, allowing the decoder to effectively reconstruct the image. The restored features will pass through the inverse non-local module to recapture the global and local information of the features, thereby enhancing the integrity and effectiveness of the features. Next, the features are processed again through the residual block to optimize the feature representation. Finally, they are normalized through the inverse generalized division normalization (IGDN) layer and mapped back to the original image dimension through a convolutional layer to generate the final decoded image. The entire SET-NET architecture not only achieves efficient image feature compression and transmission by effectively integrating the encoder and decoder, but also ensures the effective protection of privacy information. The design of this network provides an innovative solution for cloud-edge collaborative reasoning, adapting to the dual needs of high performance and high security in modern applications.

[0035] The goal of the machine analysis task is to maximize the mutual information between the intermediate features and the true labels. The basic optimization goal of privacy protection is to focus the feature space on the analysis task while suppressing redundant information that is irrelevant to the input data as much as possible. The loss function is established as follows:

[0036] in, is the compression weight parameter, which can be adjusted to obtain models with different bit rates. Represents the safety factor, by changing Models with different privacy protection strengths can be obtained. and Represents the intermediate features With the input image The mutual information between With the true label For multi-dimensional tensor data, the exact value of the mutual information is usually difficult to calculate. Therefore, in the implementation of the present invention, It is implicitly represented by an objective function tailored for machine vision tasks, such as the cross-entropy loss used in classification tasks. It reflects the similarity between the reconstructed image and the input, which can be obtained by the reconstruction loss Specifically, the definition is as follows:

[0037] in, is the reconstruction input, and are the horizontal and vertical Sobel filters, ∗ denotes the convolution operation, and n is the total number of tensor elements in the batch. Empirically, The value of is set to 0.5. Since privacy-sensitive information is usually associated with image edges, the Sobel filter is incorporated into the reconstruction loss to emphasize edge information. This method enhances the ability of RecNet to reconstruct edges, thereby forcing the encoder network to remove edge information in the bottleneck features. Represents the gradient reversal operation, which is understood as a custom computation layer in a deep network rather than a standard mathematical operation. Its operation rules are defined as follows:

[0038]

[0039] The present invention adopts a step-by-step training method. The training process of each batch of data can be summarized into the following steps: the first step is to initialize the encoder network for training, the input X passes through the front-end network, encoder and decoder, and the back-end network calculates and analyzes the accuracy loss and bit rate loss; the second step is to initialize the RecNet network for training, freeze the encoder network, and calculate the reconstruction loss between the reconstructed image and the input image. , and obtain an attackable RecNet; the third step is to perform adversarial collaborative training on the encoder network and RecNet. While ensuring the analysis accuracy, RecNet attempts to restore the input image as much as possible from the latent features, while the encoder attempts to destroy the performance of RecNet by manipulating the generated latent features, thereby achieving the purpose of privacy protection.

[0040] Example 3 The experimental settings and experimental results are given below.

[0041] Experimental setup. We use ResNet34 on CIFAR-10, ResNet50 on ImageNet for image classification tasks, and ResNet34 on CelebA for multi-label classification. All implementations are based on the PyTorch framework, and the DNN model is split at the end of the second residual block. To fairly evaluate the effectiveness of our method, we employ distributed data parallelism and train it on 8 NVIDIA GeForce RTX 4090 GPUs, each equipped with 24GB RAM. First, we conduct a comprehensive evaluation of our method from the perspectives of compression performance and privacy protection. The encoder network is initially trained using a stochastic gradient descent (SGD) optimizer with momentum set to 0.9 and a batch size of 128. At this stage, the weight decay is set to 0.0001, the learning rate is initialized to 0.1, and is adjusted using a cosine decay schedule in 300 epochs. After the initial training of the encoder network is completed, its parameters are frozen, and then the inversion network is initialized and trained based on the loss function Finally, the encoder and inversion networks were co-trained with privacy weights p set to 0.01, 0.005, and 0.001 to generate models with different privacy protection levels. Different compression rates were obtained by adjusting the compression weights. The weight values ​​used included {2.0, 1.0, 0.5, 0.3, 0.1, 0.075, 0.05, 0.025, 0.01} to achieve the desired compression effect.

[0042] In the experimental results, the comparison algorithms are traditional image coding methods including JPEG, JPEG2000, BPG, and VVC. End-to-end image compression methods include bmshj2018_factorized, bmshj2018_hyperprior, and cheng2020_anchor. They are evaluated in the same application scenario. These image compression methods are evaluated in a cloud-assisted machine vision analysis task, where the image is first encoded in the front end and the compressed bitstream is then transmitted to the back end for image reconstruction. The reconstructed lossy image is analyzed using a pre-trained inference model, and the performance of each model is evaluated by setting different QP values. In addition, we also compare the performance of the supervised feature compression algorithm Sufc based on the official code. Given the scarcity of privacy protection and feature compression algorithms, we also evaluate the latest privacy-preserving feature compression algorithm privacyFC, which is based on the HEVC video encoder. All compared algorithms are tested under the same experimental conditions.

[0043] First, the compression capability is evaluated, especially at low compression bitrates, to demonstrate its effectiveness in collaborative intelligence (CI) systems. The compression ratio is measured in bits per pixel (bpp), calculated by dividing the total output bitstream by the total number of pixels in the input image, which provides an indicator of the compression efficiency of the feature compression algorithm. Next, the privacy protection performance is analyzed, showing the effectiveness of the framework in protecting the reconstructed image and label information. In order to simulate the image reconstruction quality and image attributes that a real attacker may obtain and avoid privacy protection only through the co-trained encoder inversion network parameters, the encoder network parameters are frozen during the test phase. At the same time, the inversion network and the attribute attack network are reinitialized and retrained. In addition, the trade-off between analysis accuracy and reconstructed image quality is analyzed by the privacy protection characteristic (PPC), which is calculated as the product of the final machine analysis accuracy and the distortion of the reconstructed image, indicating that higher PPC values ​​indicate better privacy protection performance and minimal analysis accuracy loss. Table 1 shows the image classification accuracy and reconstructed image quality comparison results of different compression methods at different bitrates on the ImageNet dataset. The classification accuracy of our method is consistent with that of other comparison methods at the same bit rate, the reconstructed image quality is lower than that of other comparison methods, and the PPC value is higher than that of other comparison algorithms. Therefore, the present invention performs better in terms of compression efficiency and privacy protection comprehensive performance.

[0044] Table 1. Index results of different compression methods on the ImageNet dataset

[0045] Example 4 like Figure 4As shown, the present invention provides a privacy-preserving image feature compression system for edge-cloud collaborative reasoning, comprising: The network parameter acquisition module divides the pre-trained deep learning network model in the middle and freezes the network parameters, which are deployed on the edge as the front-end feature extraction network and on the cloud as the back-end feature analysis network. The network structure confirmation module is based on the front-end feature extraction network and the back-end feature analysis network, adopts a non-local feature domain concentration module and provides an attention mechanism, uses a convolution layer for feature fusion, and uses a fully connected layer for compression to realize an encoder network structure; reconstructs features based on the non-local feature domain concentration module and convolution operations to realize a decoder network structure; The image analysis module, based on the encoder network and the decoder network, inputs the features extracted by the front-end feature extraction network into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain the image analysis results; An image reconstruction module, based on the front-end feature extraction network and the encoder network structure, constructs an inversion attack network to reconstruct the input image from the bottleneck features; The compression model building module calculates the analysis task loss based on the image analysis results, the compression code rate loss based on the output of the encoder network, and the reconstruction image loss based on the inversion attack network. It also performs directional gradient propagation according to the loss function, optimizes the network parameters, and obtains compression models with different transmission code rates. The image feature compression module uses different transmission bit rate compression models under cloud-edge collaborative reasoning to achieve privacy-preserving image feature compression under different image data sets and deep learning networks.

[0046] Example 5 The present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of a privacy-preserving image feature compression method for edge-cloud collaborative reasoning.

[0047] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0048] The present application is described with reference to the flowcharts and / or block diagrams of the methods, systems and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0049] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0050] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0051] Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements may be made thereto based on the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection claimed by the present invention.

Claims

1. A privacy-preserving image feature compression method for edge-cloud collaborative reasoning, characterized in that: The following steps are involved: 1) Split the pre-trained deep learning network model in the middle and freeze the network parameters, and deploy them on the edge as the front-end feature extraction network and on the cloud as the back-end feature analysis network; 2) Based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided. The convolution layer is used for feature fusion and the fully connected layer is used for compression to realize the encoder network structure. The features are reconstructed based on the non-local feature domain concentration module and the convolution operation to realize the decoder network structure. 3) Based on the encoder network and decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain image analysis results; 4) Based on the front-end feature extraction network and encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck features; 5) Calculate the analysis task loss based on the image analysis results, calculate the compression rate loss based on the output of the encoder network, and calculate the reconstruction image loss based on the inversion attack network. Perform directional gradient propagation based on the loss function, optimize network parameters, and obtain compression models with different transmission bit rates. 6) Use different transmission bit rate compression models to achieve privacy-preserving image feature compression under different image data sets and deep learning networks under cloud-edge collaborative reasoning.

2. According to claim 1, a privacy-preserving image feature compression method for edge-cloud collaborative reasoning is characterized in that: In step 1), the pre-trained deep learning network model is divided in the middle and the network parameters are frozen, and deployed on the edge as a front-end feature extraction network and on the cloud as a back-end feature analysis network, including: In the image analysis task based on the cloud-edge collaborative system, the neural network model is divided into two sub-models and distributed between the front-end device and the back-end server; in the edge-cloud collaborative reasoning system, the DNN is divided into two parts, represented as ; In this structure, Deployed on edge devices, responsible for collecting input images X from the environment and generating intermediate values , which is then transmitted to the cloud server; It is executed on the cloud server and receives the intermediate features And calculate the final output ;However, in the cloud-edge collaborative intelligent system deployed in mobile cloud, the intermediate features extracted by the deep network are transmitted from the front-end to the back-end.

3. According to claim 2, a privacy-preserving image feature compression method for edge-cloud collaborative reasoning is characterized in that: In step 2), based on the front-end feature extraction network and the back-end feature analysis network, a non-local feature domain concentration module is adopted and an attention mechanism is provided, a convolution layer is used for feature fusion, and a fully connected layer is used for compression to realize an encoder network structure; based on the non-local feature domain concentration module and convolution operation, features are reconstructed to realize a decoder network structure, including: In order to remove privacy-sensitive information in intermediate features and improve compression efficiency, we first analyze the reasons why intermediate features are compressible and their privacy is easily leaked from the perspective of information theory. In the deep learning model, the entropy H(T) of the intermediate feature T is expressed as: in Representation characteristics The probability distribution of the compressed data; By optimizing the encoding method of the features, the entropy value can be reduced, thereby improving the compression efficiency; According to information theory, the goal of privacy protection is to minimize the mutual information between the compressed data and the original input. , by designing feature compression strategies, , while retaining information relevant to the analysis task ; The calculation of mutual information is expressed as: in, is the entropy of the input data, is the uncertainty of the input data under given bottleneck characteristics.

4. According to claim 3, a privacy-preserving image feature compression method for edge-cloud collaborative reasoning is characterized in that: In step 3), based on the encoder network and the decoder network, the features extracted by the front-end feature extraction network are input into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain the image analysis results, including: The SET-NET framework includes an encoder and a decoder. In the implementation of the encoder, a feature domain concentration module is designed, which combines the non-local attention mechanism with the residual network. This module generates an implicit importance mask to guide the adaptive compression of latent features. To further reduce communication overhead, entropy coding technology is used to eliminate statistical redundancy in compressed data. The compressed data is transmitted wirelessly and uploaded to the cloud server. After receiving the data, the DNN model decompresses it, and subsequent processing steps generate the final analysis results.

5. According to claim 4, a privacy-preserving image feature compression method for edge-cloud collaborative reasoning is characterized in that: In step 4), based on the front-end feature extraction network and the encoder network structure, an inversion attack network is constructed to reconstruct the input image from the bottleneck features, including: An auxiliary DNN model called the reconstruction network RecNet is built, whose goal is to reconstruct the input image from the bottleneck features; RecNet is a mirror network of the front-end network and the encoder; the encoder and RecNet are trained in an adversarial manner, RecNet tries to recover the input image from the latent features, while the encoder simultaneously tries to destroy the performance of RecNet by manipulating the generated latent features; at the same time, both the encoder and the decoder try to maintain the accuracy of machine vision analysis.

6. According to claim 5, a privacy-preserving image feature compression method for edge-cloud collaborative reasoning is characterized in that: In step 5), the analysis task loss is calculated based on the image analysis results, the compression code rate loss is calculated based on the output of the encoder network, and the reconstruction image loss is calculated based on the inversion attack network. Directional gradient propagation is performed according to the loss function to optimize the network parameters and obtain different transmission code rate compression models, including: The goal of the machine analysis task is to maximize the mutual information between the intermediate features and the true labels. The basic optimization goal of privacy protection is to focus the feature space on the analysis task while suppressing redundant information that is irrelevant to the input data as much as possible. The loss function is established as follows: in, is the compression weight parameter, which can be adjusted to obtain models with different bit rates; parameter Represents the safety factor, by changing Models with different privacy protection strengths can be obtained; and Represents the intermediate features With the input image The mutual information between With the true label mutual information between Reflects the similarity between the reconstructed image and the input, through the reconstruction loss Specifically, the definition is as follows: in, is the reconstruction input, and are horizontal and vertical Sobel filters, respectively. ∗ denotes the convolution operation and n is the total number of tensor elements in the batch. Sobel filters are incorporated into the reconstruction loss to emphasize edge information. Represents the gradient reversal operation, which is a custom calculation layer in the deep network. Its operation rules are defined as follows: 。 7. The privacy-preserving image feature compression method for edge-cloud collaborative reasoning according to claim 6, characterized in that: In step 5), different transmission bit rate compression models are used to implement privacy-preserving image feature compression under different image data sets and deep learning networks under cloud-edge collaborative reasoning, including: The SET-NET architecture can be trained in an end-to-end manner; using a step-by-step training method, the training process of each batch of data is summarized as follows: the first step is to initialize the encoder network for training, the input X passes through the front-end network, encoder and decoder, and the back-end network calculates and analyzes the accuracy loss and bit rate loss; the second step is to initialize the RecNet network for training, freeze the encoder network, and calculate the reconstruction loss between the reconstructed image and the input image , and obtain an attackable RecNet; the third step is to perform adversarial collaborative training on the encoder network and RecNet. While ensuring the analysis accuracy, RecNet attempts to restore the input image as much as possible from the latent features, while the encoder attempts to destroy the performance of RecNet by manipulating the generated latent features, thereby achieving the purpose of privacy protection.

8. The privacy-preserving image feature compression method for edge-cloud collaborative reasoning according to claim 7, characterized in that: Also includes: Use different image datasets and deep learning networks to perform performance analysis and verification on an efficient privacy-preserving image feature compression algorithm for cloud-edge collaborative reasoning. The performance of the proposed method in multiple image analysis tasks is systematically evaluated, including image classification using ResNet34 on CIFAR-10, classification using ResNet50 on ImageNet, and multi-label classification using ResNet34 on CelebA. During the performance analysis, the channel dimension c in SET-NET and the RecNet network structure during training are adjusted for different DNN models. The trade-off between analysis accuracy and reconstructed image quality is evaluated by using Privacy-Preserving Characteristics.

9. A privacy-preserving image feature compression system for edge-cloud collaborative reasoning, characterized in that: include: The network parameter acquisition module divides the pre-trained deep learning network model in the middle and freezes the network parameters, which are deployed on the edge as the front-end feature extraction network and on the cloud as the back-end feature analysis network. The network structure confirmation module is based on the front-end feature extraction network and the back-end feature analysis network, adopts a non-local feature domain concentration module and provides an attention mechanism, uses a convolution layer for feature fusion, and uses a fully connected layer for compression to realize an encoder network structure; reconstructs features based on the non-local feature domain concentration module and convolution operations to realize a decoder network structure; The image analysis module, based on the encoder network and the decoder network, inputs the features extracted by the front-end feature extraction network into the encoder for compression, and the compressed features are decoded by the decoder and input into the back-end feature analysis network for analysis to obtain the image analysis results; An image reconstruction module, based on the front-end feature extraction network and the encoder network structure, constructs an inversion attack network to reconstruct the input image from the bottleneck features; The compression model building module calculates the analysis task loss based on the image analysis results, the compression code rate loss based on the output of the encoder network, and the reconstruction image loss based on the inversion attack network. It also performs directional gradient propagation according to the loss function, optimizes the network parameters, and obtains compression models with different transmission code rates. The image feature compression module uses different transmission bit rate compression models under cloud-edge collaborative reasoning to achieve privacy-preserving image feature compression under different image data sets and deep learning networks.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a privacy-preserving image feature compression method for edge-cloud collaborative reasoning according to any one of claims 1 to 8.

Images