Method for enhancing antagonistic attack resistance in multilevel model reasoning system

Abstract

The invention provides a method for enhancing adversarial attack resistance in a multilevel model inference system, and belongs to the field of artificial intelligence and computer network security. The method comprises an execution process of an offline stage and an execution process of an online reasoning stage, the offline stage comprises the steps of extracting features and gradients of training data and optimizing the storage space and calculation efficiency of the training data by using an automatic clustering technology and a principal component analysis (PCA) technology, and the online reasoning stage comprises the steps of extracting the features and gradients of the training data through an innovative confidence calculation mechanism. Based on the cosine similarity of reasoning data and training data features and gradients, the prediction confidence of a model on an edge device is evaluated more accurately, and complex or potentially tampered input data needing to be processed by a cloud server is effectively identified, so that the safety and efficiency problems caused by resistance attacks in the prior art are solved, and the safety and efficiency of the edge device are improved. Meanwhile, training data features and gradient storage space are optimized, and high efficiency and accuracy of the system are kept.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer network security, and relates to a technology for enhancing the reliability of model decision-making based on the similarity evaluation of inference data and training data features output by the model, and specifically to a method for enhancing the ability to resist adversarial attacks in a multi-level model reasoning system. Background Art

[0002] In the model reasoning service environment of modern computer networks, multi-level model reasoning systems have become a popular solution for optimizing computing resource allocation and improving reasoning efficiency. This system usually deploys smaller models on edge devices and larger models on cloud servers. The small model on the edge device is responsible for preliminary processing of the reasoning data, and uses the maximum value of the model's predicted probability as the confidence of the prediction, determines whether the confidence exceeds a certain threshold, and decides whether to send the data to the large model on the cloud for further reasoning analysis. This architecture effectively reduces the computing burden of the server and improves the response speed.

[0003] However, a major drawback of this system is its vulnerability to adversarial attacks. Adversarial attacks are attacks that cause the model to produce incorrect outputs by inputting specially designed perturbations. In a multi-level model inference system, an attacker can specifically design input data so that the small model on the edge device misclassifies the input data with high confidence, thereby avoiding sending the data to the more reliable large model for verification. In addition, an attacker can also force the system to continuously send data to the cloud server by making all input data be identified as low confidence, thereby causing the so-called "slowdown attack", which significantly increases the server load and operating costs.

[0004] Existing technologies mainly defend against these attacks by training models to identify adversarial samples or enhancing the preprocessing of input data. However, these methods often require additional computing resources and may affect the response time and accuracy of the system. For example, training models to defend against adversarial samples requires a large number of adversarial sample generation and training processes, while data preprocessing methods (such as image smoothing or denoising) may weaken the model's classification accuracy for normal samples.

[0005] Therefore, a new solution is needed to improve the robustness of multi-level model inference systems against adversarial attacks without sacrificing the efficiency and accuracy of the system. Summary of the invention

[0006] In response to the above problems existing in the prior art, the present invention provides a method for enhancing the ability to resist adversarial attacks in a multi-level model reasoning system. Through an innovative confidence calculation mechanism, based on the cosine similarity of the inference data and training data features and gradients, the prediction confidence of the model on the edge device is more accurately evaluated, and complex or potentially tampered input data that needs to be processed by the cloud server is effectively identified, thereby solving the security and efficiency problems caused by adversarial attacks in the prior art, while optimizing the training data features and gradient storage space to maintain the efficiency and accuracy of the system.

[0007] The technical solution of the present invention is as follows:

[0008] A method for enhancing the ability to resist adversarial attacks in a multi-level model reasoning system, wherein the multi-level model reasoning system integrates a terminal device and a cloud server, prepares a training data set and a reasoning data set in advance, and the cloud server uses the training data set to train a small model configured on the terminal side and a large model configured in the server. After the training is completed, the trained small model is deployed to the terminal device, and the trained large model is deployed to the cloud server. The method is characterized in that after the system is deployed, it also includes an execution process of an offline phase and an execution process of an online reasoning phase, and the specific process includes:

[0009] A. The execution process of the offline stage includes processing the training data: extracting the features and gradients of the training data, and using automatic clustering technology and principal component analysis (PCA) technology to optimize the storage space and computing efficiency of the training data; the specific steps are:

[0010] A1. Training data feature extraction and gradient calculation

[0011] First, on the client side, input the training data into the small model, and save the features and gradients of the last layer of the model corresponding to the training data samples correctly predicted by the small model for subsequent inference services. This step requires that all the features and gradients of the training data be stored in a secure data warehouse to facilitate further processing under the premise of security.

[0012] A2. Optimizing the amount of training data stored

[0013] Use automatic clustering methods to process training data to identify and retain the most informative data points; design and use automatic clustering technology, first set the clustering threshold, identify whether the cluster center in each category of data can represent all data, that is, determine whether the cosine similarity between the cluster center point and all training data is greater than the set clustering threshold; the generated representative data set can effectively reduce the amount of stored data and reduce the processing burden;

[0014] A3. Optimizing the storage dimension of training data

[0015] The principal component analysis (PCA) technique is used to reduce the dimension of the data and remove redundant information and features. This not only reduces the complexity of the calculation, but also retains the most critical features in the data set to optimize the computing efficiency and storage efficiency of the subsequent reasoning stage and improve the reasoning efficiency of the model.

[0016] B. The execution process of the online inference stage is to first use the small model to extract features and gradients of the inference data on the client side and predict its labels; use the saved training data features and gradients to compare with the features and gradients of the inference data to determine whether the prediction results of the small model reach the expected confidence level, so as to determine whether the large model needs to intervene for further inference; the specific steps are as follows:

[0017] B1. Confidence calculation of inference data: In the online inference stage, a feature and gradient similarity calculation module is introduced to evaluate the cosine similarity between the inference data and the training data during inference. The confidence of the small model on the inference data is calculated by comparing the cosine similarity of the features and gradients extracted from the inference data at the last layer of the small model with the features and gradients of the optimized training data.

[0018] B1-1. Small model reasoning: input reasoning data to the small model for processing and predict labels;

[0019] B1-2. Feature extraction and gradient calculation: While completing the small model inference, extract the features and gradient information of the last layer of the model for subsequent cosine similarity analysis;

[0020] B1-3. Confidence calculation: calculate the cosine similarity between the features and gradients of the inference data and the training data, and average them. The average value is used as the confidence of the small model inference.

[0021] B2. Decision execution. A confidence level above a certain threshold means that the small model has a high degree of confidence in its prediction results. The small model returns the prediction label in step B1 and directly displays the results to the user. When the confidence level is below a certain threshold, the data will be automatically forwarded to the large model on the cloud server for reasoning, and the large model will be used to give more reliable results. This step ensures that the system can perform appropriate verification and correction in the face of possible adversarial attacks.

[0022] Furthermore, the terminal devices in the multi-level model reasoning system, including various edge computing devices, have the ability to carry deep neural network models and can complete reasoning and some light training tasks locally.

[0023] Furthermore, the clustering threshold set in step A2 is 0.8.

[0024] Furthermore, the specific threshold value set in step B2 is 0.85.

[0025] The present invention discloses a method for enhancing the ability to resist adversarial attacks in a multi-level model reasoning system. Through an innovative confidence calculation mechanism, based on the cosine similarity of the features and gradients of the inference data and training data, the prediction confidence of the model on the edge device is more accurately evaluated, and complex or potentially tampered input data that needs to be processed by the cloud server is effectively identified. This method not only significantly improves the robustness of the multi-level model reasoning system in the face of adversarial attacks, but also greatly improves the operating efficiency and accuracy of the system by optimizing the data processing flow. The method can be widely used in scenarios where computing tasks need to be dynamically allocated between edge devices and cloud servers, and is particularly suitable for application environments where resources are limited or where there are high requirements for real-time response. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is the execution flow of the offline phase of the present invention;

[0027] Figure 2 This is the execution flow of the online reasoning phase of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further clearly and completely described below through specific embodiments in conjunction with the accompanying drawings.

[0029] This embodiment describes in detail how to implement the present invention in a multi-level model reasoning system including multiple types of devices to improve the system's ability to resist adversarial attacks. System deployment and data preparation of this embodiment: The system integrates end-side devices and cloud servers. End-side devices include smart phones, etc. All of the above devices can be equipped with deep neural network models, and can complete reasoning and some light training tasks locally. Prepare CIFAR-10 training data sets and reasoning data sets. First, in this embodiment, the CIFAR-10 training data set is used in the cloud server to train the small model configured on the end side and the large model configured in the server. After the training is completed, the model is deployed, and the trained small model is deployed to the end-side device, and the trained large model is deployed to the cloud server. In this embodiment, the trained MobileNetV2 small model is configured in the end-side device, and the trained ViT-Large model is configured in the cloud server, which is specially designed to perform image classification tasks.

[0030] The present invention provides a method for enhancing the ability to resist adversarial attacks in a multi-level model reasoning system, including an execution process of an offline phase and an execution process of an online reasoning phase, specifically:

[0031] A. The execution process of the offline phase, such as Figure 1 As shown, it includes processing the training data: after extracting the features and gradients of the training data, using automatic clustering technology and principal component analysis (PCA) to optimize the storage space and computing efficiency of the training data; the specific steps are:

[0032] A1. Training data feature gradient acquisition

[0033] First, on the client side, input the training data into the small model, and save the features and gradients of the last layer of the model corresponding to the training data samples that the small model can correctly predict for subsequent reasoning services. This step requires that all the features and gradients of the training data be stored in a secure data warehouse to facilitate further processing under the premise of security.

[0034] A2. Optimizing the amount of training data stored

[0035] The training data is processed using an automatic clustering method to identify and retain the most informative data points; the design uses an automated clustering technique, first setting the clustering threshold to 0.8 to identify whether the cluster center in each category of data can represent all data, that is, to determine whether the cosine similarity between the center point(s) and all training data is greater than the set clustering threshold. The generated representative data set can effectively reduce the amount of stored data and reduce the processing burden.

[0036] A3. Optimizing the storage dimension of training data

[0037] The principal component analysis (PCA) technique is used to reduce the dimension of the data and remove excess information and redundant features. This not only reduces the complexity of the calculation, but also retains the most critical features in the data set to optimize the computing efficiency and storage efficiency of the subsequent reasoning stage, which is crucial to improving the reasoning efficiency of the model.

[0038] B. The execution flow of the online reasoning phase, such as Figure 2 As shown in the figure, the small model is first used to extract features and gradients of the inference data on the client side, and its labels are predicted at the same time. The saved training data features and gradients are compared with the features and gradients of the inference data to determine whether the prediction results of the small model reach the expected confidence level, and thus determine whether the large model needs to be involved for further inference.

[0039] B1. Confidence calculation of inference data: In the online inference stage, a feature and gradient similarity calculation module is introduced to evaluate the cosine similarity between the inference data and the training data during inference. The confidence of the small model on the inference data is calculated by comparing the cosine similarity of the features and gradients extracted from the last layer of the small model with the features and gradients of the optimized training data.

[0040] B1-1. Small model reasoning. Input reasoning data to the small model for processing and predicting labels.

[0041] B1-2. Feature extraction and gradient calculation. While completing the small model inference, extract the features and gradient information of the last layer of the model, which is crucial for the subsequent cosine similarity analysis.

[0042] B1-3. Confidence calculation. Calculate the cosine similarity between the features and gradients of the inference data and the training data, and average them, and use the average value as the confidence of the small model inference.

[0043] B2. Decision execution. A confidence level above a certain threshold of 0.85 means that the small model has a high degree of confidence in its prediction results, and the small model returns the predicted label in step B1 and directly displays the results to the user. When the confidence level is below a certain threshold, the data will be automatically forwarded to the large model in the cloud server for reasoning, and the large model will be used to give more reliable results. This step ensures that the system can perform appropriate verification and correction in the face of possible adversarial attacks.

[0044] Through the above-described implementation, the present invention not only improves the system's defense capability against adversarial attacks, but also effectively improves the overall operating efficiency and data processing capability through sophisticated data management and intelligent processing strategies. This method is particularly suitable for application scenarios that require high security and high response speed, such as financial services, public safety, and personal data protection.

[0045] Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for enhancing the ability to resist adversarial attacks in a multi-level model reasoning system, wherein the multi-level model reasoning system integrates a terminal device and a cloud server, prepares a training data set and a reasoning data set in advance, and the cloud server uses the training data set to train a small model configured on the terminal side and a large model configured in the server. After the training is completed, the trained small model is deployed to the terminal device, and the trained large model is deployed to the cloud server, characterized in that: After the system is deployed, it also includes the execution process of the offline phase and the execution process of the online inference phase. The specific process includes: A. The execution process of the offline stage includes processing the training data: extracting the features and gradients of the training data, and using automatic clustering technology and principal component analysis (PCA) technology to optimize the storage space and computing efficiency of the training data; the specific steps are: A1. Training data feature extraction and gradient calculation First, on the client side, input the training data into the small model, and save the features and gradients of the last layer of the model corresponding to the training data samples correctly predicted by the small model for subsequent inference services. This step requires that all the features and gradients of the training data be stored in a secure data warehouse to facilitate further processing under the premise of security. A2. Optimizing the amount of training data stored Use automatic clustering methods to process training data to identify and retain the most informative data points; design and use automatic clustering technology, first set the clustering threshold, identify whether the cluster center in each category of data can represent all data, that is, determine whether the cosine similarity between the cluster center point and all training data is greater than the set clustering threshold; the generated representative data set can effectively reduce the amount of stored data and reduce the processing burden; A3. Optimizing the storage dimension of training data The principal component analysis (PCA) technique is used to reduce the dimension of the data and remove redundant information and features. This not only reduces the complexity of the calculation, but also retains the most critical features in the data set to optimize the computing efficiency and storage efficiency of the subsequent reasoning stage and improve the reasoning efficiency of the model. B. The execution process of the online inference stage is to first use the small model to extract features and gradients of the inference data on the client side and predict its labels; use the saved training data features and gradients to compare with the features and gradients of the inference data to determine whether the prediction results of the small model reach the expected confidence level, so as to determine whether the large model needs to intervene for further inference; the specific steps are as follows: B1. Confidence calculation of inference data: In the online inference stage, a feature and gradient similarity calculation module is introduced to evaluate the cosine similarity between the inference data and the training data during inference. The confidence of the small model on the inference data is calculated by comparing the cosine similarity of the features and gradients extracted from the inference data at the last layer of the small model with the features and gradients of the optimized training data. B1-1. Small model reasoning: input reasoning data to the small model for processing and predict labels; B1-2. Feature extraction and gradient calculation: While completing the small model inference, extract the features and gradient information of the last layer of the model for subsequent cosine similarity analysis; B1-3. Confidence calculation: calculate the cosine similarity between the features and gradients of the inference data and the training data, and average them. The average value is used as the confidence of the small model inference. B2. Decision execution. A confidence level above a certain threshold means that the small model has a high degree of confidence in its prediction results. The small model returns the prediction label in step B1 and directly displays the results to the user. When the confidence level is below a certain threshold, the data will be automatically forwarded to the large model on the cloud server for reasoning, and the large model will be used to give more reliable results. This step ensures that the system can perform appropriate verification and correction in the face of possible adversarial attacks.

2. The method according to claim 1, characterized in that The terminal devices in the multi-level model reasoning system, including various edge computing devices, have the ability to carry deep neural network models and can complete reasoning and some light training tasks locally.

3. The method according to claim 1, characterized in that The clustering threshold set in step A2 is 0.

8.

4. The method according to claim 1, characterized in that The specific threshold set in step B2 is 0.85.

Images