Model security protection method and system

Abstract

The present application discloses a method and system for model security protection, which relates to the field of data security technology and includes: obtaining a first basic model and a corresponding first fine-tuned model, wherein the first fine-tuned model is obtained by training the first basic model; calculating the difference between the first basic model and the first fine-tuned model to obtain differential data; encrypting the differential data and transmitting it to the client, so as to be decrypted in the client and merged with a second basic model to obtain a second fine-tuned model, wherein the second basic model is the same as the first basic model. The present application only deploys the basic model at both the server end and the client end, and adopts the transmission mechanism of the differential model, avoiding the security risks and transmission cost problems that may be brought by directly transmitting the entire fine-tuned model, realizing the efficient transmission of model updates to the client while ensuring the model security, and improving the efficiency and security of model deployment and update.

Description

Technical Field

[0001] This application relates to the field of data security technology, and particularly to a method and system for model security protection. Background Art

[0002] The problem of model security has become increasingly prominent in the current technical environment. Especially when the model is transmitted between the server and the client, the leakage risk has become a major hidden danger. This risk not only threatens the confidentiality of the model, but also may affect the integrity and availability of the transmitted model. A malicious attacker may intercept the model data through a man-in-the-middle attack, thereby obtaining sensitive information or tampering with the model, resulting in a decline in model performance or unforeseen consequences. For example, in the medical field, during the process of using a deep learning model for patient diagnosis, if the model is intercepted during transmission, the sensitive information of the patient will face the risk of leakage, seriously infringing on personal privacy; similarly, in the financial industry, if a model used to predict market trends is stolen by a competitor, it may lead to unfair competition and cause economic losses to the company.

[0003] Currently, in order to protect the security of the model during transmission, the industry has taken various measures. For example, by using symmetric or asymmetric encryption algorithms, the confidentiality of model data during transmission can be ensured; digital signatures and hash functions are also used to verify the integrity and authenticity of the model, preventing data from being tampered with; access control and identity authentication mechanisms are used to restrict access to the model only to authorized users, thereby improving security. However, these methods also have some limitations in practical applications: ordinary encryption technologies still have the risk of being cracked, and encryption technologies often increase the computational overhead, affecting the transmission efficiency of the model; although digital signatures and hash functions can verify the integrity of the data, they cannot prevent the data from being stolen; access control and identity authentication mechanisms require complex permission management and user authentication processes, increasing the complexity of the system. Therefore, although these methods can protect the security of the model to a certain extent, there is still room for improvement, and they cannot guarantee the performance of the model while protecting the model security.

[0004] Therefore, how to reliably guarantee both the security and performance of the model is an urgent problem to be solved currently. Summary of the Invention

[0005] The main purpose of this application is to provide a method and system for model security protection, aiming to solve the technical problem of how to reliably guarantee both the security and performance of the model.

[0006] To achieve the above object, this application proposes a method for model security protection, which is applied to the server, and the method for model security protection includes:

[0007] Obtain a first base model and a corresponding first fine-tuned model, where the first fine-tuned model is obtained by training the first base model;

[0008] Calculate the difference between the first base model and the first fine-tuned model to obtain differential data;

[0009] Encrypt and transmit the differential data to the client, so that after decryption in the client, it is merged with a second base model to obtain a second fine-tuned model, where the second base model is the same as the first base model.

[0010] In one embodiment, the step of calculating the difference between the first base model and the first fine-tuned model to obtain differential data includes:

[0011] Perform block partitioning on the first base model and the first fine-tuned model synchronously, and calculate the difference metric of each weight parameter in the corresponding partitioned block based on a preset norm;

[0012] Use the weight parameters with the difference metric greater than a preset difference threshold as the differential items to be transmitted, and construct multiple groups of differential data based on the differential items to be transmitted and the indexes corresponding to the differential items to be transmitted, where one group of differential data corresponds to one block.

[0013] In one embodiment, the step of using the weight parameters with the difference metric greater than the preset difference threshold as the differential items to be transmitted includes:

[0014] Use the weight parameters with the difference metric greater than the preset difference threshold as the differential items to be extracted;

[0015] Calculate the divergence of the parameter distribution in the model layer where each differential item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the change degree and importance of the convolutional kernels in the corresponding model layer;

[0016] Construct a layer importance graph based on the change degree and importance to determine the importance of the model layer where each differential item to be extracted is located based on the layer importance graph;

[0017] Obtain the gradient accumulation graph during the training process of the fine-tuned model to determine the sensitivity of each differential item to be extracted based on the gradient accumulation graph;

[0018] Determine the extraction priority of the differential items to be extracted based on the importance and the sensitivity, and extract the differential items to be extracted according to the extraction priority to obtain the differential items to be transmitted.

[0019] In one embodiment, the step of encrypting and transmitting the differential data to the client includes:

[0020] Generate an asymmetric encryption key pair and send the public key in the asymmetric encryption key pair to the client;

[0021] After the client obtains the public key based on the generated first random number, generate a second random number, and receive the third random number encrypted and returned by the client based on the public key, where the third random number is encrypted based on the fourth random number generated by the client;

[0022] Decrypt the third random number to obtain the fourth random number, and calculate the symmetric encryption key based on the first random number, the second random number, and the fourth random number;

[0023] After encrypting the differential data according to the symmetric encryption key, transmit the encrypted differential data to the client.

[0024] In addition, the present application also proposes a model security protection method, which is applied to the client. The model security protection method includes:

[0025] Receive the encrypted differential data sent by the server and decrypt the differential data, where the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuned model, and the first fine-tuned model is obtained by training the first basic model;

[0026] Obtain a second basic model and merge the decrypted differential data with the second basic model to obtain a second fine-tuned model.

[0027] In an embodiment, before the step of decrypting the differential data includes:

[0028] Generate a first random number, and call the server interface based on the first random number to obtain the public key, where the public key is the public key in the asymmetric encryption key pair generated in the server;

[0029] Obtain the second random number generated by the server and generate a fourth random number, where the second random number is generated by the server after the client obtains the public key;

[0030] Calculate the symmetric decryption key based on the first random number, the second random number, and the fourth random number, so as to perform the step of decrypting the differential data based on the symmetric decryption key.

[0031] In an embodiment, the step of decrypting the differential data includes:

[0032] Load the bytecode of the differential data into memory and decrypt the bytecode;

[0033] Call a preset class loader to convert the decrypted bytecode into each target class, so as to use the each target class as the decoded differential data;

[0034] After the step of merging the decrypted differential data with the second basic model to obtain a second fine-tuning model, the following steps are further included:

[0035] After detecting that the second fine-tuning model has been used up, clear the preset class loader and the each target class in the memory.

[0036] In one embodiment, the step of merging the decrypted differential data with the second basic model to obtain a second fine-tuning model includes:

[0037] Merge the decrypted differential data with the second basic model to obtain a candidate fine-tuning model;

[0038] Perform performance evaluation on the candidate fine-tuning model, and when the performance evaluation result does not meet the preset index, determine the parameters to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model;

[0039] Calculate the performance gradient of the candidate fine-tuning model, and adjust the parameters to be optimized according to the performance gradient, so as to return to execute the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model;

[0040] When the performance evaluation result meets the preset index, use the candidate fine-tuning model as the second fine-tuning model.

[0041] In one embodiment, after the step of merging the decrypted differential data with the second basic model to obtain a second fine-tuning model, the following steps are further included:

[0042] Perform a hash check on the second fine-tuning model to obtain a first hash check result;

[0043] Obtain a second hash check result of the merged model in the server, where the merged model in the server is obtained by merging the differential data and the first basic model;

[0044] Compare the first hash check result and the second hash check result to evaluate the merging effect of the second fine-tuning model according to the comparison result.

[0045] In addition, to achieve the above object, the present application further proposes a model security protection system, and the model security protection system includes:

[0046] A server, configured to obtain a first base model and a corresponding first fine-tuned model, where the first fine-tuned model is obtained by training the first base model; calculate the difference between the first base model and the first fine-tuned model to obtain differential data; encrypt and transmit the differential data to a client, so that after decryption in the client, it is merged with a second base model to obtain a second fine-tuned model, where the second base model is the same as the first base model;

[0047] A client, configured to receive the encrypted differential data sent by the server and decrypt the differential data, where the differential data is obtained by the server after calculating the difference between the first base model and the first fine-tuned model, and the first fine-tuned model is obtained by training the first base model; obtain a second base model, and merge the decrypted differential data with the second base model to obtain a second fine-tuned model.

[0048] One or more technical solutions proposed in this application have at least the following technical effects:

[0049] This application first obtains a first base model and a corresponding first fine-tuned model, and calculates the difference between the first base model and the first fine-tuned model to obtain differential data, so as to use the means of model difference calculation to realize the identification of the difference between the models before and after fine-tuning, providing a basis for subsequent model transmission and merging; then encrypts and transmits the differential data to the client, so that after decryption in the client, it is merged with the second base model to obtain a second fine-tuned model. By using the means of encrypted transmission and model merging, the secure transmission of the differential model is realized, effectively preventing the model from being illegally obtained or tampered with, and greatly improving the security and efficiency of model deployment.

[0050] In summary, this application only deploys the base model at both the server and the client ends respectively, and adopts the transmission mechanism of the differential model, avoiding the security risks and transmission cost problems that may be brought by directly transmitting the entire fine-tuned model, realizing the efficient transmission of model updates to the client on the premise of ensuring model security, so as to quickly generate a fine-tuned model consistent with the server at the client, improving the efficiency and security of model deployment and update. Description of the Drawings

[0051] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0052] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0053] Figure 1 It is a schematic flowchart provided for the first embodiment of the model security protection method of the present application;

[0054] Figure 2 It is a schematic flowchart provided for the second embodiment of the model security protection method of the present application;

[0055] Figure 3 It is a schematic flowchart provided for the third embodiment of the model security protection method of the present application

[0056] Figure 4 It is a schematic flowchart of the brief process of the model security protection method provided for the third embodiment of the present application;

[0057] Figure 5 It is a schematic flowchart of the accuracy maintenance process of the model security protection method provided for the third embodiment of the present application;

[0058] Figure 6 It is a schematic diagram of the module structure of the model security protection system of the embodiment of the present application.

[0059] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0060] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0061] To better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0062] The main solution of the embodiment of the present application is: obtaining a first basic model and a corresponding first fine-tuned model, where the first fine-tuned model is obtained by training the first basic model; calculating the difference between the first basic model and the first fine-tuned model to obtain differential data; encrypting and transmitting the differential data to the client, so as to be decrypted in the client and merged with a second basic model to obtain a second fine-tuned model, where the second basic model is the same as the first basic model.

[0063] Since various means have been adopted in the industry to protect the security of the model during transmission, there are still some limitations in these methods in practical applications: ordinary encryption technology still has the risk of being cracked, and encryption technology often increases the computational overhead and affects the transmission efficiency of the model; digital signatures and hash functions can verify the integrity of data, but cannot prevent data from being stolen; access control and identity authentication mechanisms require complex permission management and user authentication processes, increasing the complexity of the system. Therefore, although these methods can protect the security of the model to a certain extent, there is still room for improvement. They cannot guarantee the performance of the model while protecting its security. Therefore, how to reliably guarantee the security and performance of the model simultaneously is an urgent problem to be solved at present.

[0064] This application provides a solution. By only deploying the base model at both the server side and the client side and adopting the transmission mechanism of the differential model, it avoids the security risks and transmission cost problems that may be brought by directly transmitting the entire fine-tuned model. It realizes the efficient transmission of model updates to the client on the premise of ensuring the security of the model, so as to quickly generate a fine-tuned model consistent with the server side at the client side, improving the efficiency and security of model deployment and update.

[0065] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a model security protection system, etc. that can implement the above functions. Taking the model security protection system as an example, this embodiment and the following embodiments will be described.

[0066] Based on this, the embodiments of this application provide a model security protection method, referring to Figure 1 , Figure 1 which is the schematic flowchart of the first embodiment of the model security protection method of this application.

[0067] In this embodiment, the model security protection method is applied to the server side, and the model security protection method includes steps S10 to S30:

[0068] Step S10, obtain a first base model and the corresponding first fine-tuned model, where the first fine-tuned model is obtained by training the first base model;

[0069] It should be noted that the first base model refers to the pre-trained model that has not been fine-tuned initially and only has certain general performance; the first fine-tuned model refers to the model adjusted for specific tasks through further training and optimization of the first base model.

[0070] Step S20, calculate the difference between the first basic model and the first fine-tuned model to obtain differential data;

[0071] It should be noted that the differential data refers to the parameter difference between the first basic model and the first fine-tuned model, reflecting the changed part of the model during the fine-tuning process.

[0072] It can be understood that in order to efficiently transmit the fine-tuned model, step S20 is performed, which can avoid the risk of model leakage and the high bandwidth and time costs caused by directly transmitting the entire fine-tuned model, thus providing an effective data basis for the secure transmission and efficient transmission in small volume of model data.

[0073] In a feasible implementation manner, step S20 may include steps S21 to S22:

[0074] Step S21, perform block partitioning on the first basic model and the first fine-tuned model synchronously, and calculate the difference metric of each weight parameter in the corresponding blocks after partitioning based on a preset norm;

[0075] It should be noted that the preset norm refers to the mathematical norm used when calculating the weight parameter difference, such as the L1 norm or the L2 norm, which is used to measure the magnitude or length of a vector; a block refers to an independent part into which the model is divided. Taking the YOLO model as an example, the model can be divided into five blocks, namely the input processing layer and the initial convolutional layer, the backbone network convolutional blocks 1-3, the backbone network convolutional blocks 4-5, the feature pyramid network, and the detection head; the difference metric refers to the degree of difference of the weight parameters in the block calculated using the preset norm, which is used to determine whether to transmit the weight parameters of this block.

[0076] It can be understood that during the process of transmitting the differential model, if the volume of the differential data is too large, once it is leaked, it will still pose a great hidden danger to the security of the model. Therefore, step S21 is performed, which provides effective data support for the block encryption of the subsequent differential data by performing block processing on the model.

[0077] Exemplarily, the first basic model and the first fine-tuned model are partitioned into blocks in units of network layers, divided into multiple blocks, and then for each corresponding block, the difference metric of the weight parameters in the two models is calculated using the preset L2 norm, that is, the Euclidean distance between the two vectors is calculated.

[0078] Step S22, use the weight parameters with the difference metric greater than the preset difference threshold as the differential items to be transmitted, and construct multiple groups of differential data based on the differential items to be transmitted and the indexes corresponding to the differential items to be transmitted, where one group of differential data corresponds to one block.

[0079] It should be noted that the difference items to be transmitted refer to the weight parameters whose difference metrics are greater than a preset difference threshold, and these parameters need to be transmitted for model deployment or update; the indexes corresponding to the difference items to be transmitted refer to the positions or indexes of the difference items to be transmitted in the model, which are used to correctly apply these difference items on the client side.

[0080] It can be understood that due to the bandwidth waste and time consumption caused by transmitting minor changes, step S22 is performed. By setting the difference threshold, those weight parameters whose difference metrics exceed the threshold can be screened out, and the changes of these parameters have a significant impact on the model performance. Then, based on these difference items to be transmitted and their corresponding indexes, differential data is constructed for efficient transmission and update, and at the same time, the inefficiency of model deployment or update caused by transmitting unnecessary parameters is avoided, thus realizing a more efficient and accurate model transmission and update process.

[0081] Exemplarily, a difference threshold is set, and the weight parameters whose difference metrics exceed the threshold are used as the difference items to be transmitted. Then, according to these difference items to be transmitted and their indexes in the model, multiple groups of differential data are constructed, and each group of differential data corresponds to a block, including the weight parameters to be transmitted in the block and their indexes.

[0082] In this embodiment, through synchronous block division and difference metric calculation, combined with difference threshold screening and differential data construction, the high bandwidth occupancy and long transmission delay caused by full - scale transmission of the entire model are avoided, and at the same time, the bandwidth waste and time consumption caused by transmitting minor changes are also avoided. And by performing block - based processing on the model, a reliable basis is provided for block - based encryption of subsequent differential data, improving the efficiency and flexibility of model deployment or update, and at the same time ensuring the security of model transmission.

[0083] In step S30, the differential data is encrypted and transmitted to the client, and after being decrypted in the client, it is merged with the second basic model to obtain a second fine - tuned model, where the second basic model is the same as the first basic model.

[0084] It should be noted that the second basic model refers to the basic model deployed on the client, which is the same as the first basic model and has not been fine - tuned; the second fine - tuned model refers to the fine - tuned model obtained by merging the decrypted differential data with the second basic model on the client, which is the same as or basically reaches the performance of the first fine - tuned model.

[0085] It is understandable that, in order to protect the transmission security of the model and at the same time quickly generate a fine-tuned model consistent with the server on the client side, step S30 is performed. Transmitting a small amount of differential data through encrypted transmission can prevent the model from being leaked or tampered with, and transmitting differential data can effectively deploy the model on the client side, avoiding the risks of unauthorized access or malicious attacks during the model transmission process, as well as the high computational cost and time consumption caused by retraining the model on the client side. Thus, while ensuring the transmission security and efficiency of the model, rapid deployment and update of the model are achieved, improving the availability and real-time performance of the model.

[0086] In a feasible implementation manner, the step of encrypting and transmitting the differential data to the client in step S30 may include steps S31 to S34:

[0087] Step S31, generating an asymmetric encryption key pair and sending the public key in the asymmetric encryption key pair to the client;

[0088] Exemplarily, the server processes the login request sent by the client, generates an RSA key pair (K1, K2), and in response to the login request, sends the public key K1 in the key pair to the client.

[0089] Step S32, after the client obtains the public key based on the generated first random number, generating a second random number and receiving the third random number encrypted by the client based on the public key, where the third random number is encrypted based on the fourth random number generated by the client;

[0090] Exemplarily, the client generates a first random number R1 and uses R1 to call the server interface to obtain the RSA public key K1. Then the server generates a second random number R2 and returns the random number R2 and the public key K1. Next, the client generates a fourth random number R4 and encrypts R4 using the RSA public key K1 to obtain a third random number R3 and sends it to the server, and the server receives the R3 returned by the client.

[0091] Step S33, decrypting the third random number to obtain the fourth random number and calculating a symmetric encryption key based on the first random number, the second random number, and the fourth random number;

[0092] Exemplarily, the server decrypts the third random number R3 based on the private key to obtain the fourth random number R4, and calculates the symmetric encryption AES key K3 using R1, R2, and R4 based on the algorithm agreed upon with the client.

[0093] Step S34, encrypting the differential data according to the symmetric encryption key and then transmitting the encrypted differential data to the client.

[0094] It can be understood that since traditional encryption methods often have difficulty in ensuring the relatively high security requirements of data through a single encryption mechanism, step S34 is carried out. By adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, that is, using a symmetric encryption and decryption algorithm to encrypt and decrypt the model, and using an asymmetric encryption and decryption algorithm to encrypt and decrypt the model key, the cost of decrypting the model after leakage or directly obtaining the decrypted model from the disk is increased.

[0095] In this embodiment, by adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, the security risk of decrypting the model after leakage or directly obtaining the decrypted model from the disk is greatly avoided, and the model is further effectively prevented from being illegally obtained or tampered with.

[0096] This embodiment provides a model security protection method. By only deploying the basic model at both the server side and the client side respectively, and adopting a differential model transmission mechanism, the security risks and transmission cost problems that may be brought about by directly transmitting the entire fine-tuned model are avoided. On the premise of ensuring the model security, the model update is efficiently transmitted to the client, so that a fine-tuned model consistent with the server side is quickly generated at the client, and the efficiency and security of model deployment and update are improved.

[0097] In a feasible embodiment, the step of taking the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be transmitted in step S22 may include steps S221 to S225:

[0098] Step S221, taking the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be extracted;

[0099] It should be noted that the difference items to be extracted refer to the items in which the weight parameters change significantly during the model fine-tuning process, and these changes may have an important impact on the model performance.

[0100] Exemplarily, after the model fine-tuning, calculate the difference metric of each weight parameter, such as using the absolute difference or the relative change rate. Set a preset difference threshold, and mark the weight parameters with the difference metric exceeding the threshold as the difference items to be extracted. This method ensures that only the weight parameters that have a significant impact on the model performance are transmitted, reducing unnecessary data transmission.

[0101] Step S222, calculate the divergence of the parameter distribution in the model layer where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the change degree and importance of the convolution kernel in the corresponding model layer;

[0102] It should be noted that the degree of change of the convolution kernel refers to the amount of change in the weight parameters (i.e., the convolution kernel) of the convolutional layer during the model training process, and the importance refers to the degree of influence of these changes on the model performance.

[0103] It can be understood that since it is often difficult to determine which parameters need to be transmitted or which parameters are not accurately judged to need to be transmitted during the model transmission process, so step S222 is performed. By calculating the divergence of the parameter distribution and applying the structural similarity index, the degree of change and importance of the convolution kernel are evaluated, avoiding blindly transmitting all the changed weight parameters, and improving the pertinence and efficiency of the transmission.

[0104] Exemplarily, for each model layer where the differential item to be extracted is located, calculate the divergence of the parameter distribution, such as using the Kullback-Leibler divergence or the Jensen-Shannon divergence, and then apply the structural similarity index (SSIM) to evaluate the degree of change and importance of the convolution kernel. This helps to identify which convolution kernels have changed significantly during the fine-tuning process and have an important impact on the model performance.

[0105] Step S223, construct a hierarchical importance graph based on the degree of change and importance, so as to determine the importance of each model layer where the differential item to be extracted is located based on the hierarchical importance graph;

[0106] It should be noted that the hierarchical importance graph is a visualization tool used to display the importance degree of each model layer in the feature extraction and decision-making process, helping to understand the model structure and optimize the model; the importance of the model layer refers to the contribution degree of different layers in the model to the final output result, and the layer with higher importance has a greater impact on the model performance.

[0107] It can be understood that in order to clarify the contribution degree of each layer to the model performance, so step S223 is performed. By constructing a hierarchical importance graph, it is possible to avoid excessive attention to unimportant layers, and optimize the model structure and performance.

[0108] Exemplarily, based on the calculated divergence and structural similarity index, construct a hierarchical importance graph. This graph intuitively shows the importance degree of each model layer in the feature extraction and decision-making process. By analyzing the hierarchical importance graph, it is possible to determine which layers contribute the most to the model performance, thereby optimizing the model structure and transmission strategy.

[0109] Step S224, obtain the gradient accumulation graph of the fine-tuned model during the training process, so as to determine the sensitivity of each differential item to be extracted based on the gradient accumulation graph;

[0110] It should be noted that the gradient accumulation graph is a graph that records the gradient change of each weight parameter during the model training process and is used to analyze the sensitivity of the weight parameter to the loss function; the sensitivity of the item to be extracted refers to the degree of sensitivity of the item to be extracted to the change of the model performance, and the item with higher sensitivity has a greater impact on the model performance.

[0111] It can be understood that since there is a certain correlation between the sensitivity of the item and the model performance, step S224 is performed. By analyzing the gradient accumulation graph to determine the sensitivity of the item to be extracted, it is possible to avoid excessive transmission of insensitive items, improving the transmission efficiency and the model performance.

[0112] Exemplarily, during the model training process, the gradient change of each item to be extracted is recorded to form a gradient accumulation graph. By analyzing the gradient accumulation graph, the sensitivity of the item to be extracted to the loss function can be determined. The item with higher sensitivity has a greater impact on the model performance and should be transmitted preferentially.

[0113] Step S225, determining the extraction priority of the item to be extracted based on the importance and the sensitivity, and extracting the item to be extracted according to the extraction priority to obtain the item to be transmitted.

[0114] It should be noted that the extraction priority is to determine the priority of the item in the transmission process according to factors such as the importance and sensitivity of the item, and preferentially transmit the item that has a greater impact on the model performance.

[0115] It can be understood that since there is a sequential order in the transmission of model parameters, step S225 is performed. By comprehensively considering factors such as the importance and sensitivity of the item to determine the extraction priority, it is possible to avoid the decline of the model performance caused by unreasonable transmission order, improving the efficiency and accuracy of model deployment and update.

[0116] Exemplarily, by comprehensively considering the hierarchical importance and gradient sensitivity, an extraction priority is determined for each item to be extracted. The item with higher priority has a greater impact on the model performance and should be transmitted preferentially. The items are sorted according to the extraction priority, and the items with higher priority are selected for transmission to obtain the item to be transmitted. This method improves the efficiency and accuracy of model update and reduces the waste of transmission bandwidth and time.

[0117] In this embodiment, a method for optimizing the weight transmission of a neural network model is proposed. First, weight parameters with a difference metric exceeding a threshold are screened out as the difference items to be extracted, ensuring that only weight changes that significantly affect the model performance are transmitted. Then, for each model layer where a difference item to be extracted is located, the divergence of the parameter distribution is calculated to quantify the degree of change in the parameter distribution. Next, the structural similarity index is applied to evaluate the degree of change and importance of the convolutional kernels in the corresponding model layer, and the convolutional kernels that are important for feature extraction and model decision-making are identified. Based on the results of the divergence and SSIM, a layer importance graph is constructed to intuitively display the contribution degree of each model layer to the overall model performance. At the same time, by obtaining the gradient accumulation graph of the fine-tuned model during the training process, the cumulative sum of the absolute values of the gradients of each difference item to be extracted is calculated to determine its sensitivity, that is, its impact on the model performance. Finally, considering both the layer importance and the gradient sensitivity, an extraction priority is set for each difference item to be extracted, and the difference items are sorted according to the priority. The difference items with higher priority are transmitted first, so as to obtain the difference items to be transmitted. This method effectively avoids the problems of high bandwidth and time costs caused by transmitting all weight parameters, realizes the efficient update and optimization of the model, and at the same time maintains the performance improvement of the model.

[0118] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as that in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 2 , in this embodiment, the model security protection method is applied to the client, and the model security protection method includes steps S01~S02:

[0119] Step S01, receiving the encrypted differential data sent by the server and decrypting the differential data, wherein the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuned model, and the first fine-tuned model is obtained by training the first basic model;

[0120] It can be understood that in order to ensure the security of the differential data during the transmission process and prevent data leakage and tampering, so step S01 is performed, which can avoid the high bandwidth consumption and data security risks caused by transmitting the entire model. By transmitting the encrypted differential data, the amount of transmitted data is greatly reduced, the transmission efficiency is improved, and the data security is guaranteed at the same time.

[0121] Step S02, obtaining a second basic model and merging the decrypted differential data with the second basic model to obtain a second fine-tuned model.

[0122] It can be understood that in order to quickly apply the model update on the server side based on the local second base model and generate the second fine-tuned model, step S02 is performed, which can avoid the high bandwidth, long waiting time required for re-transmitting the entire model, and the risk of security leakage. By merging the differential data, the model can be efficiently deployed or updated, while maintaining the improvement of the model performance, significantly reducing the bandwidth usage and transmission time, and improving the security of the model transmission.

[0123] In a feasible implementation manner, the step of decrypting the differential data in step S01 may include steps S011 to S012:

[0124] Step S011, loading the bytecode of the differential data into the memory and decrypting the bytecode;

[0125] It should be noted that the decryption process can use the symmetric decryption key calculated based on a random number and is performed in an independent thread different from the main thread to reduce the risk of memory leakage in the main thread.

[0126] Step S012, calling a preset class loader to convert the decrypted bytecode into each target class, so as to use the each target class as the decoded differential data.

[0127] It should be noted that the preset class loader refers to the class loader used to dynamically load the decrypted bytecode. This class loader has specific permissions and functions, such as class isolation and unloading, that is, when certain classes are no longer needed, the reference to the entire custom class loader can be discarded; on-demand loading and unloading, that is, it can be designed to load the decrypted model only when needed and clean it immediately after use; memory usage control, that is, it can more precisely control the life cycle of classes and avoid occupying memory for a long time; the target class refers to the class converted from the decrypted differential data, and these classes represent the updated part of the model.

[0128] It can be understood that since the decrypted bytecode can be converted into an executable class object and then applied to model deployment or update, step S012 is performed, which can avoid the problem that a common class loader may not be able to safely load the dynamically generated class, resulting in security risks. By loading the target class through the preset class loader, the security and correctness of class loading are ensured, which is convenient for the dynamic update of the model.

[0129] After step S02, the model security protection method may further include step S013:

[0130] Step S013, after detecting that the second fine-tuned model has been used up, clearing the preset class loader and the each target class in the memory.

[0131] It can be understood that since the classes loaded by the standard class loaders (such as the application class loader AppClassLoader) will be kept in memory until the class loader is garbage collected, performing step S013 can avoid the situation where the dynamically generated classes, if not cleared in time, will occupy a large amount of memory, leading to a decline in system performance. By clearing the preset class loader and the target class in time, the memory resources are released, and the security and stability of the system are improved.

[0132] Exemplarily, immediately after the model is used up, call System.gc() and manually set the object to null, and implement a custom TensorBuffer class. After use, actively call the cleaning method, force the finalizer to be triggered after key operations to ensure that resources are released in time. At the same time, develop a memory leak prevention daemon thread to regularly check and clean up the unreleased model resources.

[0133] In this embodiment, by safely loading and processing the encrypted differential data in memory, using the preset class loader to dynamically load the target class, and clearing the resources in memory in time after use, the efficient and secure update of the model is realized, avoiding problems such as data leakage and memory leakage, and ensuring the performance and security of the system.

[0134] In a feasible embodiment, the step of merging the decrypted differential data with the second base model in step S02 to obtain the second fine-tuning model may include steps S021 to S024:

[0135] Step S021, merge the decrypted differential data with the second base model to obtain a candidate fine-tuning model;

[0136] It should be noted that the candidate fine-tuning model refers to the preliminary fine-tuning model generated after the differential data is merged on the client side and is used for subsequent performance evaluation and optimization.

[0137] Exemplarily, after the client receives the encrypted differential data, it uses a pre-agreed key to decrypt it to obtain the original differential data, and then loads these differential data into the second base model. Through the model's parameter update mechanism (such as the backpropagation algorithm), the differential data is merged into the model's parameters to generate a candidate fine-tuning model. The specific merging process can use the SIMD instruction set (NEON / SSE) to accelerate the weight merging process, and implement multi-threaded parallel merging of large network layers to improve the processing efficiency. At the same time, use the GPU to accelerate large matrix operations (applicable to devices that support GPUs). In addition, develop a memory pre-allocation strategy to reduce the memory allocation overhead during the merging process.

[0138] Step S022: Perform a performance evaluation on the candidate fine-tuning model. In the case where the performance evaluation result does not meet the preset metrics, determine the parameters to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model;

[0139] It should be noted that the sensitivity of each parameter in the candidate fine-tuning model refers to the degree of influence of each parameter on the model performance and is used to identify the parameters that need to be optimized.

[0140] It can be understood that since there will be a problem of model accuracy loss during the differential data merging process, performing step S022 can avoid the problem that the model performance may be poor and unable to meet the actual application requirements due to the lack of performance evaluation. By identifying the parameters that need to be optimized through performance evaluation, it provides a reliable direction for model optimization.

[0141] Exemplarily, the client uses a set of test data sets to perform a performance evaluation on the candidate fine-tuning model. The evaluation metrics may include accuracy, recall rate, F1 value, etc. If the evaluation result does not meet the preset performance metrics, the client will analyze the sensitivity of each parameter in the model and determine which parameters have the greatest impact on the model performance by calculating gradients or using feature importance analysis methods, so as to determine the parameters to be optimized.

[0142] Step S023: Calculate the performance gradient of the candidate fine-tuning model and adjust the parameters to be optimized according to the performance gradient, so as to return to the step of performing the performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model;

[0143] It should be noted that the performance gradient of the candidate fine-tuning model refers to the derivative of the model performance with respect to each parameter and is used to guide the direction and amplitude of parameter adjustment.

[0144] It can be understood that since the lack of gradient calculation and parameter adjustment may lead to the inability to improve the model performance, performing step S023 precisely optimizes the model performance through gradient calculation and parameter adjustment, thereby improving the merging accuracy and efficiency of the model.

[0145] Exemplarily, the client uses the backpropagation algorithm to calculate the performance gradient of the candidate fine-tuning model on the test data set, that is, the derivative of the model performance with respect to each parameter, and then uses an optimization algorithm (such as stochastic gradient descent, Adam, etc.) to adjust the parameters to be optimized according to this gradient information. The adjusted model is evaluated again for performance, and this process may need to be iterated multiple times until the model performance reaches the preset metrics.

[0146] Step S024: In the case where the performance evaluation result meets the preset metrics, use the candidate fine-tuning model as the second fine-tuning model.

[0147] Exemplarily, when the performance evaluation result of the candidate fine-tuning model on the test data set reaches or exceeds the preset performance metrics, the client confirms that the model optimization is completed and officially uses the candidate fine-tuning model as the second fine-tuning model. This model will be used for subsequent inference tasks or further training.

[0148] In this embodiment, a candidate fine-tuning model is generated by merging differential data, performance evaluation and parameter optimization are performed, and finally a fine-tuning model that meets the requirements is confirmed, achieving efficient optimization and verification of the model, avoiding problems such as poor performance and improper parameter adjustment caused by the loss of merging accuracy of the model, and ensuring the accuracy and efficiency of the model.

[0149] In this embodiment, by only deploying the basic model at both the server side and the client side and adopting the transmission mechanism of the differential model, the security risks and transmission cost problems that may be brought about by directly transmitting the entire fine-tuning model are avoided. On the premise of ensuring the security of the model, the model update is efficiently transmitted to the client, so that a fine-tuning model consistent with the server side can be quickly generated at the client side, improving the efficiency and security of model deployment and update.

[0150] In a feasible embodiment, before the step of decrypting the differential data in step S01, steps S100 to S300 may further be included:

[0151] Step S100, generating a first random number and obtaining a public key by calling the server interface based on the first random number, where the public key is the public key in the asymmetric encryption key pair generated in the server;

[0152] Exemplarily, the client generates a first random number R1 and uses R1 to call the server interface to obtain the public key K1 in the asymmetric encryption key pair (K1, K2) generated by the server in processing and responding to the login request.

[0153] Step S200, obtaining the second random number generated by the server and generating a fourth random number, where the second random number is generated by the server after the client obtains the public key;

[0154] Exemplarily, the server generates a second random number R2 after the client obtains the public key K1. The client obtains R2 and generates a fourth random number R4. Among them, R4 can be encrypted with K1 to obtain a third random number R3 and then sent to the server, so that the server uses the private key to decrypt R3 to obtain R4 after receiving R3, and thus calculates the symmetric encryption key K3 using R1, R2, and R4 in the server.

[0155] Step S300, calculate a symmetric decryption key based on the first random number, the second random number, and the fourth random number, and perform the step of decrypting the differential data based on the symmetric decryption key.

[0156] It can be understood that since traditional encryption methods often have difficulty in meeting the high security requirements of data through a single encryption mechanism, step S300 is performed. By adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, that is, using symmetric encryption and decryption algorithms to encrypt and decrypt the model, and using asymmetric encryption and decryption algorithms to encrypt and decrypt the model key, the cost of decrypting the model after leakage or directly obtaining the decrypted model from the disk is increased.

[0157] Exemplarily, calculate the AES symmetric decryption key K3 based on the first random number R1, the second random number R2, and the fourth random number R4, and store K3 in the Keystore for subsequent decryption of the differential model.

[0158] In this embodiment, by adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, the security risk of decrypting the model after leakage or directly obtaining the decrypted model from the disk is greatly avoided, and the model is further effectively prevented from being illegally obtained or tampered with.

[0159] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar content as in the above-mentioned second embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , after step S02, steps S03 to S05 may further be included:

[0160] Step S03, perform a hash check on the second fine-tuned model to obtain a first hash check result;

[0161] It should be noted that the first hash check result is a unique identifier obtained by performing a hash operation on the second fine-tuned model generated by the client.

[0162] It can be understood that in order to ensure that the second fine-tuned model generated by the client is consistent with the merged model on the server, step S03 is performed, which can avoid the differential model being maliciously modified or damaged during transmission, thereby ensuring the integrity and security of the model and verifying the correctness of the model.

[0163] Exemplarily, the client uses the SHA-256 hash algorithm to perform an operation on the second fine-tuned model to generate a first hash check result to ensure the integrity of the model data.

[0164] Step S04, obtain a second hash check result of the merged model in the server, where the merged model in the server is obtained by merging the differential data and the first basic model;

[0165] It should be noted that the second hash verification result is a unique identifier obtained by performing a hash operation on the merged model obtained by merging the differential data and the first base model in the server.

[0166] It can be understood that in order to compare with the hash verification result of the client and confirm that the merged model of the server is consistent with the model of the client, step S04 is performed, which can avoid errors in subsequent applications caused by the inconsistency between the server model and the client model, thus ensuring the consistency of the server and client models and guaranteeing the correctness of the application.

[0167] Exemplarily, the client sends a request to the server, requesting to obtain the SHA-256 hash value of the merged model. After receiving the request, the server performs a hash operation on the merged model, generates the second hash verification result, and sends it back to the client.

[0168] Step S05, compare the first hash verification result and the second hash verification result to evaluate the merging effect of the second fine-tuning model according to the comparison result.

[0169] It should be noted that the comparison result is the result of comparing the first hash verification result and the second hash verification result. If the two are the same, it means that the model merging is correct; if they are different, it means that there is a problem with the model merging.

[0170] In addition, it should be noted that after evaluating the merging effect, if the merging effect does not reach the preset effect, a resend request can be sent to the server through the client to request the server to re-transmit the differential data, and after continuously sending the resend request multiple times, a differential data reconstruction request can be sent to the server through the client to enable the server to recalculate the difference between the first base model and the first fine-tuning model, obtain the differential data, and return the differential data.

[0171] It can be understood that by comparing the hash values, it can be confirmed whether the second fine-tuning model of the client is consistent with the merged model of the server and evaluate the merging effect. Therefore, performing step S05 can avoid problems such as the inability to recognize incorrect model merging, resulting in performance degradation or application errors, thus ensuring the correctness and effectiveness of model merging and improving the reliability and performance of applying the model in the client.

[0172] Exemplarily, the client compares the first hash verification result with the second hash verification result. If the two are consistent, it means that the second fine-tuning model is consistent with the merged model of the server and the merging effect is good; if they are inconsistent, it indicates that there may be errors or data tampering during the merging process, and further investigation and processing are required.

[0173] In this embodiment, by synchronously merging differential data and the base model in the client and the server and adopting the method of hash verification, the integrity and correctness of the model during transmission and merging can be ensured, data tampering and damage can be avoided, and the reliability and performance of applying the model in the client are improved.

[0174] Exemplarily, to help understand the implementation process of the model security protection method obtained by combining the above Embodiment 1 and Embodiment 2, please refer to Figure 4 , Figure 4 A brief flowchart of a model security protection method is provided. Specifically:

[0175] In terms of model security protection, three aspects are considered: model transmission security, APP security, and model file security, which is particularly important on Android devices (the following are some security measures adopted on Android devices):

[0176] For transmission security, to increase the cost of obtaining the model or key during transmission, an internal transmission protocol is implemented for model transmission. At the same time, the SSL certificate public key is fixed in the APP, and third-party certificates are not trusted. If it is a third-party certificate, the connection is interrupted to prevent packet capture, and the public key needs to be updated regularly according to the certificate validity period.

[0177] For APP security, to increase the cost of dynamic debugging, APP code obfuscation can be performed to delete debugging information, symbol tables, etc. (ProGuard). At the same time, APP shelling, ROOT device detection, virtual machine detection, etc. are carried out. For ROOT devices, a relatively poor model can be loaded. In addition, anti-debugging (Frida, Xpose) is introduced.

[0178] For model security, to increase the cost of decrypting the model or directly obtaining the decrypted model from the disk after the model is leaked, AES can be used to encrypt and decrypt the model, and RSA can be used to encrypt and decrypt the model key, and the key is stored in the keyStore. In addition, ensure that the decrypted model can only be in memory and cannot appear in any accessible area, uninstall it in time after use, and build the pre-trained model into the APP. The server extracts the weight layers with differences from the pre-trained model after fine-tuning, and the client compares and merges them to ensure that the data does not land.

[0179] The system architecture in the figure consists of four parts, namely, the key management service, the server and the terminal (i.e., the client), and the HTTP transmission channel between the server and the client. Among them, the key management service stores and returns keys to the server and the terminal. In the server, the original model (i.e., the first basic model and the corresponding first fine-tuned model) is first subjected to differential extraction to obtain differential data, and then the differential data is encrypted and transmitted based on the key provided by the key management service. At the terminal, the transmitted data is decrypted for the model and the key, and after decryption, it is differentially merged with the second basic model to obtain the second fine-tuned model.

[0180] Furthermore, please refer to Figure 5 , the figure shows the implementation process of maintaining accuracy when merging differential data and the basic model in the client. First, model merging and accuracy loss analysis are performed. Among them, after the differential model is merged, preliminary quantization analysis, model loss analysis, and sensitivity parameter identification are performed. Then, in the quantization-aware merging, the differential weights are de-quantized and high-precision merging is performed. Next, quantization awareness and parameter adjustment are introduced for re-quantization. Then, the key layer insertion points are identified, and a calibration layer with dynamically updated parameters is created, and corresponding feature statistics and distribution alignment are performed to calibrate and compensate the merged model based on this calibration layer. Finally, iterative correction of the model accuracy is performed, including first evaluating the performance of the model, and then calculating the model performance gradient and parameter sensitivity analysis in the case of non-compliance with the evaluation, and adjusting the adaptive parameters. Based on the adjusted parameters, the performance evaluation is performed again until the evaluation is qualified, and the calibration layer is frozen to obtain the second fine-tuned model with excellent performance.

[0181] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the model security protection method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0182] This application also provides a model security protection system. Please refer to Figure 6 , the model security protection system includes:

[0183] The server 10 is used to obtain the first basic model and the corresponding first fine-tuned model, where the first fine-tuned model is obtained by training the first basic model; calculate the difference between the first basic model and the first fine-tuned model to obtain differential data; encrypt and transmit the differential data to the client 20 for merging with the second basic model after decryption in the client 20 to obtain the second fine-tuned model, where the second basic model is the same as the first basic model;

[0184] The client 20 is used to receive the encrypted differential data sent by the server 10 and decrypt the differential data. The differential data is obtained by the server 10 after calculating the difference between the first basic model and the first fine-tuned model, and the first fine-tuned model is obtained by training the first basic model. The client 20 acquires the second basic model and merges the decrypted differential data with the second basic model to obtain the second fine-tuned model.

[0185] Optionally, the server 10 is further used for:

[0186] Perform block partitioning on the first basic model and the first fine-tuned model synchronously, and calculate the difference metric of each weight parameter in the corresponding block after partitioning based on a preset norm.

[0187] Use the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be transmitted, and construct multiple groups of differential data based on the difference items to be transmitted and the indexes corresponding to the difference items to be transmitted, where one group of differential data corresponds to one block.

[0188] Optionally, the server 10 is further used for:

[0189] Use the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be extracted.

[0190] Calculate the divergence of the parameter distribution in the model layer where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the change degree and importance of the convolution kernel in the corresponding model layer.

[0191] Construct a hierarchical importance graph based on the change degree and importance to determine the importance of the model layer where each difference item to be extracted is located based on the hierarchical importance graph.

[0192] Acquire the gradient accumulation graph during the training process of the fine-tuned model to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph.

[0193] Determine the extraction priority of the difference items to be extracted based on the importance and the sensitivity, and extract the difference items to be extracted according to the extraction priority to obtain the difference items to be transmitted.

[0194] Optionally, the server 10 is further used for:

[0195] Generate an asymmetric encryption key pair and send the public key in the asymmetric encryption key pair to the client 20.

[0196] After the client 20 obtains the public key based on the generated first random number, it generates a second random number and receives the third random number encrypted and returned by the client 20 based on the public key, where the third random number is encrypted based on the fourth random number generated by the client 20;

[0197] Decrypt the third random number to obtain the fourth random number, and calculate the symmetric encryption key based on the first random number, the second random number, and the fourth random number;

[0198] After encrypting the differential data according to the symmetric encryption key, transmit the encrypted differential data to the client 20.

[0199] Optionally, the client 20 is further configured to:

[0200] Generate a first random number, and call the server 10 interface based on the first random number to obtain the public key, where the public key is the public key in the asymmetric encryption key pair generated in the server 10;

[0201] Obtain the second random number generated by the server 10 and generate a fourth random number, where the second random number is generated by the server 10 after the client 20 obtains the public key;

[0202] Calculate the symmetric decryption key based on the first random number, the second random number, and the fourth random number, so as to perform the step of decrypting the differential data in the memory based on the symmetric decryption key.

[0203] Optionally, the client 20 is further configured to:

[0204] Load the bytecode of the differential data in the memory and decrypt the bytecode;

[0205] Call a preset class loader to convert the decrypted bytecode into each target class, so as to use the each target class as the decoded differential data;

[0206] After the step of merging the decrypted differential data with the second base model to obtain the second fine-tuning model, the following steps are further included:

[0207] After detecting that the second fine-tuning model has been used, clear the preset class loader and the each target class in the memory.

[0208] Optionally, the client 20 is further configured to:

[0209] Merge the decrypted differential data with the second base model to obtain a candidate fine-tuning model;

[0210] Perform performance evaluation on the candidate fine-tuning model, and when the performance evaluation result does not meet the preset metrics, determine the parameters to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model;

[0211] Calculate the performance gradient of the candidate fine-tuning model, and adjust the parameters to be optimized according to the performance gradient, so as to return to perform the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model;

[0212] When the performance evaluation result meets the preset metrics, use the candidate fine-tuning model as the second fine-tuning model.

[0213] Optionally, the client 20 is further configured to:

[0214] Perform hash verification on the second fine-tuning model to obtain a first hash verification result;

[0215] Obtain a second hash verification result of the merged model in the server 10, where the merged model is obtained by merging the differential data and the first basic model in the server 10;

[0216] Compare the first hash verification result and the second hash verification result to evaluate the merging effect of the second fine-tuning model according to the comparison result.

[0217] The model security protection system provided by this application adopts the model security protection method in the above embodiment, and can solve the technical problem of how to reliably guarantee the security and performance of the model at the same time. Compared with the prior art, the beneficial effects of the model security protection system provided by this application are the same as those of the model security protection method provided by the above embodiment, and other technical features in the model security protection system are the same as the features disclosed in the method of the above embodiment, and will not be elaborated here.

[0218] The above are only some embodiments of this application, and thus do not limit the patent scope of this application. Any equivalent structural transformation made under the technical concept of this application, or direct / indirect application in other related technical fields, is included in the patent protection scope of this application.

Claims

1. A method for model security protection, characterized in that, Applied to the server side, the model security protection method includes: Obtain a first basic model and a corresponding first fine-tuned model, where the first fine-tuned model is obtained by training the first basic model; Perform block partitioning on the first basic model and the first fine-tuned model synchronously, and calculate the difference metric of each weight parameter in the corresponding block after partitioning based on a preset norm; Take the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be extracted; Calculate the divergence of the parameter distribution in the model layer where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the change degree and importance of the convolution kernels in the corresponding model layer; Construct a layer importance graph based on the change degree and importance to determine the importance of the model layer where each difference item to be extracted is located based on the layer importance graph; Obtain the gradient accumulation graph during the training process of the fine-tuned model to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; Determine the extraction priority of the difference items to be extracted based on the importance and the sensitivity, extract the difference items to be extracted according to the extraction priority to obtain the difference items to be transmitted, and construct multiple groups of differential data based on the difference items to be transmitted and the indexes corresponding to the difference items to be transmitted, where one group of differential data corresponds to one block; Encrypt and transmit the differential data to the client, and after decryption in the client, merge it with the second basic model to obtain the second fine-tuned model, where the second basic model is the same as the first basic model.

2. The model security protection method according to claim 1, wherein The step of encrypting and transmitting the differential data to the client includes: Generate an asymmetric encryption key pair, and send the public key in the asymmetric encryption key pair to the client; After the client obtains the public key based on the generated first random number, generate a second random number, and receive the third random number encrypted and returned by the client based on the public key, where the third random number is encrypted based on the fourth random number generated by the client; Decrypt the third random number to obtain the fourth random number, and calculate the symmetric encryption key based on the first random number, the second random number, and the fourth random number; After encrypting the differential data according to the symmetric encryption key, transmit the encrypted differential data to the client.

3. A model security protection method, characterized in that, Applied to the client side, the model security protection method includes: Receive the encrypted differential data sent by the server and decrypt the differential data. Among them, the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuned model, and the first fine-tuned model is obtained by training the first basic model. The calculation of the difference between the first basic model and the first fine-tuned model includes: performing block division on the first basic model and the first fine-tuned model synchronously, and calculating the difference metric of each weight parameter in the corresponding divided blocks based on a preset norm; taking the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be extracted; calculating the divergence of the parameter distribution in the model layer where each difference item to be extracted is located, and applying the structural similarity index to the divergence to evaluate the change degree and importance of the convolution kernel in the corresponding model layer; constructing a layer importance graph based on the change degree and importance to determine the importance of the model layer where each difference item to be extracted is located based on the layer importance graph; obtaining the gradient accumulation graph during the training process of the fine-tuned model to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; determining the extraction priority of the difference items to be extracted based on the importance and the sensitivity, and extracting the difference items to be extracted according to the extraction priority to obtain the difference items to be transmitted, and constructing multiple groups of differential data based on the difference items to be transmitted and the indexes corresponding to the difference items to be transmitted, where one group of differential data corresponds to one block; Obtain a second basic model and merge the decrypted differential data with the second basic model to obtain a second fine-tuned model.

4. The model security protection method according to claim 3, characterized in that Before the step of decrypting the differential data: Generate a first random number and call the server interface based on the first random number to obtain the public key. Among them, the public key is the public key in the asymmetric encryption key pair generated in the server; Obtain the second random number generated by the server and generate a fourth random number. Among them, the second random number is generated by the server after the client obtains the public key; Calculate the symmetric decryption key based on the first random number, the second random number and the fourth random number to perform the step of decrypting the differential data based on the symmetric decryption key.

5. The model security protection method according to claim 3, wherein The step of decrypting the differential data includes: Load the bytecode of the differential data into the memory and decrypt the bytecode; Call a preset class loader to convert the decrypted bytecode into each target class to use the each target class as the decoded differential data; After the step of merging the decrypted differential data with the second basic model to obtain a second fine-tuned model, it further includes: After detecting that the second fine-tuned model is used up, clear the preset class loader and the each target class in the memory.

6. The model security protection method according to claim 3, characterized in that, The step of merging the decrypted differential data with the second basic model to obtain a second fine-tuned model includes: Merge the decrypted differential data with the second basic model to obtain a candidate fine-tuned model; Perform performance evaluation on the candidate fine-tuning model, and in the case where the performance evaluation result does not meet the preset metrics, determine the parameters to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model; Calculate the performance gradient of the candidate fine-tuning model, and adjust the parameters to be optimized according to the performance gradient, so as to return to execute the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model; In the case where the performance evaluation result meets the preset metrics, use the candidate fine-tuning model as the second fine-tuning model.

7. The model security protection method according to claim 3, wherein, After the step of merging the decrypted differential data with the second base model to obtain the second fine-tuning model, the following steps are further included: Perform hash verification on the second fine-tuning model to obtain the first hash verification result; Obtain the second hash verification result of the merged model in the server, where the merged model in the server is obtained by merging the differential data and the first base model; Compare the first hash verification result with the second hash verification result to evaluate the merging effect of the second fine-tuning model according to the comparison result.

8. A model security protection system, characterized in that, The model security protection system includes: A server, configured to obtain a first base model and a corresponding first fine-tuning model, where the first fine-tuning model is obtained by training the first base model; perform block partitioning on the first base model and the first fine-tuning model synchronously, and calculate the difference metric of each weight parameter in the corresponding blocks after partitioning based on a preset norm; use the weight parameters with the difference metric greater than the preset difference threshold as the difference items to be extracted; calculate the divergence of the parameter distribution in the model layer where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the change degree and importance of the convolution kernels in the corresponding model layer; construct a layer importance graph based on the change degree and importance to determine the importance of the model layer where each difference item to be extracted is located based on the layer importance graph; obtain the gradient accumulation graph during the training process of the fine-tuning model to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; determine the extraction priority of the difference items to be extracted based on the importance and the sensitivity, and extract the difference items to be extracted according to the extraction priority to obtain the difference items to be transmitted, and construct multiple groups of differential data based on the difference items to be transmitted and the indexes corresponding to the difference items to be transmitted, where one group of differential data corresponds to one block; encrypt and transmit the differential data to the client, so as to merge the decrypted differential data with the second base model in the client to obtain the second fine-tuning model, where the second base model is the same as the first base model; A client, configured to receive the encrypted differential data sent by the server and decrypt the differential data, where the differential data is obtained by the server after calculating the difference between the first base model and the first fine-tuning model, and the first fine-tuning model is obtained by training the first base model; obtain the second base model, and merge the decrypted differential data with the second base model to obtain the second fine-tuning model.

Images