Model security protection method and system

Abstract

The invention discloses a model security protection method and system, and relates to the technical field of data security, and the method comprises the steps: obtaining a first basic model and a corresponding first fine tuning model, and the first fine tuning model is obtained by training the first basic model; calculating the difference between the first basic model and the first fine tuning model to obtain difference data; the differential data are encrypted and then transmitted to the client side, the differential data are decrypted in the client side and then combined with a second basic model, a second fine tuning model is obtained, and the second basic model is consistent with the first basic model. According to the method, only the basic models are deployed at the server and the client respectively, and a transmission mechanism of the differential model is adopted, so that the problems of security risk and transmission cost possibly caused by direct transmission of the whole fine tuning model are avoided, the model is updated and transmitted to the client efficiently on the premise of ensuring the model security, and the user experience is improved. And the model deployment and updating efficiency and security are improved.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a model security protection method and system. Background Art

[0002] Model security issues are becoming increasingly prominent in the current technological environment, especially when the model is transmitted between the server and the client, the risk of leakage becomes a major hidden danger. This risk not only threatens the confidentiality of the model, but may also affect the integrity and availability of the transmitted model. A malicious attacker may intercept model data through a man-in-the-middle attack to obtain sensitive information or tamper with the model, resulting in a decline in model performance or unforeseen consequences. For example, in the medical field, when a deep learning model is used for patient diagnosis, if the model is intercepted during transmission, the patient's sensitive information will be at risk of leakage, seriously infringing personal privacy; similarly, in the financial industry, if the model used to predict market trends is stolen by competitors, it may lead to unfair competition and cause economic losses to the company.

[0003] At present, in order to protect the security of the model during transmission, the industry has adopted a variety of measures. For example, by using symmetric or asymmetric encryption algorithms, the confidentiality of model data during transmission can be ensured; digital signatures and hash functions are also used to verify the integrity and authenticity of the model to prevent data tampering; access control and identity authentication mechanisms are used to restrict access to the model to authorized users, thereby improving security. However, these methods also have some limitations in practical applications: ordinary encryption technology still has the risk of being cracked, and encryption technology often increases computing overhead and affects the transmission efficiency of the model; although digital signatures and hash functions can verify the integrity of data, they cannot prevent data from being stolen; access control and identity authentication mechanisms require complex permission management and user authentication processes, which increases the complexity of the system. Therefore, although these methods can protect the security of the model to a certain extent, there is still room for improvement, and it is not possible to protect the security of the model while ensuring the performance of the model.

[0004] Therefore, how to reliably ensure the security and performance of the model at the same time is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide a model security protection method and system, aiming to solve the technical problem of how to reliably ensure the security and performance of the model at the same time.

[0006] To achieve the above purpose, the present application proposes a model security protection method, which is applied to the server. The model security protection method includes: Obtaining a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; Calculating the difference between the first basic model and the first fine-tuning model to obtain differential data; The differential data is encrypted and transmitted to the client, so as to be decrypted in the client and merged with the second basic model to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model.

[0007] In one embodiment, the step of calculating the difference between the first basic model and the first fine-tuning model to obtain differential data includes: Synchronously dividing the first basic model and the first fine-tuning model into blocks, and calculating the difference measure of each weight parameter in the corresponding blocks after the division based on a preset norm; The weight parameter of the difference metric greater than the preset difference threshold is used as the difference item to be transmitted, and multiple groups of differential data are constructed based on the difference item to be transmitted and the index corresponding to the difference item to be transmitted, wherein a group of differential data corresponds to one block.

[0008] In one embodiment, the step of using the weight parameter of the difference metric greater than a preset difference threshold as the difference item to be transmitted includes: The weight parameter of the difference measurement greater than the preset difference threshold is used as the difference item to be extracted; Calculate the divergence of the parameter distribution in the model level where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the degree of change and importance of the convolution kernel in the corresponding model level; Constructing a hierarchical importance graph based on the degree of change and the importance, so as to determine the importance of the model hierarchy where each difference item to be extracted is located based on the hierarchical importance graph; Obtaining a gradient accumulation graph of the fine-tuning model during the training process, so as to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; An extraction priority of the difference item to be extracted is determined based on the importance and the sensitivity, and the difference item to be extracted is extracted according to the extraction priority to obtain the difference item to be transmitted.

[0009] In one embodiment, the step of encrypting the differential data and transmitting it to the client comprises: Generate an asymmetric encryption key pair, and send the public key in the asymmetric encryption key pair to the client; After the client obtains the public key based on the generated first random number, a second random number is generated, and a third random number encrypted and returned by the client based on the public key is received, wherein the third random number is encrypted based on the fourth random number generated by the client; decrypting the third random number to obtain a fourth random number, and calculating a symmetric encryption key based on the first random number, the second random number and the fourth random number; After encrypting the differential data according to the symmetric encryption key, the encrypted differential data is transmitted to the client.

[0010] In addition, the present application also proposes a model security protection method, which is applied to the client, and the model security protection method includes: receiving encrypted differential data sent by the server, and decrypting the differential data, wherein the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuning model, and the first fine-tuning model is obtained by training the first basic model; A second basic model is obtained, and the decrypted differential data is merged with the second basic model to obtain a second fine-tuning model.

[0011] In one embodiment, the step of decrypting the differential data includes: Generate a first random number, and call a server interface based on the first random number to obtain a public key, wherein the public key is a public key in an asymmetric encryption key pair generated in the server; Obtaining a second random number generated by the server, and generating a fourth random number, wherein the second random number is generated by the server after the client obtains the public key; A symmetric decryption key is calculated based on the first random number, the second random number, and the fourth random number, so as to perform the step of decrypting the differential data based on the symmetric decryption key.

[0012] In one embodiment, the step of decrypting the differential data includes: Loading the bytecode of the differential data into a memory and decrypting the bytecode; Calling a preset class loader to convert the decrypted bytecode into each target class, so as to use each target class as the decoded differential data; After the step of merging the decrypted differential data with the second basic model to obtain the second fine-tuning model, the method further includes: After detecting that the use of the second fine-tuning model is completed, clearing the preset class loader and the target classes in the memory.

[0013] In one embodiment, the step of merging the decrypted differential data with the second basic model to obtain the second fine-tuning model includes: Merging the decrypted differential data with the second basic model to obtain a candidate fine-tuning model; Performing a performance evaluation on the candidate fine-tuning model, and determining a parameter to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model when the performance evaluation result does not meet the preset indicator; Calculating the performance gradient of the candidate fine-tuning model, and adjusting the parameter to be optimized according to the performance gradient, so as to return to the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model; When the performance evaluation result meets the preset indicator, the candidate fine-tuning model is used as the second fine-tuning model.

[0014] In one embodiment, after the step of merging the decrypted differential data with the second basic model to obtain the second fine-tuning model, the step further includes: Performing a hash check on the second fine-tuning model to obtain a first hash check result; Obtaining a second hash verification result of the merged model in the server, wherein the merged model is obtained by merging the differential data and the first basic model in the server; The first hash check result and the second hash check result are compared to evaluate the merging effect of the second fine-tuning model according to the comparison result.

[0015] In addition, to achieve the above purpose, the present application also proposes a model security protection system, which includes: A server is configured to obtain a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; calculate the difference between the first basic model and the first fine-tuning model to obtain differential data; encrypt the differential data and transmit it to the client, so that the differential data is decrypted in the client and merged with the second basic model to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model; The client is used to receive the encrypted differential data sent by the server and decrypt the differential data, wherein the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuning model, and the first fine-tuning model is obtained by training the first basic model; obtain the second basic model, and merge the decrypted differential data with the second basic model to obtain the second fine-tuning model.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: The present application first obtains a first basic model and a corresponding first fine-tuning model, and calculates the difference between the first basic model and the first fine-tuning model to obtain differential data, and adopts the method of model difference calculation to realize the identification of the difference between the model before and after fine-tuning, providing a basis for subsequent model transmission and merging; the differential data is then encrypted and transmitted to the client, and is merged with the second basic model after decryption in the client to obtain the second fine-tuning model. By adopting the method of encrypted transmission and model merging, the secure transmission of the differential model is realized, which effectively prevents the model from being illegally obtained or tampered with, and greatly improves the security and efficiency of model deployment.

[0017] In summary, this application avoids the security risks and transmission cost issues that may be caused by directly transmitting the entire fine-tuning model by deploying only the basic model on the server and client ends respectively, and adopts the differential model transmission mechanism. It achieves efficient transmission of model updates to the client while ensuring the security of the model, thereby quickly generating a fine-tuning model consistent with the server on the client, improving the efficiency and security of model deployment and updating. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flowchart of the first embodiment of the model security protection method of the present application is provided; Figure 2 A flowchart of the second embodiment of the model security protection method of the present application is provided; Figure 3 Schematic diagram of the process provided for the third embodiment of the model security protection method of this application Figure 4 A brief flowchart of the model security protection method provided in Example 3 of the present application; Figure 5 A schematic diagram of the accuracy maintenance process of the model security protection method provided in Example 3 of the present application; Figure 6 This is a schematic diagram of the module structure of the model security protection system of the embodiment of the present application.

[0021] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0023] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of the embodiment of the present application is: obtain a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; calculate the difference between the first basic model and the first fine-tuning model to obtain differential data; encrypt the differential data and transmit it to the client, so as to merge it with the second basic model after decryption in the client to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model.

[0025] At present, the industry has adopted a variety of methods to protect the security of the model during transmission. However, these methods also have some limitations in practical applications: ordinary encryption technology still has the risk of being cracked, and encryption technology often increases computing overhead, affecting the transmission efficiency of the model; although digital signatures and hash functions can verify the integrity of data, they cannot prevent data from being stolen; access control and identity authentication mechanisms require complex permission management and user authentication processes, which increases the complexity of the system. Therefore, although these methods can protect the security of the model to a certain extent, there is still room for improvement. It is not possible to protect the security of the model while ensuring the performance of the model at the same time. Therefore, how to reliably ensure the security and performance of the model at the same time is a problem that needs to be solved urgently.

[0026] The present application provides a solution, which avoids the security risks and transmission cost issues that may be caused by directly transmitting the entire fine-tuning model by deploying only the basic model on the server and the client respectively, and adopts the transmission mechanism of the differential model. It realizes the efficient transmission of model updates to the client while ensuring the security of the model, so that a fine-tuning model consistent with the server is quickly generated on the client, improving the efficiency and security of model deployment and updating.

[0027] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a model security protection system, etc. The following takes the model security protection system as an example to illustrate this embodiment and the following embodiments.

[0028] Based on this, the present application embodiment provides a model security protection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the model security protection method of the present application.

[0029] In this embodiment, the model security protection method is applied to the server, and the model security protection method includes steps S10 to S30: Step S10, obtaining a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; It should be noted that the first basic model refers to a pre-trained model that has not undergone any fine-tuning and only has certain general performance; the first fine-tuning model refers to a model that is adjusted for a specific task by further training and optimizing the first basic model.

[0030] Step S20, calculating the difference between the first basic model and the first fine-tuning model to obtain differential data; It should be noted that the differential data refers to the parameter difference between the first basic model and the first fine-tuning model, reflecting the changes in the model during the fine-tuning process.

[0031] It can be understood that in order to efficiently transmit the fine-tuned model, step S20 is performed, which can avoid the risk of model leakage and high bandwidth and time costs caused by directly transmitting the entire fine-tuned model, thereby providing an effective data foundation for the safe transmission of model data and the efficient transmission of small volumes.

[0032] In a feasible implementation, step S20 may include steps S21-S22: Step S21, synchronously dividing the first basic model and the first fine-tuning model into blocks, and calculating the difference measure of each weight parameter in the corresponding block after the division based on a preset norm; It should be noted that the preset norm refers to the mathematical norm used when calculating the difference in weight parameters, such as the L1 norm or the L2 norm, which is used to measure the size or length of the vector; the block refers to the independent parts into which the model is divided. Taking the YOLO model as an example, the model can be divided into five blocks, namely the input processing layer and the initial convolution layer, the backbone network convolution blocks 1-3, the backbone network convolution blocks 4-5, the feature pyramid network, and the detection head; the difference metric refers to the degree of difference in the weight parameters in the block calculated using the preset norm, which is used to determine whether the weight parameters of the block need to be transmitted.

[0033] It is understandable that, during the transmission of the differential model, if the volume of differential data is too large, once leaked, it will still pose a great threat to the security of the model. Therefore, step S21 is performed to provide effective data support for the subsequent block encryption of the differential data by processing the model in blocks.

[0034] Exemplarily, the first base model and the first fine-tuning model are divided into multiple blocks by network layer, and then for each corresponding block, a preset L2 norm is used to calculate the difference measure of weight parameters in the two models, that is, the Euclidean distance between the two vectors is calculated.

[0035] Step S22, taking the weight parameter of the difference metric greater than the preset difference threshold as the difference item to be transmitted, and constructing multiple groups of differential data based on the difference item to be transmitted and the index corresponding to the difference item to be transmitted, wherein one group of differential data corresponds to one block.

[0036] It should be noted that the difference items to be transmitted refer to the weight parameters whose difference measurements are greater than the preset difference thresholds. These parameters need to be transmitted to deploy or update the model; the index corresponding to the difference items to be transmitted refers to the position or index of the difference items to be transmitted in the model, which is used to correctly apply these difference items on the client.

[0037] It is understandable that due to the waste of bandwidth and time caused by transmitting small changes, step S22 is performed. By setting the difference threshold, the weight parameters whose difference metric exceeds the threshold can be screened out. The changes in these parameters have a significant impact on the model performance. Then, differential data is constructed based on these difference items to be transmitted and their corresponding indexes for efficient transmission and update, while also avoiding the inefficiency of model deployment or update caused by the transmission of unnecessary parameters, thereby achieving a more efficient and accurate model transmission and update process.

[0038] Exemplarily, a difference threshold is set, and the weight parameters whose difference metric exceeds the threshold are used as difference items to be transmitted. Then, based on these difference items to be transmitted and their indexes in the model, multiple sets of difference data are constructed, each set of difference data corresponds to a block, and contains the weight parameters and their indexes to be transmitted in the block.

[0039] In this implementation, by synchronizing block division and difference metric calculation, combined with difference threshold screening and differential data construction, high bandwidth occupancy and long transmission delays caused by full transmission of the entire model are avoided, while bandwidth waste and time consumption caused by transmitting small changes are also avoided. In addition, by processing the model in blocks, a reliable foundation is provided for subsequent block encryption of differential data, thereby improving the efficiency and flexibility of model deployment or updating, while ensuring the security of model transmission.

[0040] Step S30, encrypting the differential data and transmitting it to the client, so that it is decrypted in the client and merged with the second basic model to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model.

[0041] It should be noted that the second basic model refers to the basic model deployed on the client, which is consistent with the first basic model and has not been fine-tuned; the second fine-tuned model refers to the fine-tuned model obtained on the client by merging the decrypted differential data with the second basic model, which is consistent with the first fine-tuned model or basically reaches the performance of the first fine-tuned model.

[0042] It is understandable that since it is necessary to protect the transmission security of the model and to quickly generate a fine-tuning model consistent with the server on the client, step S30 is performed. By encrypting and transmitting a small amount of differential data, the model can be prevented from being leaked or tampered with. By transmitting differential data, the model can be effectively deployed on the client, avoiding the risk of unauthorized access or malicious attacks on the model during transmission, as well as the high computing cost and time consumption caused by retraining the model on the client. This ensures the security and efficiency of model transmission while achieving rapid deployment and updating of the model, thereby improving the availability and real-time performance of the model.

[0043] In a feasible implementation manner, the step of encrypting the differential data and transmitting it to the client in step S30 may include steps S31 to S34: Step S31, generating an asymmetric encryption key pair, and sending the public key in the asymmetric encryption key pair to the client; Exemplarily, the server processes the login request sent by the client, generates an RSA key pair (K1, K2), and sends the public key K1 of the key pair to the client in response to the login request.

[0044] Step S32, after the client obtains the public key based on the generated first random number, a second random number is generated, and a third random number encrypted and returned by the client based on the public key is received, wherein the third random number is encrypted based on the fourth random number generated by the client; For example, the client generates a first random number R1, and uses R1 to call the server interface to obtain the RSA public key K1, after which the server generates a second random number R2 and returns the random number R2 and the public key K1. The client then generates a fourth random number R4, and uses the RSA public key K1 to encrypt R4 to obtain a third random number R3 and sends it to the server, and the server receives R3 returned by the client.

[0045] Step S33, decrypting the third random number to obtain a fourth random number, and calculating a symmetric encryption key based on the first random number, the second random number and the fourth random number; Exemplarily, the server decrypts the third random number R3 based on the private key to obtain a fourth random number R4, and uses R1, R2, and R4 to calculate a symmetric encryption AES key K3 based on an algorithm agreed upon with the client.

[0046] Step S34: After encrypting the differential data according to the symmetric encryption key, the encrypted differential data is transmitted to the client.

[0047] It is understandable that since traditional encryption methods only use a single encryption mechanism, it is often difficult to ensure the high security requirements of data. Therefore, step S34 is performed by adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption. That is, a symmetric encryption and decryption algorithm is used to encrypt and decrypt the model, and an asymmetric encryption and decryption algorithm is used to encrypt and decrypt the model key, thereby increasing the cost of decrypting the model after it is leaked or directly obtaining the decrypted model from the disk.

[0048] In this embodiment, by adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, the security risk of the model being decrypted after leakage or the decrypted model being directly obtained from the disk is greatly avoided, and the model is further effectively prevented from being illegally obtained or tampered with.

[0049] This embodiment provides a model security protection method, which deploys only the basic model on the server and the client respectively, and adopts the transmission mechanism of the differential model, thereby avoiding the security risks and transmission cost problems that may be caused by directly transmitting the entire fine-tuning model. It achieves efficient transmission of model updates to the client while ensuring the security of the model, thereby quickly generating a fine-tuning model consistent with the server on the client, thereby improving the efficiency and security of model deployment and updating.

[0050] In a feasible implementation manner, the step of using the weight parameter of the difference metric greater than the preset difference threshold as the difference item to be transmitted in step S22 may include steps S221 to S225: Step S221, taking the weight parameter of the difference measurement greater than the preset difference threshold as the difference item to be extracted; It should be noted that the difference items to be extracted refer to the items whose weight parameters have changed significantly during the model fine-tuning process, and these changes may have an important impact on the model performance.

[0051] Exemplarily, after fine-tuning the model, a difference metric is calculated for each weight parameter, such as using an absolute difference or a relative rate of change. A preset difference threshold is set, and weight parameters whose difference metric exceeds the threshold are marked as difference items to be extracted. This method ensures that only weight parameters that have a significant impact on model performance are transmitted, reducing unnecessary data transmission.

[0052] Step S222, calculating the divergence of the parameter distribution in the model level where each difference item to be extracted is located, and applying the structural similarity index to the divergence to evaluate the degree of change and importance of the convolution kernel in the corresponding model level; It should be noted that the degree of change of the convolution kernel refers to the amount of change in the weight parameters of the convolution layer (i.e., the convolution kernel) during the model training process, and the importance refers to the impact of these changes on the model performance.

[0053] It is understandable that since it is often difficult to determine which parameters need to be transmitted or it is inaccurate to determine which parameters need to be transmitted during the model transmission process, step S222 is performed to evaluate the degree and importance of change of the convolution kernel by calculating the divergence of the parameter distribution and applying the structural similarity index, thereby avoiding blindly transmitting all changing weight parameters and improving the pertinence and efficiency of the transmission.

[0054] For example, for each model level where the difference item to be extracted is located, the divergence of the parameter distribution is calculated, such as using the Kullback-Leibler divergence or Jensen-Shannon divergence, and then the structural similarity index (SSIM) is applied to evaluate the degree and importance of the change of the convolution kernel. This helps to identify which convolution kernels have changed significantly during the fine-tuning process and have an important impact on the model performance.

[0055] Step S223, constructing a hierarchical importance graph based on the degree of change and the importance, so as to determine the importance of the model hierarchy where each difference item to be extracted is located based on the hierarchical importance graph; It should be noted that the layer importance diagram is a visualization tool used to show the importance of each layer of the model in the feature extraction and decision-making process, helping to understand the model structure and optimize the model; the importance of the model layer refers to the contribution of different layers in the model to the final output results. The layers with higher importance have a greater impact on the model performance.

[0056] It can be understood that in order to clarify the contribution of each level to the model performance, step S223 is performed. By constructing a level importance graph, excessive attention to unimportant levels can be avoided, thereby optimizing the model structure and performance.

[0057] Exemplarily, based on the calculated divergence and structural similarity index, a hierarchical importance graph is constructed. This graph intuitively shows the importance of each model level in the feature extraction and decision-making process. By analyzing the hierarchical importance graph, it is possible to determine which levels contribute most to the model performance, thereby optimizing the model structure and transmission strategy.

[0058] Step S224, obtaining a gradient accumulation graph of the fine-tuning model during the training process, so as to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; It should be noted that the gradient accumulation graph records the gradient changes of each weight parameter during the model training process, which is used to analyze the sensitivity of the weight parameter to the loss function; the sensitivity of the difference item to be extracted refers to the sensitivity of the difference item to be extracted to the change of model performance. The difference item with high sensitivity has a greater impact on the model performance.

[0059] It can be understood that, since the sensitivity of the difference item has a certain correlation with the model performance, step S224 is performed to determine the sensitivity of the difference item to be extracted by analyzing the gradient accumulation graph, which can avoid excessive transmission of insensitive difference items and improve the transmission efficiency and model performance.

[0060] For example, during the model training process, the gradient change of each difference item to be extracted is recorded to form a gradient accumulation graph. By analyzing the gradient accumulation graph, the sensitivity of the difference item to be extracted to the loss function can be determined. Difference items with high sensitivity have a greater impact on model performance and should be transmitted first.

[0061] Step S225 , determining an extraction priority of the difference items to be extracted based on the importance and the sensitivity, and extracting the difference items to be extracted according to the extraction priority to obtain difference items to be transmitted.

[0062] It should be noted that the extraction priority is to determine the priority of the difference item in the transmission process according to factors such as the importance and sensitivity of the difference item, and give priority to transmitting the difference item that has a greater impact on the model performance.

[0063] It is understandable that since there is a sequence in the transmission of model parameters, step S225 is performed to determine the extraction priority by comprehensively considering factors such as the importance and sensitivity of the difference items. This can avoid the degradation of model performance caused by unreasonable transmission order and improve the efficiency and accuracy of model deployment and updating.

[0064] For example, considering the importance of the hierarchy and the sensitivity of the gradient, an extraction priority is determined for each difference item to be extracted. Difference items with high priorities have a greater impact on model performance and should be transmitted first. The difference items are sorted according to the extraction priority, and the difference items with high priorities are selected for transmission to obtain the difference items to be transmitted. This method improves the efficiency and accuracy of model updates and reduces the waste of transmission bandwidth and time.

[0065] In this embodiment, a method for optimizing the weight transmission of a neural network model is proposed. First, the weight parameters whose difference metrics exceed the threshold are selected as the difference items to be extracted, ensuring that only the weight changes that have a significant impact on the model performance are transmitted; then, for each model level where the difference items to be extracted are located, the divergence of the parameter distribution is calculated to quantify the degree of change in the parameter distribution; then, the structural similarity index is applied to evaluate the degree of change and importance of the convolution kernel in the corresponding model level, and the convolution kernel that has an important impact on feature extraction and model decision is identified; based on the results of the divergence and SSIM, a hierarchical importance graph is constructed to intuitively display the contribution of each model level to the overall model performance, and at the same time, by obtaining the gradient accumulation graph of the fine-tuning model during the training process, the cumulative sum of the absolute value of the gradient of each difference item to be extracted is calculated to determine its sensitivity, that is, the degree of its impact on the model performance; finally, considering the hierarchical importance and gradient sensitivity comprehensively, an extraction priority is set for each difference item to be extracted, and the difference items are sorted according to the priority, and the difference items with high priority are transmitted first, so as to obtain the difference items to be transmitted. This method effectively avoids the high bandwidth and time cost problems caused by transmitting all weight parameters, achieves efficient updating and optimization of the model, and maintains the performance improvement of the model.

[0066] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 In this embodiment, the model security protection method is applied to the client, and the model security protection method includes steps S01-S02: Step S01, receiving encrypted differential data sent by a server, and decrypting the differential data, wherein the differential data is obtained by the server after calculating the difference between a first basic model and a first fine-tuning model, and the first fine-tuning model is obtained by training the first basic model; It is understandable that in order to ensure the security of differential data during transmission and prevent data leakage and tampering, step S01 is performed to avoid the high bandwidth consumption and data security risks caused by the full transmission model. By transmitting encrypted differential data, the amount of transmitted data is greatly reduced, the transmission efficiency is improved, and data security is guaranteed at the same time.

[0067] Step S02: obtain a second basic model, and merge the decrypted differential data with the second basic model to obtain a second fine-tuning model.

[0068] It can be understood that in order to quickly apply the server-side model update based on the local second basic model to generate a second fine-tuning model, step S02 is performed. This can avoid the high bandwidth and long waiting time required to retransmit the entire model and the risk of security leakage. By merging differential data and efficiently deploying or updating the model, while maintaining the model performance improvement, it significantly reduces bandwidth usage and transmission time, and improves the security of model transmission.

[0069] In a feasible implementation manner, the step of decrypting the differential data in step S01 may include steps S011 to S012: Step S011, loading the bytecode of the differential data into a memory, and decrypting the bytecode; It should be noted that the decryption process can use a symmetric decryption key calculated based on a random number, and is performed in an independent thread different from the main thread to reduce the risk of memory leakage in the main thread.

[0070] Step S012, calling a preset class loader to convert the decrypted bytecode into each target class, so as to use the each target class as the decoded differential data.

[0071] It should be noted that the preset class loader refers to the class loader used to dynamically load the decrypted bytecode. This class loader has specific permissions and functions, such as class isolation and unloading, that is, when certain classes are no longer needed, the reference to the entire custom class loader can be discarded; on-demand loading and unloading, that is, it can be designed to load the decrypted model only when needed and clean up immediately after use; memory usage control, that is, the life cycle of the class can be more accurately controlled to avoid long-term memory occupation; the target class refers to the class converted from the decrypted differential data, which represents the updated part of the model.

[0072] It can be understood that if the decrypted bytecode is converted into an executable class object, it can be applied to model deployment or update. Therefore, performing step S012 can avoid the problem that the ordinary class loader may not be able to safely load the dynamically generated class, resulting in security risks. By loading the target class with the preset class loader, the security and correctness of the class loading are ensured, which facilitates the dynamic update of the model.

[0073] After step S02, the model security protection method may further include step S013: Step S013: after detecting that the use of the second fine-tuning model is completed, clearing the preset class loader and the target classes in the memory.

[0074] It is understandable that since the classes loaded by the standard class loader (such as the application class loader AppClassLoader) will be kept in the memory until the class loader is garbage collected, performing step S013 can avoid the situation where the dynamically generated classes are not cleared in time, thereby occupying a large amount of memory and causing system performance to degrade. By clearing the preset class loader and target class in time, memory resources are released and the security and stability of the system are improved.

[0075] For example, after the model is used, System.gc() is called immediately and the object is manually set to null. A custom TensorBuffer class is implemented. After use, the cleanup method is actively called to force the finalizer to be triggered after key operations to ensure that resources are released in time. At the same time, a memory leak prevention daemon thread is developed to regularly check and clean up unreleased model resources.

[0076] In this implementation, by securely loading and processing encrypted differential data in memory, dynamically loading the target class using a preset class loader, and promptly clearing resources in memory after use, efficient and secure updates of the model are achieved, avoiding problems such as data leakage and memory leakage, thereby ensuring the performance and security of the system.

[0077] In a feasible implementation manner, the step of merging the decrypted differential data with the second basic model in step S02 to obtain the second fine-tuning model may include steps S021 to S024: Step S021, merging the decrypted differential data with the second basic model to obtain a candidate fine-tuning model; It should be noted that the candidate fine-tuning model refers to the preliminary fine-tuning model generated after the client merges the differential data, which is used for subsequent performance evaluation and optimization.

[0078] Exemplarily, after receiving the encrypted differential data, the client uses a pre-agreed key to decrypt it and obtain the original differential data, and then loads the differential data into the second basic model, and merges the differential data into the model parameters through the model's parameter update mechanism (such as the back propagation algorithm) to generate a candidate fine-tuning model. The specific merging process can use the SIMD instruction set (NEON / SSE) to accelerate the weight merging process, and implement multi-threaded parallel merging of large network layers to improve processing efficiency, while using GPU to accelerate large matrix operations (applicable to devices that support GPU). In addition, a memory pre-allocation strategy is developed to reduce memory allocation overhead during the merging process.

[0079] Step S022, performing performance evaluation on the candidate fine-tuning model, and determining the parameters to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model when the performance evaluation result does not meet the preset index; It should be noted that the sensitivity of each parameter in the candidate fine-tuning model refers to the degree of influence of each parameter on the model performance, which is used to identify the parameters that need to be optimized.

[0080] It is understandable that since there may be a problem of model accuracy loss in the differential data merging process, performing step S022 can avoid the problem that the failure to perform performance evaluation may lead to poor model performance and inability to meet actual application needs. Performance evaluation can be used to identify the parameters that need to be optimized, providing a reliable direction for model optimization.

[0081] Exemplarily, the client uses a set of test data sets to evaluate the performance of the candidate fine-tuning model, and the evaluation indicators may include accuracy, recall, F1 value, etc. If the evaluation result does not meet the preset performance indicators, the client will analyze the sensitivity of each parameter in the model, and determine which parameters have the greatest impact on the model performance by calculating the gradient or using the feature importance analysis method, thereby determining the parameters to be optimized.

[0082] Step S023, calculating the performance gradient of the candidate fine-tuning model, and adjusting the parameter to be optimized according to the performance gradient, so as to return to the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model; It should be noted that the performance gradient of the candidate fine-tuning model refers to the derivative of the model performance with respect to each parameter, which is used to guide the direction and magnitude of parameter adjustment.

[0083] It is understandable that the failure to perform gradient calculation and parameter adjustment may result in the inability to improve model performance, so step S023 is performed to accurately optimize model performance through gradient calculation and parameter adjustment, thereby improving the merging accuracy and efficiency of the model.

[0084] For example, the client uses the back-propagation algorithm to calculate the performance gradient of the candidate fine-tuning model on the test data set, that is, the derivative of the model performance with respect to each parameter, and then uses the optimization algorithm (such as stochastic gradient descent, Adam, etc.) to adjust the parameters to be optimized based on this gradient information. The performance of the adjusted model is evaluated again, and this process may require multiple iterations until the model performance reaches the preset indicator.

[0085] Step S024: when the performance evaluation result meets the preset indicator, the candidate fine-tuning model is used as the second fine-tuning model.

[0086] For example, when the performance evaluation result of the candidate fine-tuning model on the test data set reaches or exceeds the preset performance index, the client confirms that the model optimization is complete and officially uses the candidate fine-tuning model as the second fine-tuning model. This model will be used for subsequent reasoning tasks or further training.

[0087] In this implementation, the candidate fine-tuning models are generated by merging differential data, and performance evaluation and parameter optimization are performed. Finally, the fine-tuning model that meets the requirements is confirmed, thereby achieving efficient optimization and verification of the model, avoiding problems such as poor performance and improper parameter adjustment due to loss of merging accuracy, and ensuring the accuracy and efficiency of the model.

[0088] In this embodiment, by deploying only the basic model on the server and client ends respectively and adopting the differential model transmission mechanism, the security risks and transmission cost issues that may be caused by directly transmitting the entire fine-tuning model are avoided. Under the premise of ensuring the security of the model, the model update is efficiently transmitted to the client, so that a fine-tuning model consistent with the server is quickly generated on the client, thereby improving the efficiency and security of model deployment and updating.

[0089] In a feasible implementation manner, before the step of decrypting the differential data in step S01, steps S100 to S300 may also be included: Step S100, generating a first random number, and calling a server interface based on the first random number to obtain a public key, wherein the public key is a public key in an asymmetric encryption key pair generated in the server; Exemplarily, the client generates a first random number R1, and uses R1 to call a server interface to obtain a public key K1 in an asymmetric encryption key pair (K1, K2) generated by the server in response to a login request.

[0090] Step S200, obtaining a second random number generated by the server, and generating a fourth random number, wherein the second random number is generated by the server after the client obtains the public key; Exemplarily, the server generates a second random number R2 after the client obtains the public key K1, the client obtains R2, and generates a fourth random number R4, wherein R4 can be encrypted using K1 to obtain a third random number R3, which is then sent to the server, so that after receiving R3, the server uses the private key to decrypt R4, thereby calculating the symmetric encryption key K3 using R1, R2, and R4 on the server.

[0091] Step S300, calculating a symmetric decryption key based on the first random number, the second random number and the fourth random number, so as to perform the step of decrypting the differential data based on the symmetric decryption key.

[0092] It is understandable that since traditional encryption methods only use a single encryption mechanism, it is often difficult to ensure the high security requirements of data. Therefore, step S300 is performed by adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, that is, using a symmetric encryption and decryption algorithm to encrypt and decrypt the model, and using an asymmetric encryption and decryption algorithm to encrypt and decrypt the model key, thereby increasing the cost of decrypting the model after it is leaked or directly obtaining the decrypted model from the disk.

[0093] Exemplarily, an AES symmetric decryption key K3 is calculated based on the first random number R1, the second random number R2, and the fourth random number R4, and K3 is stored in the Keystore for subsequent differential model decryption.

[0094] In this embodiment, by adopting a key negotiation mechanism that combines asymmetric encryption and symmetric encryption, the security risk of the model being decrypted after leakage or the decrypted model being directly obtained from the disk is greatly avoided, and the model is further effectively prevented from being illegally obtained or tampered with.

[0095] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned second embodiment can refer to the above introduction, and will not be repeated in the following. Figure 3 , after step S02, steps S03 to S05 may also be included: Step S03, performing a hash check on the second fine-tuning model to obtain a first hash check result; It should be noted that the first hash verification result is a unique identifier obtained by performing a hash operation on the second fine-tuning model generated by the client.

[0096] It is understandable that in order to ensure that the second fine-tuning model generated by the client is consistent with the merged model of the server, step S03 is performed to prevent the differential model from being maliciously modified or damaged during transmission, thereby ensuring the integrity and security of the model and verifying the correctness of the model.

[0097] Exemplarily, the client uses the SHA-256 hash algorithm to operate on the second fine-tuning model to generate a first hash verification result to ensure the integrity of the model data.

[0098] Step S04, obtaining a second hash verification result of the merged model in the server, wherein the merged model is obtained by merging the differential data and the first basic model in the server; It should be noted that the second hash verification result is a unique identifier obtained by performing a hash operation on the merged model obtained by merging the differential data and the first basic model in the server.

[0099] It can be understood that in order to compare with the hash verification result of the client and confirm that the merged model of the server is consistent with the client's model, step S04 is performed, which can avoid inconsistency between the server model and the client model, resulting in errors in subsequent applications, thereby ensuring the consistency of the server and client models and ensuring the correctness of the application.

[0100] Exemplarily, the client sends a request to the server to obtain the SHA-256 hash value of the merged model. After receiving the request, the server performs a hash operation on the merged model, generates a second hash verification result, and sends it back to the client.

[0101] Step S05: compare the first hash check result and the second hash check result to evaluate the merging effect of the second fine-tuning model according to the comparison result.

[0102] It should be noted that the comparison result is the result of comparing the first hash check result and the second hash check result. If the two are the same, it means that the model merging is correct; if they are different, it means that there is a problem with the model merging.

[0103] In addition, it should be noted that after evaluating the merging effect, if the merging effect does not reach the preset effect, a retransmission request can be sent to the server through the client to request the server to retransmit the differential data, and after sending the retransmission request for multiple consecutive times, a differential data reconstruction request can be sent to the server through the client to enable the server to recalculate the difference between the first basic model and the first fine-tuning model, obtain the differential data, and return the differential data.

[0104] It can be understood that by comparing the hash values, it is possible to confirm whether the second fine-tuning model of the client is consistent with the merged model of the server and evaluate the merging effect. Therefore, performing step S05 can avoid the problem of being unable to identify incorrect model merging, resulting in performance degradation or application errors, thereby ensuring the correctness and effectiveness of the model merging and improving the reliability and performance of the model applied in the client.

[0105] Exemplarily, the client compares the first hash check result with the second hash check result. If the two are consistent, it means that the second fine-tuning model is consistent with the merged model of the server, and the merge effect is good; if the two are inconsistent, it indicates that there may be errors or data tampering in the merge process, which requires further investigation and processing.

[0106] In this implementation, by synchronously merging the differential data and the basic model in the client and the server and using a hash check method, the integrity and correctness of the model during the transmission and merging process can be ensured, data tampering and damage can be avoided, and the reliability and performance of the model application in the client can be improved.

[0107] For example, to help understand the implementation process of the model security protection method obtained by combining the above-mentioned embodiment 1 and embodiment 2, please refer to Figure 4 , Figure 4 A brief flowchart of a model security protection method is provided, specifically: In terms of model security protection, it is important to consider model transmission security, APP security, and model file security, which is especially important on Android devices (the following are some security measures used on Android devices): For transmission security, in order to increase the cost of obtaining models or keys during the transmission process, a set of internal transmission protocols is implemented for model transmission. At the same time, the SSL certificate public key is fixed in the APP, and third-party certificates are not trusted. If it is a third-party certificate, the connection is interrupted to prevent packet capture. The public key needs to be updated on time according to the length of the certificate validity period.

[0108] For APP security, in order to increase the dynamic debugging cost, you can obfuscate the APP code, delete the debugging information, symbol table, etc. (ProGuard), and at the same time, add a shell to the APP, as well as ROOT device detection, virtual machine detection, etc. For ROOT devices, you can load a poor model. In addition, introduce anti-debugging (Frida, Xpose).

[0109] For model security, in order to reduce the cost of decrypting the model after it is leaked or directly obtaining the decrypted model from the disk, you can use AES to encrypt and decrypt the model, use RSA to encrypt and decrypt the model key, and store the key in the keyStore. In addition, ensure that the decrypted model can only be in memory and cannot appear in any accessible area. Uninstall it in time after use, and build the pre-trained model into the APP. The server extracts the weight layer that is different from the pre-trained model after fine-tuning the model, and the client compares and merges them to ensure that the data is not lost.

[0110] The system architecture in the figure includes four parts, namely the key management service, the server and the terminal (i.e., the client), and the HTTP transmission channel between the server and the client. The key management service stores and returns the key to the server and the terminal. The server first performs differential extraction on the original model (i.e., the first basic model and the corresponding first fine-tuning model) to obtain differential data, and then encrypts and transmits the differential data based on the key provided by the key management service. The terminal decrypts the model and key of the transmitted data, and after decryption, it is differentially merged with the second basic model to obtain the second fine-tuning model.

[0111] For further information, please refer to Figure 5, the figure shows the accuracy preservation implementation process when merging differential data with the basic model in the client. First, the model merging and accuracy loss analysis are performed, in which preliminary quantization analysis, model loss analysis and sensitivity parameter identification are performed after the differential model is merged. Then, the difference weights are dequantized in the quantization-aware merging and high-precision merging is performed, and then quantization awareness and parameter adjustment are introduced for re-quantization. Then, the key layer insertion point is identified, and a calibration layer with dynamically updated parameters is created, and corresponding feature statistics and distribution alignment are performed to calibrate and compensate the merged model based on the calibration layer. Finally, the model accuracy is iteratively corrected, including first evaluating the performance of the model, and then calculating the model performance gradient and parameter sensitivity analysis when the evaluation does not meet the standard, and adjusting the adaptive parameters. The performance evaluation is performed again based on the adjusted parameters until the evaluation meets the standard, and the calibration layer is frozen to obtain a second fine-tuning model with excellent performance.

[0112] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the model security protection method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0113] This application also provides a model safety protection system, please refer to Figure 6 , the model safety protection system comprises: The server 10 is used to obtain a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; calculate the difference between the first basic model and the first fine-tuning model to obtain differential data; encrypt the differential data and transmit it to the client 20, so that the differential data is decrypted in the client 20 and merged with the second basic model to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model; The client 20 is used to receive the encrypted differential data sent by the server 10 and decrypt the differential data, wherein the differential data is obtained by the server 10 after calculating the difference between the first basic model and the first fine-tuning model, and the first fine-tuning model is obtained by training the first basic model; obtain the second basic model, and merge the decrypted differential data with the second basic model to obtain the second fine-tuning model.

[0114] Optionally, the server 10 is further used for: Synchronously dividing the first basic model and the first fine-tuning model into blocks, and calculating the difference measure of each weight parameter in the corresponding blocks after the division based on a preset norm; The weight parameter of the difference metric greater than the preset difference threshold is used as the difference item to be transmitted, and multiple groups of differential data are constructed based on the difference item to be transmitted and the index corresponding to the difference item to be transmitted, wherein a group of differential data corresponds to one block.

[0115] Optionally, the server 10 is further used for: The weight parameter of the difference measurement greater than the preset difference threshold is used as the difference item to be extracted; Calculate the divergence of the parameter distribution in the model level where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the degree of change and importance of the convolution kernel in the corresponding model level; Constructing a hierarchical importance graph based on the degree of change and the importance, so as to determine the importance of the model hierarchy where each difference item to be extracted is located based on the hierarchical importance graph; Obtaining a gradient accumulation graph of the fine-tuning model during the training process, so as to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; An extraction priority of the difference item to be extracted is determined based on the importance and the sensitivity, and the difference item to be extracted is extracted according to the extraction priority to obtain the difference item to be transmitted.

[0116] Optionally, the server 10 is further used for: Generate an asymmetric encryption key pair, and send the public key in the asymmetric encryption key pair to the client 20; After the client 20 obtains the public key based on the generated first random number, it generates a second random number, and receives a third random number returned by the client 20 based on encryption of the public key, wherein the third random number is encrypted based on the fourth random number generated by the client 20; decrypting the third random number to obtain a fourth random number, and calculating a symmetric encryption key based on the first random number, the second random number and the fourth random number; After the differential data is encrypted according to the symmetric encryption key, the encrypted differential data is transmitted to the client 20 .

[0117] Optionally, the client 20 is further used for: Generate a first random number, and call the server 10 interface to obtain a public key based on the first random number, wherein the public key is a public key in an asymmetric encryption key pair generated in the server 10; Obtaining a second random number generated by the server 10, and generating a fourth random number, wherein the second random number is generated by the server 10 after the client 20 obtains the public key; A symmetric decryption key is calculated based on the first random number, the second random number and the fourth random number, so as to perform the step of decrypting the differential data in the memory based on the symmetric decryption key.

[0118] Optionally, the client 20 is further used for: Loading the bytecode of the differential data into a memory and decrypting the bytecode; Calling a preset class loader to convert the decrypted bytecode into each target class, so as to use each target class as the decoded differential data; After the step of merging the decrypted differential data with the second basic model to obtain the second fine-tuning model, the method further includes: After detecting that the use of the second fine-tuning model is completed, clearing the preset class loader and the target classes in the memory.

[0119] Optionally, the client 20 is further used for: Merging the decrypted differential data with the second basic model to obtain a candidate fine-tuning model; Performing a performance evaluation on the candidate fine-tuning model, and determining a parameter to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model when the performance evaluation result does not meet the preset indicator; Calculating the performance gradient of the candidate fine-tuning model, and adjusting the parameter to be optimized according to the performance gradient, so as to return to the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model; When the performance evaluation result meets the preset indicator, the candidate fine-tuning model is used as the second fine-tuning model.

[0120] Optionally, the client 20 is further used for: Performing a hash check on the second fine-tuning model to obtain a first hash check result; Obtaining a second hash verification result of the merged model in the server 10, wherein the merged model is obtained by merging the differential data and the first basic model in the server 10; The first hash check result and the second hash check result are compared to evaluate the merging effect of the second fine-tuning model according to the comparison result.

[0121] The model security protection system provided by the present application adopts the model security protection method in the above-mentioned embodiment, which can solve the technical problem of how to reliably ensure the security and performance of the model at the same time. Compared with the prior art, the beneficial effects of the model security protection system provided by the present application are the same as the beneficial effects of the model security protection method provided by the above-mentioned embodiment, and other technical features in the model security protection system are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0122] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A model security protection method, characterized in that: Applied to the server, the model security protection method includes: Obtaining a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; Calculating the difference between the first basic model and the first fine-tuning model to obtain differential data; The differential data is encrypted and transmitted to the client, so as to be decrypted in the client and merged with the second basic model to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model.

2. The model security protection method according to claim 1, characterized in that: The step of calculating the difference between the first basic model and the first fine-tuning model to obtain differential data comprises: Synchronously dividing the first basic model and the first fine-tuning model into blocks, and calculating the difference measure of each weight parameter in the corresponding blocks after the division based on a preset norm; The weight parameter of the difference metric greater than the preset difference threshold is used as the difference item to be transmitted, and multiple groups of differential data are constructed based on the difference item to be transmitted and the index corresponding to the difference item to be transmitted, wherein a group of differential data corresponds to one block.

3. The model security protection method according to claim 2, characterized in that: The step of using the weight parameter of the difference metric greater than the preset difference threshold as the difference item to be transmitted includes: The weight parameter of the difference measurement greater than the preset difference threshold is used as the difference item to be extracted; Calculate the divergence of the parameter distribution in the model level where each difference item to be extracted is located, and apply the structural similarity index to the divergence to evaluate the degree of change and importance of the convolution kernel in the corresponding model level; Constructing a hierarchical importance graph based on the degree of change and the importance, so as to determine the importance of the model hierarchy where each difference item to be extracted is located based on the hierarchical importance graph; Obtaining a gradient accumulation graph of the fine-tuning model during the training process, so as to determine the sensitivity of each difference item to be extracted based on the gradient accumulation graph; An extraction priority of the difference items to be extracted is determined based on the importance and the sensitivity, and the difference items to be extracted are extracted according to the extraction priority to obtain the difference items to be transmitted.

4. The model security protection method according to claim 1, characterized in that: The step of encrypting the differential data and transmitting it to the client comprises: Generate an asymmetric encryption key pair, and send the public key in the asymmetric encryption key pair to the client; After the client obtains the public key based on the generated first random number, a second random number is generated, and a third random number encrypted and returned by the client based on the public key is received, wherein the third random number is encrypted based on the fourth random number generated by the client; decrypting the third random number to obtain a fourth random number, and calculating a symmetric encryption key based on the first random number, the second random number and the fourth random number; After encrypting the differential data according to the symmetric encryption key, the encrypted differential data is transmitted to the client.

5. A model security protection method, characterized in that: Applied to the client, the model security protection method includes: receiving encrypted differential data sent by the server, and decrypting the differential data, wherein the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuning model, and the first fine-tuning model is obtained by training the first basic model; A second basic model is obtained, and the decrypted differential data is merged with the second basic model to obtain a second fine-tuning model.

6. The model security protection method according to claim 5, characterized in that: The step of decrypting the differential data includes: Generate a first random number, and call a server interface based on the first random number to obtain a public key, wherein the public key is a public key in an asymmetric encryption key pair generated in the server; Obtaining a second random number generated by the server, and generating a fourth random number, wherein the second random number is generated by the server after the client obtains the public key; A symmetric decryption key is calculated based on the first random number, the second random number, and the fourth random number, so as to perform the step of decrypting the differential data based on the symmetric decryption key.

7. The model security protection method according to claim 5, characterized in that: The step of decrypting the differential data comprises: Loading the bytecode of the differential data into a memory and decrypting the bytecode; Calling a preset class loader to convert the decrypted bytecode into each target class, so as to use each target class as the decoded differential data; After the step of merging the decrypted differential data with the second basic model to obtain the second fine-tuning model, the method further includes: After detecting that the use of the second fine-tuning model is completed, clearing the preset class loader and the target classes in the memory.

8. The model security protection method according to claim 5, characterized in that: The step of merging the decrypted differential data with the second basic model to obtain a second fine-tuning model includes: Merging the decrypted differential data with the second basic model to obtain a candidate fine-tuning model; Performing a performance evaluation on the candidate fine-tuning model, and determining a parameter to be optimized based on the sensitivity of each parameter in the candidate fine-tuning model when the performance evaluation result does not meet the preset indicator; Calculating the performance gradient of the candidate fine-tuning model, and adjusting the parameter to be optimized according to the performance gradient, so as to return to the step of performing performance evaluation on the candidate fine-tuning model based on the adjusted candidate fine-tuning model; When the performance evaluation result meets the preset indicator, the candidate fine-tuning model is used as the second fine-tuning model.

9. The model security protection method according to claim 5, characterized in that: After the step of merging the decrypted differential data with the second basic model to obtain the second fine-tuning model, the method further includes: Performing a hash check on the second fine-tuning model to obtain a first hash check result; Obtaining a second hash verification result of the merged model in the server, wherein the merged model is obtained by merging the differential data and the first basic model in the server; The first hash check result and the second hash check result are compared to evaluate the merging effect of the second fine-tuning model according to the comparison result.

10. A model safety protection system, characterized in that: The model safety protection system comprises: A server is configured to obtain a first basic model and a corresponding first fine-tuning model, wherein the first fine-tuning model is obtained by training the first basic model; calculate the difference between the first basic model and the first fine-tuning model to obtain differential data; encrypt the differential data and transmit it to the client, so that the differential data is decrypted in the client and merged with the second basic model to obtain a second fine-tuning model, wherein the second basic model is consistent with the first basic model; The client is used to receive the encrypted differential data sent by the server and decrypt the differential data, wherein the differential data is obtained by the server after calculating the difference between the first basic model and the first fine-tuning model, and the first fine-tuning model is obtained by training the first basic model; obtain the second basic model, and merge the decrypted differential data with the second basic model to obtain the second fine-tuning model.

Images