Large model data protection method, system, device, storage medium and program product

Abstract

The application discloses a large model data protection method, system, device, storage medium and program product, wherein the method comprises the following steps: sending a first adapter and a second adapter obtained by performing feature extraction on a large model to a second node, wherein the large model comprises the first adapter, a main model and the second adapter; inputting first feature data received from the second node to the large model for model training, outputting second feature data, and sending the second feature data to the second node, wherein the first feature data is obtained by processing sample business data features according to the second adapter by the second node; receiving a target gradient value sent by the second node, adjusting the large model according to the target gradient value until the large model is adjusted completely, and the target gradient value is determined by the second node according to the first adapter, the second feature data and a business label. The application can protect the data privacy of the model demander while not damaging the model intellectual property right of the large model owner.

Description

Technical Field

[0001] This invention relates to the field of model data processing technology, and in particular to a method, system, device, storage medium, and program product for protecting large model data. Background Technology

[0002] The emergence of large-scale foundational models (hereinafter referred to as large models, such as language models like GPT-2-XL and OPT-1.3B) has brought about technological changes and opportunities to various industries. However, training large models is costly, and most companies cannot afford the cost of training a complete large model.

[0003] Currently, large model owners provide general-purpose large models to enterprises that need them. These enterprises then fine-tune the large models using their own domain-specific private data, thereby reducing the cost of using the large models. However, because large models typically require enormous computing power and data, resulting in high training costs, the trained large models are generally proprietary and not open-source. If large model owners provide their models to enterprises that need them for fine-tuning, it could easily infringe on the intellectual property rights of the large model owners. On the other hand, if enterprises that need large models send their domain-specific private data to the large model owners for fine-tuning, it could easily lead to privacy data leaks. Therefore, how to protect the data privacy of the model-demanding parties (such as enterprises that need large models) without infringing on the intellectual property rights of the large model owners has become an urgent technical problem to be solved. Summary of the Invention

[0004] The main objective of this invention is to provide a method, system, device, storage medium, and program product for protecting large model data, aiming to solve the technical problem of protecting the data privacy of the model user without infringing on the intellectual property rights of the large model owner.

[0005] To achieve the above objectives, this invention provides a large model data protection method. The method is applied to a first node connected to a second node, where a large model is deployed. The second node possesses sample business data features and business tags. The method includes the following steps:

[0006] The first adapter and the second adapter obtained by feature extraction of the large model are sent to the second node, wherein the large model includes the first adapter, the main model and the second adapter;

[0007] The system receives first representation data sent by the second node, inputs the first representation data into the large model for model training, outputs second representation data, and sends the second representation data to the second node. The first representation data is data obtained by the second node through processing based on the features of the second adapter and the sample service data.

[0008] The system receives the target gradient value sent by the second node and adjusts the large model based on the target gradient value until the large model adjustment is completed. The target gradient value is determined by the second node based on the first adapter, the second representation data, and the service label.

[0009] Optionally, after the step of adjusting the large model based on the target gradient value until the large model adjustment is complete, the method includes:

[0010] If the predictable representation data is received from the second node, the predictable representation data is input into the adjusted large model for model training, and the predicted representation data is output. The predicted representation data is then input into the second node, wherein the second node performs format conversion processing on the predicted representation data according to the first adapter to obtain the actual predicted data.

[0011] Optionally, the step of inputting the first representation data into the large model for model training and outputting the second representation data includes:

[0012] The large model is compressed to obtain a compressed model. The first representation data is input into the compressed model for model training, and the second representation data is output.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a large model data protection method, which is applied to a second node connected to a first node. A large model is deployed on the first node, and the second node possesses sample business data features and business tags. The method includes the following steps:

[0014] The system receives the first adapter and the second adapter sent by the first node, performs format transformation processing on the sample service data features according to the second adapter to obtain the first characterization data, and sends the first characterization data to the first node.

[0015] The system receives second representation data sent by the first node, performs format transformation on the second representation data according to the first adapter, and obtains a first prediction result, wherein the second representation data is the prediction result of the large model in the first node predicting the first representation data.

[0016] Based on the first prediction result and the business label, a target gradient value is determined, and the target gradient value is sent to the first node, wherein the first node adjusts the large model based on the target gradient value.

[0017] Optionally, after sending the target gradient value to the first node, the method further includes:

[0018] After the large model adjustment in the first node is completed, if there are sample business data features to be predicted, the second adapter is used to perform format transformation on the sample business data features to be predicted to obtain the representation data to be predicted, and the representation data to be predicted is sent to the first node.

[0019] The system receives the prediction representation data sent by the first node, performs format transformation processing on the prediction representation data according to the first adapter, and obtains the actual prediction data.

[0020] Optionally, the step of performing format transformation processing on the sample service data features according to the second adapter includes:

[0021] A dataset is constructed by selecting a predetermined number of sample business data features from multiple sample business data features with business labels;

[0022] For at least one sample service data in the dataset, perform format transformation processing on the features of the sample service data according to the second adapter.

[0023] Optionally, the step of performing format transformation processing on the sample service data features according to the second adapter to obtain the first representation data includes:

[0024] Determine the original format corresponding to the sample business data features, and the model format corresponding to the large model;

[0025] The second adapter converts the format corresponding to the sample service data features from the original format to the model format to obtain privacy representation data.

[0026] The privacy representation data is input into a preset first protection model for encryption processing, and the first representation data is output.

[0027] Optionally, the step of performing format transformation processing on the second representation data according to the first adapter to obtain the first prediction result includes:

[0028] The second characterization data is input into a preset second protection model for decryption processing, and the third characterization data is output.

[0029] The first prediction result is obtained by converting the format corresponding to the third representation data from the model format to the original format based on the first adapter.

[0030] Optionally, the step of determining a target gradient value based on the first prediction result and the service tag, and sending the target gradient value to the first node, includes:

[0031] Based on the first prediction result and the business label, a gradient value is calculated to obtain a first gradient value;

[0032] The first adapter is adjusted according to the first gradient value to obtain a target gradient value, and the target gradient value is sent to the first node. The first node adjusts the large model according to the target gradient value to obtain a second gradient value, and the second gradient value is sent to the second node.

[0033] After the step of sending the target gradient value to the first node, the following is included:

[0034] The system receives the second gradient value sent by the first node, adjusts the parameters of the second adapter based on the second gradient value, obtains new sample service data features, and re-executes the step of performing format transformation processing on the sample service data features based on the second adapter based on the new sample service data features, until the large model in the first node is detected to have converged or the preset number of training iterations has been reached.

[0035] Furthermore, to achieve the above objectives, the present invention also provides a large model data protection system, which includes a first node and a second node interconnected. The first node deploys a large model, and the second node possesses sample business data features and business tags.

[0036] The first node is used to send the first adapter and the second adapter obtained by feature extraction of the large model to the second node, wherein the large model includes the first adapter, the main model and the second adapter;

[0037] The second node is used to receive the first adapter and the second adapter sent by the first node, perform format transformation processing on the sample service data features according to the second adapter to obtain the first characterization data, and send the first characterization data to the first node.

[0038] The first node is configured to receive first representation data sent by the second node, input the first representation data into the large model for model training, output second representation data, and send the second representation data to the second node.

[0039] The second node is configured to receive the second representation data sent by the first node, perform format transformation processing on the second representation data according to the first adapter to obtain a first prediction result, determine the target gradient value according to the first prediction result and the service tag, and send the target gradient value to the first node.

[0040] The first node is used to receive the target gradient value sent by the second node, and adjust the large model according to the target gradient value until the large model adjustment is completed.

[0041] In addition, to achieve the above objectives, the present invention also provides a large model data protection device, which includes: a memory, a processor, and a large model data protection program stored in the memory and executable on the processor. When the large model data protection program is executed by the processor, it implements the steps of the large model data protection method as described above.

[0042] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a large model data protection program, which, when executed by a processor, implements the steps of the large model data protection method as described above.

[0043] Furthermore, to achieve the above objectives, the present invention also proposes a computer program product, comprising a computer program that, when executed by a processor, implements the steps of the large model data protection method described above.

[0044] In this embodiment of the invention, by deploying a large model on a first node, when the second node possesses sample business data features and business tags, the first node sends a first adapter and a second adapter for feature extraction of the large model to the second node. Since the large model includes a main model, the first adapter, and the second adapter, the main part of the large model, i.e., the main model, remains in the first node and is not sent to the second node. Therefore, the intellectual property rights of the large model owner in the first node are protected, preventing the second node from infringing upon the intellectual property rights of the large model after becoming aware of its overall architecture parameters.

[0045] Furthermore, during large-scale model adjustments, the second node transforms the sample business data features using the second adapter to obtain first representation data, which is then sent to the first node. The first node processes this first representation data using the large model to obtain second representation data, which is then sent back to the second node. The second node receives the target gradient value determined by the first adapter, the second representation data, and the business label, and adjusts the large model accordingly until the adjustment is complete. This avoids the risk of training sample data leakage from the large model owner in the first node directly accessing the training sample data of the model requester in the second node. Since the second node processes the sample business data features of the training sample data using the second adapter to obtain the second representation data, the large model in the first node only processes the first representation data and is unaware of the specific training sample data in the second node. This also protects data privacy in the second node and reduces the risk of data leakage. In short, it achieves both the protection of the large model owner's intellectual property rights and the protection of the model requester's data privacy. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention;

[0047] Figure 2 This is a flowchart illustrating the first embodiment of the large model data protection method of the present invention;

[0048] Figure 3 This is a schematic diagram of the large model after it has been divided in the large model data protection method of the present invention;

[0049] Figure 4 This is a schematic diagram illustrating the model training and adjustment of a large model in the large model data protection method of the present invention;

[0050] Figure 5 This is a flowchart illustrating the second embodiment of the large model data protection method of the present invention;

[0051] Figure 6 This is a schematic diagram illustrating model prediction based on a large model in the large model data protection method of the present invention;

[0052] Figure 7 This is a schematic diagram illustrating the model training and adjustment of the compressed model in the large model data protection method of the present invention;

[0053] Figure 8 This is a flowchart illustrating the third embodiment of the large model data protection method of the present invention;

[0054] Figure 9 This is a flowchart illustrating the fourth embodiment of the large model data protection method of the present invention;

[0055] Figure 10 This is a schematic diagram of the large model data protection system of the present invention.

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0059] It should be noted that the large model data protection device in the embodiments of the present invention can be a smartphone, a personal computer, a server, or other devices, and no specific limitations are imposed here.

[0060] like Figure 1 As shown, the large model data protection device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on large model data protection devices and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0062] like Figure 1 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a large-scale data protection program. The operating system is a program that manages and controls the device's hardware and software resources, supporting the operation of the large-scale data protection program and other software or programs. Figure 1In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for the server to establish a communication connection; and the processor 1001 can be used to call the large model data protection program stored in the memory 1005 and execute the operations described in the following embodiments of the large model data protection method of the present invention.

[0063] Based on the above structure, various embodiments of the large model data protection method are proposed.

[0064] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the large model data protection method of the present invention.

[0065] This invention provides an embodiment of a large model data protection method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order. In this embodiment, the large model data protection method can be applied to a first node, which is connected to a second node. A large model is deployed on the first node, and the second node possesses sample business data features and business tags. Optionally, the first node can be the node where the large model owner is located, and the second node can be the node where the model requester is located. A first device can be set up in the first node, and the large model is stored in the first device. A second device can be set up in the second node, and the sample business data features and business tags are stored in the second device. The first device can be the large model owner, and the second device can be the model requester. Both the first device and the second device can be at least one of devices such as smartphones, personal computers, servers, and terminals; this embodiment does not impose any limitations.

[0066] Optionally, a first-party feature extractor can be deployed in the first node, and a second-party feature extractor can be deployed in the second node. The first-party feature extractor can be a processor in the first device, used to extract model features from the large model, such as separating the large model into an adapter and a main model, extracting the adapter, and storing the main model in memory. The second-party feature extractor can be a processor in the second device, used to extract sample features from the second device.

[0067] Large models can be neural network models with many parameters, or large foundational models. Large foundational models exhibit superior performance in various tasks, including natural language processing, computer vision, and speech recognition. Examples include GPT-2 (a pre-trained language model), OPT (Open Pre-trained Transformer), BLOOM (BigScience), CLIP (Contrastive Language-Image Pre-Training), and EVA (a distributed database system). For instance, a large model can be a transformer-based model.

[0068] In one optional implementation scenario, the owner of the large model in the first node, A, splits the large model into three parts: upper, middle, and lower. The upper and lower parts are adapters, and the middle part is the main model of the large model. Furthermore, the main model's data structure is far more complex than the adapter's data structure, and its data volume is greater. For example, ... Figure 3 As shown, the large model is divided into three parts: upper, middle, and lower. The upper and lower parts are adapters, namely adapter 1 and adapter 2, and the middle part is the main large model. Then, the owner of the large model, A, compresses the main large model to obtain a compressed model. Owner A sends the compressed model, adapter 1, and adapter 2 to the data owner, B, in the second node. Data owner B uses local data to fine-tune the compressed model, adapter 1, and adapter 2. After fine-tuning, data owner B returns adapter 1 and adapter 2 to owner A. Then, data owner B remotely calls the fine-tuned large model in owner A. Adapter 1 can be the first adapter, and adapter 2 can be the second adapter.

[0069] Since data owner A sends the compressed model to data owner B, the intellectual property rights of the model can be protected to some extent. Furthermore, data owner B does not need to transmit data to the large model owner A, thus protecting data privacy during model training. In addition, data owner B performs fine-tuning on an adapter, which is a small model, thus improving fine-tuning efficiency, i.e., improving the efficiency of adjusting the large model.

[0070] In this embodiment, in order to better protect the intellectual property rights of the large model owner A, the large model is split into an adapter and a main model in the first node, and only the adapter is sent to the second node.

[0071] Optionally, the large model data protection method in this embodiment can be applied to the fine-tuning of large models in a federated learning scenario. In this scenario, at least two nodes are set up, such as a first node where the large model owner A is located and a second node where the data owner B is located.

[0072] Data owner B wants to use its private data to fine-tune the large model owned by model owner A. However, data owner B does not want to send its private data to model owner A for model training. Model owner A is also unwilling to send its large model to B for fine-tuning. Therefore, we can assume that data owner B possesses private data (X, Y), i.e., sample business data and business labels, and model owner A possesses the large model. Model owner A wants to lease the capabilities of its developed large model to the requesting party (i.e., the data owner) and profit from it, but is unwilling to send the large model to data owner B for fine-tuning; data owner B wants to fine-tune the large model to adapt it to its own business, but is unwilling to send its business data to model owner A for fine-tuning and inference. Therefore, in the first node, the large model owner A splits the large model into an adapter and a main model. The adapter includes a first adapter above the main model and a second adapter below the main model. The first and second adapters are then sent to the second node, while the main model remains in the first node. Then, the data owner B sends sample business data to the second adapter for transformation, then back to the large model in the first node for prediction, and finally back to the first adapter in the second node for transformation and recovery, obtaining the prediction result. A loss function value is generated based on the business label and the prediction result, and the large model in large model owner A is updated based on the loss function value. This allows large model owner A to lease its large model to data owner B, protecting both the model owner A's copyright and the data owner B's privacy. For example, if large model owner A is a natural language processing company that has trained a large language model, and another bank (i.e., data owner B) wants to use this large language model from the natural language processing company and fine-tune it based on its own business data into a professional large language model suitable for the bank's auditing business. However, since audit business data is considered private and cannot be shared externally, and due to model copyright concerns, the natural language processing company also does not wish to directly send the large language model to the bank. Therefore, the large model owner, A, can send the first and second adapters of the large language model to the data owner, B. Data owner B determines the business labels corresponding to the audit business data to be trained and the available data (i.e., sample business data). The sample business data is then sent to the second adapter for transformation, then to the large model in the first node for prediction, and finally back to the first adapter in the second node for transformation and recovery to obtain the prediction result. A loss function value is generated based on the business labels and the prediction result, and the large semantic model in large model owner A is updated based on the loss function value. This allows large model owner A to provide the large language model to data owner B for use.

[0073] Furthermore, in this embodiment, the large model data protection method includes:

[0074] Step S10: The first adapter and the second adapter obtained by feature extraction of the large model are sent to the second node, wherein the large model includes the first adapter, the main model and the second adapter;

[0075] In this embodiment, the adapter is a lightweight model component that can fine-tune only a small portion of parameters to adapt to a specific task while keeping most of the pre-trained model parameters unchanged. The adapter includes a first adapter and a second adapter. The first adapter consists of the top layers of the large model, for example, the top 1-2 layers of the large model. The second adapter consists of the bottom layers of the large model, for example, the bottom 1-2 layers of the large model. The main model can be any model in the large model other than the first and second adapters. Furthermore, the first adapter, the second adapter, and the main model can all be neural networks or functions.

[0076] Optionally, in the first node, the large model can be mapped to obtain a mapped model that is completely consistent with the structure of the large model. Then, the mapped model can be split into three parts. For example, first determine the model architecture hierarchy of the mapped model, split the top two layers of the mapped model to obtain the first adapter, split the bottom two layers of the mapped model to obtain the second adapter, and then use the remaining model architecture as the main model. Optionally, the mapped model can be split using a first-party feature extractor, and the split first and second adapters can be extracted.

[0077] Optionally, the first node can store the main model in a preset storage area and compress the first and second adapters before sending them to the second node. The second node receives the compressed first and second adapters from the first node, decompresses them to obtain the decompressed first and second adapters. Then, the second node performs fine-tuning based on the first and second adapters.

[0078] Optionally, in the first node, after performing the mapping operation on the large model, the large model can be directly split to obtain the first adapter, the second adapter, and the main model, and the first adapter and the second adapter can be sent to the second node.

[0079] Step S20: Receive the first representation data sent by the second node, input the first representation data into the large model for model training, output the second representation data, and send the second representation data to the second node. The first representation data is data obtained by the second node based on the features of the second adapter and the sample service data.

[0080] Optionally, after the first node sends the first adapter and the second adapter to the second node, the second device in the second node, upon receiving the first adapter and the second adapter, determines the training sample data to be used for model training, including sample business data features and business tags. The sample business data features may be privacy data stored in the second node, such as private file data or private image data. Optionally, a portion of the sample business data features with business tags can be extracted from multiple data stored in the second node using a second-party feature extractor as training sample data. Optionally, the second node processes the first sample data features according to the second adapter, such as format conversion processing, or format conversion processing and encryption processing, to obtain first representation data, which is then sent to the first node.

[0081] Optionally, after receiving the first representation data sent by the second node, the first node can store the first representation data in a preset folder and extract the stored first representation data from the preset folder to input into the large model for model training. Alternatively, the first representation data can be directly input into the large model for model training to obtain the output result, which is then sent to the second node as the second representation data. Optionally, the model training of the large model can include model training of any of the following: natural language processing, computer vision, speech recognition, etc.

[0082] Step S30: Receive the target gradient value sent by the second node, and adjust the large model according to the target gradient value until the large model adjustment is completed, wherein the target gradient value is the gradient value determined by the second node based on the first adapter, the second representation data and the service label.

[0083] Optionally, after receiving the second representation data, the second device in the second node determines the target gradient value based on the first adapter and the second representation data, and sends it to the first node. The first device in the first node receives the target gradient value sent by the second node. Optionally, in the second node, the second representation data can be decrypted using a protection model, such as a differential privacy model. Then, the decrypted second representation data is transformed using the first adapter. The transformed second representation data and the business label are input into a pre-set loss function to calculate the gradient value. The model parameters in the first adapter and the protection model are then updated sequentially according to the calculated gradient value. At this time, the gradient value is also updated to the target gradient value, and this target gradient value is sent to the first node.

[0084] Optionally, after receiving the target gradient value, the first device in the first node can adjust the model parameters of the large model based on the target gradient value to achieve the adjustment of the large model. Furthermore, when adjusting the parameters of the large model, the target gradient value will also be updated accordingly. At this time, the updated target gradient value can be sent to the second node to update the parameters of the second adapter and the protection model connected to the second adapter in the second node.

[0085] Optionally, the model parameters of the large model can be adjusted by subtracting a certain percentage of the target gradient value from the model parameters. Then, it is checked whether the large model meets the preset training termination conditions. These conditions include reaching a preset number of training iterations, the large model achieving optimal performance, or model convergence.

[0086] If the preset training termination condition is not met, then new first representation data in the second node is determined, and steps S20-S30 above are executed based on the new first representation data until the large model meets the training termination condition.

[0087] If the conditions are met, training the large model stops. At this point, the second node can use the large model from the first node to process the privacy data. For example, it extracts data features from the actual privacy data using a second-party feature extractor to obtain the first data feature. This first data feature is then processed by the second adapter and encrypted using a protection model to obtain the actual representation data. This actual representation data is sent to the first node, where the large model trains on it to obtain the training result. This training result is then sent to the second node, where the protection model decrypts the training result and processes it using the first adapter to obtain the final output result. This output result is the result of processing the first data feature using the large model.

[0088] In addition, to help understand the fine-tuning of the large model in this embodiment, examples are given below.

[0089] For example, such as Figure 4As shown, in the first node, model owner A extracts features from the large model to obtain adapter 1 and adapter 2. After sending adapter 1 and adapter 3 to data owner B in the second node, protection model 1 can be set for adapter 1 and protection model 2 can be set for adapter 2 in the second node. The second node also stores sample business data features and sample labels, and selects a small subset of sample business data features x and sample labels y for model training. Sample business data feature x is input into adapter 2 so that adapter 2 can convert the format of sample business data feature x into a format that the large model can accept and recognize, such as encoding the sample business data feature x to obtain representation z. Then, representation z is encrypted through protection model 2 to obtain the first representation data z. s The first representation data z s Send to the first node. In the first node, the first representation data z is... s The data is input into a large model for training, resulting in the second representation data q. s The second representation data q s Send to the second node. In the second node, the second representation data q is... s The input is fed into protection model 1 for decryption processing to obtain the representation v. s And will characterize v s The input is fed into adapter 1 for format conversion to obtain the prediction result. Prediction results The sample label y is input into the loss function to calculate the gradient value. Then, based on the calculated gradient value, adapter 1 and protection model 1 are updated sequentially to obtain the target gradient value. This target gradient value is sent to the first node. The large model in the first node updates itself based on the target gradient value, obtaining a gradient value, which is then sent to the second node to update protection model 2 and adapter 2. The first node also performs a check on the large model. If the large model has not reached the preset number of training iterations or has converged, it can continue training based on a small subset of sample business data features x and sample labels y from the second node until the model converges or reaches the preset number of training iterations.

[0090] Optionally, this embodiment uses federated learning to fine-tune the model for both the large model owner and the data owner. During federated fine-tuning and federated inference, neither the large model owner's model nor the data owner's model leaves their local machine. This protects the intellectual property rights of the large model and data privacy.

[0091] In this embodiment, by deploying a large model on the first node, when the second node possesses sample business data features and business tags, the first node sends a first adapter and a second adapter for feature extraction from the large model to the second node. Since the large model includes a main model, the first adapter, and the second adapter, the main part of the large model remains in the first node and is not sent to the second node. This protects the intellectual property rights of the large model owner in the first node and prevents the second node from infringing upon the intellectual property rights of the large model after becoming aware of its overall architecture parameters.

[0092] Furthermore, during large-scale model adjustments, the second node transforms the sample business data features using the second adapter to obtain first representation data, which is then sent to the first node. The first node processes this first representation data using the large model to obtain second representation data, which is then sent back to the second node. The second node receives the target gradient value determined by the first adapter, the second representation data, and the business label, and adjusts the large model accordingly until the adjustment is complete. This avoids the risk of training sample data leakage from the large model owner in the first node directly accessing the training sample data of the model requester in the second node. Since the second node processes the sample business data features of the training sample data using the second adapter to obtain the second representation data, the large model in the first node only processes the first representation data and is unaware of the specific training sample data in the second node. This also protects data privacy in the second node and reduces the risk of data leakage. In short, it achieves both the protection of the large model owner's intellectual property rights and the protection of the model requester's data privacy.

[0093] Based on the first embodiment described above, a second embodiment of the large model data protection method of the present invention is proposed. In this embodiment, referring to... Figure 5 Step S30, after adjusting the large model based on the target gradient value until the large model adjustment is complete, includes:

[0094] Step S40: If the predictable representation data is received from the second node, the predictable representation data is input into the adjusted large model for model training, and the predicted representation data is output. The predicted representation data is then input into the second node, wherein the second node performs format conversion processing on the predicted representation data according to the first adapter to obtain the actual predicted data.

[0095] In this embodiment, after the large model in the first node has been trained, if the second node needs to use the large model in the first node for related operations, such as speech recognition or language processing, the second node can process the business data to be predicted through the second adapter, or through the second adapter and the protection model (such as format conversion or encryption), to obtain the representation data to be predicted, and then send it to the first node.

[0096] Optionally, after receiving the data to be predicted, the first node can input the data to be predicted into a large model for model training to obtain a prediction result corresponding to the data to be predicted, and then use this prediction data as the prediction data to input into the second node. In the second node, the prediction data can be decrypted by protecting the model and then converted into actual prediction data that the second node can accurately recognize through the first adapter.

[0097] For example, refer to Figure 6 During the model training phase, model owner A in the first node has sent adapter 1 and adapter 2 to data owner B in the second node. The second node also has protection model 1 and protection model 2. After training and adjusting the large model in the first node, the second node can use the large model in the first node for corresponding processing. Therefore, the sample business data feature x to be predicted can be input into adapter 2, so that adapter 2 can convert the format of the sample business data feature x2 to be predicted into a format that the large model can accept and recognize. For example, the sample business data feature x2 can be encoded to obtain representation data, and then the representation data can be encrypted through protection model 2 to obtain the representation data to be predicted, which is then sent to the first node. In the first node, the representation data to be predicted is input into the large model for prediction to obtain predicted representation data, which is then sent to the second node. In the second node, the predicted representation data is input into protection model 1 for decryption and then input into adapter 1 for format conversion to obtain the prediction result y2.

[0098] In this embodiment, the first node inputs the predictable representation data sent by the second node into the large model for model training, outputs the predicted representation data, and sends it to the second node. The second node performs format conversion processing on the predicted representation data according to the first adapter to obtain the actual predicted data, thereby ensuring the validity of the actual predicted data.

[0099] Further, the step of performing prediction processing on the first representation data based on the large model to obtain the second representation data includes:

[0100] Step a: Compress the large model to obtain a compressed model, input the first representation data into the compressed model for model training, and output the second representation data.

[0101] In this embodiment, to improve the efficiency of adjusting large models, the large models can be compressed. This compression can involve shrinking the large model into a smaller one, such as by performing knowledge distillation on the large model. The compressed model is then used as the compressed model.

[0102] Optionally, in the first node, the first representation data sent by the second node is input into the compressed model for model training, and the second representation data is output. The second representation data is then sent to the second node. Optionally, after completing the training of the compressed model, the compressed model can be restored to obtain a restored and adjusted large model.

[0103] For example, such as Figure 7 As shown, in the first node, model owner A extracts features from the large model to obtain adapter 1 and adapter 2. After sending adapter 1 and adapter 3 to data owner B in the second node, protection model 1 can be set for adapter 1 and protection model 2 can be set for adapter 2 in the second node. The large model in the first node can also be compressed to obtain a compressed model. Optionally, the second node also stores sample business data features and sample labels, and selects a small subset of sample business data features x and sample labels y for model training. Sample business data features x are input into adapter 2 so that adapter 2 can convert the format of sample business data features x into a format that the compressed model can accept and recognize, such as encoding sample business data features x to obtain representation z. Then, representation z is encrypted through protection model 2 to obtain the first representation data z. s The first representation data z s Send to the first node. In the first node, the first representation data z is... s The data is input into the compressed model for model training, resulting in the second representation data q. s The second representation data q s Send to the second node. In the second node, the second representation data q is... s The input is fed into protection model 1 for decryption processing to obtain the representation v. s And will characterize v s The input is fed into adapter 1 for format conversion to obtain the prediction result. Prediction results The sample label y is input into the loss function to calculate the gradient value. Then, based on the calculated gradient value, adapter 1 and protection model 1 are updated sequentially to obtain the target gradient value. This target gradient value is sent to the first node, where the compressed model updates itself based on the target gradient value, obtaining a gradient value. This gradient value is then sent to the second node to update protection model 2 and adapter 2. Furthermore, the first node will inspect the compressed model. If the compressed model has not reached the preset number of training iterations or has not converged, it can continue training using a small subset of sample business data features x and sample labels y from the second node until the model converges or reaches the preset number of training iterations.

[0104] In this embodiment, a compressed model is obtained by compressing the large model, and then the first representation data is input into the compressed model for model training to obtain the second representation data. This improves model efficiency compared to directly training the model using the large model.

[0105] Furthermore, based on the first or second embodiment described above, a third embodiment of the large model data protection method of the present invention is proposed. In this embodiment, reference is made to... Figure 8 The large model data protection method is applied to a second node connected to the first node. A large model is deployed on the first node, and the second node possesses sample business data features and business tags. The large model data protection method includes the following steps:

[0106] Step S100: Receive the first adapter and the second adapter sent by the first node; perform format transformation processing on the sample service data features according to the second adapter to obtain the first representation data; and send the first representation data to the first node.

[0107] In this embodiment, both the first node and the second node can be the same as those in the first or second embodiment described above. The first node contains a first device, and the second node contains a second device. The large model is stored in the first device, while the sample business data features and business tags are stored in the second device. The first device can be the owner of the large model, and the second device can be the model requester or the data owner. Optionally, in the second node, the data owner stores multiple pieces of private data, such as private image data, text data, and video data.

[0108] Optionally, a first-party feature extractor can be set in the first node, and a second-party feature extractor can be set in the second node. The first-party and second-party feature extractors can be processors of the devices in their respective nodes. Alternatively, the first-party and second-party feature extractors can also be functional modules in a computer that have feature separation and extraction capabilities. No restrictions are imposed here.

[0109] Optionally, the second device in the second node receives an adapter sent by the first node, such as a first adapter and a second adapter.

[0110] Optionally, since the large model consists of a first adapter at the upper layer, a main model in the middle layer, and a second adapter at the lower layer, and the main architecture of the large model is in the main model, the first-party feature extractor in the first node separates the large model or the mapping model corresponding to the large model and performs feature extraction to obtain the first adapter, the main model, and the second adapter. The first and second adapters are then sent to the second node, while the main model remains in the first node. Therefore, even after the second node receives the first and second adapters sent by the first node, it is unaware of the specific architecture of the large model in the first node, thus protecting the intellectual property rights of the large model.

[0111] Optionally, in the second node, pre-stored sample business data features with business tags can be identified. Optionally, the sample business data features stored in the second node can be extracted using a second-party feature extractor. The sample business data features are input into a second adapter for format transformation processing, such as encoding format transformation processing, encryption processing, etc. The first representation data is output and sent to the first node.

[0112] Step S200: Receive the second representation data sent by the first node, perform format transformation processing on the second representation data according to the first adapter, and obtain a first prediction result, wherein the second representation data is the prediction result of the large model in the first node predicting the first representation data;

[0113] Optionally, the large model in the first node receives the first representation data and performs training processing on the first representation data, such as prediction training processing, classification training processing, etc., and performs different processing according to different tasks to obtain the second representation data, and sends the second representation data to the second node.

[0114] Optionally, after the large model in the first node is compressed into a compressed model, the first representation data can be trained using the compressed model to obtain the second representation data, and then the second representation data can be sent to the second node.

[0115] Optionally, the second node receives the second representation data sent by the first node, and then performs data conversion and decryption processing on the second representation data through the first adapter to obtain the prediction result of the large model, and uses it as the first prediction result.

[0116] Step S300: Determine the target gradient value based on the first prediction result and the service label, and send the target gradient value to the first node, wherein the first node adjusts the large model based on the target gradient value.

[0117] Optionally, the business label corresponding to the features of the sample business data and the first prediction result are input together into a pre-set loss function to calculate the gradient value.

[0118] Optionally, the second node determines the business label corresponding to the sample business data features. For example, if there are sample business data features A1, A2, and A3, the business label corresponding to A1 is B1, the business label corresponding to A2 is B2, and the business label corresponding to A3 is B3. When the sample business data input to the second adapter is A1, the business label B1 and the first prediction result are input to a preset loss function to calculate the gradient value.

[0119] Optionally, a second-party feature extractor can be used to extract business labels corresponding to the features of the sample business data. Then, the second sample data features and the prediction results are input into a pre-set loss function to calculate the gradient value. This gradient value is then sent to the first adapter for parameter adjustment, yielding the adjusted gradient value. Based on this adjusted gradient value, a target gradient value is determined and sent to the first node to adjust the model parameters of the large model in the first node. Optionally, the loss function can be any language task loss function, such as the error rate for predicting the next word. Gradient calculation can involve calculating the first derivative of the loss function with respect to the model parameters.

[0120] Optionally, the first node adjusts the model parameters of the large model, which can be done by subtracting a certain percentage of the gradient values ​​from the model parameters. It then checks whether the large model meets the preset training termination conditions. These conditions include reaching a preset number of training iterations or the large model achieving optimal performance. If not, the second node determines new first representation data and continues to acquire new gradient values ​​based on this new first representation data. In other words, the second node acquires new sample business data features and continues processing these features using the second adapter to obtain the first representation data. This process continues until the large model meets the training termination conditions. If it does, model training stops. At this point, the second node can use the large model to predict privacy data.

[0121] In this embodiment, by receiving only the first adapter and the second adapter sent by the first node in the second node, and since the large model includes the main model, the first adapter, and the second adapter, and the main model is retained in the first node, the phenomenon of providing the entire main body of the large model to the second node can be avoided. Since the main part of the large model, i.e. the main model, is in the first node, the intellectual property rights of the large model owner in the first node are also protected, and the phenomenon of the second node damaging the intellectual property rights of the large model after knowing the specific architecture data of the large model is avoided.

[0122] Furthermore, during large-scale model adjustments, the second node transforms the sample business data features using the second adapter to obtain first representation data, which is then sent to the first node. The first node processes this first representation data using the large model to obtain second representation data, which is then sent back to the second node. The second node receives the target gradient value determined by the first adapter, the second representation data, and the business label, and adjusts the large model accordingly until the adjustment is complete. This avoids the risk of training sample data leakage from the large model owner in the first node directly accessing the training sample data of the model requester in the second node. Since the second node processes the sample business data features of the training sample data using the second adapter to obtain the second representation data, the large model in the first node only processes the first representation data and is unaware of the specific training sample data in the second node. This also protects data privacy in the second node and reduces the risk of data leakage. In short, it achieves both the protection of the large model owner's intellectual property rights and the protection of the model requester's data privacy.

[0123] Furthermore, based on any one of the first to third embodiments described above, a second embodiment of the large model data protection method of the present invention is proposed. In this embodiment, reference is made to... Figure 9 After sending the target gradient value to the first node in step S300, the method further includes:

[0124] Step S400: After determining that the large model adjustment in the first node is completed, if there are sample service data features to be predicted, perform format transformation processing on the sample service data features to be predicted according to the second adapter to obtain the representation data to be predicted, and send the representation data to be predicted to the first node.

[0125] Step S500: Receive the prediction representation data sent by the first node, and perform format transformation processing on the prediction representation data according to the first adapter to obtain the actual prediction data.

[0126] In this embodiment, after the second node receives confirmation that the first node has completed the training of the large model and that the training times of the large model have reached a preset number, or that the model has converged, the second node can use the large model in the first node to perform corresponding processing operations.

[0127] Optionally, determining the features of the sample business data to be predicted in the second node can be achieved by extracting these features using a second-party feature extractor. Optionally, the features of the sample business data to be predicted are input to a second adapter, which performs format transformation on them. Optionally, encryption can also be performed to obtain the representation data to be predicted. The second node sends the representation data to be predicted to the first node, which inputs it into a large model for model training and prediction to obtain the corresponding prediction result, which is then used as the prediction representation data. The prediction representation data is then sent to the second node. Optionally, after receiving the prediction representation data from the first node, the second node can input it into a first adapter for format transformation to obtain the actual prediction data. The format of the actual prediction data is consistent with the format of the business tag.

[0128] In this embodiment, after the large model in the first node is adjusted, the format transformation processing of the sample business data features to be predicted is performed according to the second adapter to obtain the representation data to be predicted, and the representation data to be predicted is sent to the first node. The predicted representation data sent by the first node is received, and the format transformation processing is performed according to the first adapter to obtain the actual predicted data, thereby realizing the process of data prediction through the large model in the first node.

[0129] Further, the step of performing format transformation processing on the sample service data features according to the second adapter includes:

[0130] Step b: Select a preset number of sample business data features from multiple sample business data features with business labels to construct a dataset;

[0131] Step c: For at least one sample service data in the dataset, perform format transformation processing on the features of the sample service data according to the second adapter.

[0132] Optionally, in the second node, multiple sample business data features with business labels can be stored in advance. These multiple sample business data features can be extracted using a second-party feature extractor to construct a dataset by extracting a preset number of sample business data features. The preset number can be any number set by the user in advance. The dataset can be training sample data used for model training, including extracted small batches of sample business data features and their corresponding business labels.

[0133] Optionally, a sample business data can be randomly selected from the dataset and input into the second adapter for format transformation processing and subsequent corresponding operations. After completing one model training of the large model in the first node, if the model does not converge or does not reach the preset number of training iterations, a new sample business data can be randomly selected from the dataset again until the large model in the first node is detected to have converged or reached the preset number of training iterations.

[0134] In this embodiment, the second node constructs a dataset by selecting a preset number of sample business data features from multiple sample business data features with business labels, and then performs subsequent model training processing based on the sample business data in the dataset, thereby ensuring the effective execution of subsequent model training.

[0135] Further, the step of performing format transformation processing on the sample service data features according to the second adapter to obtain the first representation data includes:

[0136] Step d: Determine the original format corresponding to the sample business data features and the model format corresponding to the large model;

[0137] Step e: Based on the second adapter, the format corresponding to the sample service data features is converted from the original format to the model format to obtain privacy representation data;

[0138] Step f: Input the privacy representation data into a preset first protection model for encryption processing, and output the first representation data.

[0139] Optionally, in the second node, the original format corresponding to the sample business data features can be determined, such as the format that the second node can recognize. The model format that the large model in the first node can recognize can also be determined. Optionally, the original format and the model format can be the same or different.

[0140] Optionally, the sample business data features can be input into a second adapter, which converts the format of the sample business data features from the original format to the model format to obtain privacy representation data. The second adapter can achieve this conversion by encoding the sample business data features. Other methods can also be used to achieve the format conversion of the sample business data features; no restrictions are placed here.

[0141] Optionally, the second node can input the privacy representation data into the first protection model for encryption processing to obtain the first representation data, and then send the first representation data to the first node. The first protection model can be an algorithmic model capable of protecting and encrypting data, such as a differential privacy algorithm model.

[0142] In this embodiment, privacy representation data is obtained by converting the format of the first sample data features from the original format to the model format according to the second adapter. Then, the privacy representation data is encrypted according to the first protection model to obtain the first representation data. This avoids the phenomenon of directly sending the privacy data in the second node to the first node and improves the security of the privacy data.

[0143] Further, the step of performing format transformation processing on the second representation data based on the first adapter to obtain the first prediction result includes:

[0144] Step g: Input the second characterization data into the preset second protection model for decryption processing, and output the third characterization data;

[0145] Step h: Based on the first adapter, the format corresponding to the third representation data is converted from the model format to the original format to obtain the first prediction result.

[0146] In this embodiment, the second protection model can be a model with the same architecture as the first protection model. For example, if the first protection model adopts a differential model, the second protection model can also continue to adopt a differential model.

[0147] Optionally, after the second node receives the second representation data sent by the first node, it can decrypt the second representation data through the second protection model to obtain the third representation data. Then, the third representation data is input into the first adapter for format transformation to obtain the final prediction result.

[0148] In this embodiment, the third characterization data is obtained by decrypting the second characterization data according to the second protection model, and then the third characterization data is transformed according to the first adapter to obtain the prediction result, thereby ensuring the validity of the obtained prediction result.

[0149] Further, the step of determining the target gradient value based on the first prediction result and the features of the second sample data, and sending the target gradient value to the first node, also includes:

[0150] Step i: Calculate the gradient value based on the first prediction result and the business label to obtain the first gradient value;

[0151] Step j: Adjust the parameters of the first adapter according to the first gradient value to obtain a target gradient value, and send the target gradient value to the first node. The first node adjusts the large model according to the target gradient value to obtain a second gradient value, and sends the second gradient value to the second node.

[0152] After the step of sending the target gradient value to the first node, the following is included:

[0153] Step k: Receive the second gradient value sent by the first node, adjust the parameters of the second adapter according to the second gradient value, and obtain new sample service data features. Then, re-execute the step of performing format transformation processing on the sample service data features according to the second adapter based on the new sample service data features, until the large model in the first node is detected to have converged or the preset number of training iterations has been reached.

[0154] Optionally, in the second node, the prediction result and the features of the second sample data can be input into a pre-set loss function to calculate the gradient, obtaining the first gradient value. Optionally, the loss function can be any language task loss function, such as the error rate for predicting the next word. The gradient calculation can be the calculation of the first derivative of the loss function with respect to the model parameters.

[0155] Optionally, in the second node, the parameters of the first adapter can be adjusted using the first gradient value, for example, by subtracting a certain proportion of the gradient value from the model parameters of the first adapter. When the first adapter is adjusted based on the gradient value, its first gradient value is also updated, such as by decreasing the gradient value. The updated first gradient value can be used as the adjusted first gradient value. Then, the adjusted first gradient value is sent to the first protection model. The first protection model also updates its parameters based on the adjusted first gradient value, and the adjusted first gradient value is updated synchronously during the parameter update to obtain the target gradient value, which is then sent to the first node. The large model in the first node updates its parameters based on the target gradient value, and the target gradient value is updated synchronously during the parameter update to obtain the second gradient value, which is then sent to the second node. The second protection model in the second node updates its parameters based on the second gradient value, and the second gradient value is updated synchronously during the parameter update, which is then sent to the second adapter. The second adapter adjusts its parameters based on the updated second gradient value, for example, by subtracting a certain percentage of the gradient value from the model parameters of the second adapter. Furthermore, the parameter adjustment process for the first adapter, the large model, and the second adapter can be fine-tuned multiple times based on different gradient values ​​until the large model meets certain requirements, such as achieving a preset effect.

[0156] In this embodiment, by adjusting the parameters of the first adapter, the second adapter in the second node, and the large model in the first node based on the gradient value, it is possible to directly adjust the parameters of the large model in the first node while protecting the security of privacy data in the second node.

[0157] Furthermore, embodiments of the present invention also propose a large model data protection system, referring to... Figure 10 The large model data protection system includes an interconnected first node and a second node. The large model is deployed on the first node, and the second node possesses sample business data features and business tags.

[0158] The first node is used to send the first adapter and the second adapter obtained by feature extraction of the large model to the second node, wherein the large model includes the first adapter, the main model and the second adapter;

[0159] The second node is used to receive the first adapter and the second adapter sent by the first node, perform format transformation processing on the sample service data features according to the second adapter to obtain the first characterization data, and send the first characterization data to the first node.

[0160] The first node is configured to receive first representation data sent by the second node, input the first representation data into the large model for model training, output second representation data, and send the second representation data to the second node.

[0161] The second node is configured to receive the second representation data sent by the first node, perform format transformation processing on the second representation data according to the first adapter to obtain a first prediction result, determine the target gradient value according to the first prediction result and the service tag, and send the target gradient value to the first node.

[0162] The first node is used to receive the target gradient value sent by the second node, and adjust the large model according to the target gradient value until the large model adjustment is completed.

[0163] Furthermore, this invention also proposes a large model data protection device, which includes: a memory, a processor, and a large model data protection program stored in the memory and executable on the processor. When the large model data protection program is executed by the processor, it implements the steps of the large model data protection method described above.

[0164] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a large model data protection program, wherein the large model data protection program, when executed by a processor, implements the steps of the large model data protection method described below.

[0165] The present invention also proposes a computer program product, comprising a computer program that, when executed by a processor, implements the steps of the large model data protection method described above.

[0166] The various embodiments of the large model data protection system, large model data protection device, computer-readable storage medium, and computer program product of the present invention can all refer to the various embodiments of the large model data protection method of the present invention, and will not be repeated here.

[0167] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0168] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0169] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0170] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for protecting large model data, characterized in that, The large model data protection method is applied to a first node connected to a second node. A large model is deployed on the first node, and the second node possesses sample business data features and business tags. The method includes the following steps: The first adapter and the second adapter obtained by feature extraction of the large model are sent to the second node, wherein the large model includes the first adapter, the main model and the second adapter; The system receives first representation data sent by the second node, inputs the first representation data into the large model for model training, outputs second representation data, and sends the second representation data to the second node. The first representation data is data obtained by the second node through processing based on the features of the second adapter and the sample service data. The system receives the target gradient value sent by the second node and adjusts the large model based on the target gradient value until the large model adjustment is completed. The target gradient value is the gradient value determined by the second node based on the first adapter, the second representation data, and the service label.

2. The large model data protection method as described in claim 1, characterized in that, After the step of adjusting the large model based on the target gradient value until the large model adjustment is complete, the following steps are included: If the predictable representation data is received from the second node, the predictable representation data is input into the adjusted large model for model training, and the predicted representation data is output. The predicted representation data is then input into the second node, wherein the second node performs format conversion processing on the predicted representation data according to the first adapter to obtain the actual predicted data.

3. The large model data protection method as described in claim 1, characterized in that, The step of inputting the first representation data into the large model for model training and outputting the second representation data includes: The large model is compressed to obtain a compressed model. The first representation data is input into the compressed model for model training, and the second representation data is output.

4. A method for protecting large model data, characterized in that, The large model data protection method is applied to a second node connected to the first node. A large model is deployed on the first node, and the second node possesses sample business data features and business tags. The method includes the following steps: The system receives the first adapter and the second adapter sent by the first node, performs format transformation processing on the sample service data features according to the second adapter to obtain the first characterization data, and sends the first characterization data to the first node. The system receives second representation data sent by the first node, performs format transformation on the second representation data according to the first adapter, and obtains a first prediction result, wherein the second representation data is the prediction result of the large model in the first node predicting the first representation data. Based on the first prediction result and the business label, a target gradient value is determined, and the target gradient value is sent to the first node, wherein the first node adjusts the large model based on the target gradient value.

5. The large model data protection method as described in claim 4, characterized in that, After sending the target gradient value to the first node, the method further includes: After the large model adjustment in the first node is completed, if there are sample business data features to be predicted, the second adapter is used to perform format transformation on the sample business data features to be predicted to obtain the representation data to be predicted, and the representation data to be predicted is sent to the first node. The system receives the prediction representation data sent by the first node, performs format transformation processing on the prediction representation data according to the first adapter, and obtains the actual prediction data.

6. The large model data protection method as described in claim 4, characterized in that, The step of performing format transformation processing on the sample service data features according to the second adapter includes: A dataset is constructed by selecting a predetermined number of sample business data features from multiple sample business data features with business labels; For at least one sample service data in the dataset, perform format transformation processing on the features of the sample service data according to the second adapter.

7. The large model data protection method as described in claim 4, characterized in that, The step of performing format transformation processing on the sample service data features according to the second adapter to obtain the first representation data includes: Determine the original format corresponding to the sample business data features, and the model format corresponding to the large model; The second adapter converts the format corresponding to the sample service data features from the original format to the model format to obtain privacy representation data. The privacy representation data is input into a preset first protection model for encryption processing, and the first representation data is output.

8. The large model data protection method as described in claim 7, characterized in that, The step of performing format transformation processing on the second representation data according to the first adapter to obtain the first prediction result includes: The second characterization data is input into a preset second protection model for decryption processing, and the third characterization data is output. The first prediction result is obtained by converting the format corresponding to the third representation data from the model format to the original format based on the first adapter.

9. The large model data protection method as described in claim 4, characterized in that, The step of determining the target gradient value based on the first prediction result and the service tag, and sending the target gradient value to the first node, includes: Based on the first prediction result and the business label, a gradient value is calculated to obtain a first gradient value; The first adapter is adjusted according to the first gradient value to obtain a target gradient value, and the target gradient value is sent to the first node. The first node adjusts the large model according to the target gradient value to obtain a second gradient value, and the second gradient value is sent to the second node. After the step of sending the target gradient value to the first node, the following is included: The system receives the second gradient value sent by the first node, adjusts the parameters of the second adapter based on the second gradient value, and obtains new sample service data features. Based on the new sample service data features, the system re-executes the step of performing format transformation processing on the sample service data features based on the second adapter until the large model in the first node is detected to have converged or the preset number of training iterations has been reached.

10. A large model data protection system, characterized in that, The large model data protection system includes an interconnected first node and a second node. The large model is deployed on the first node, and the second node possesses sample business data features and business tags. The first node is used to send the first adapter and the second adapter obtained by feature extraction of the large model to the second node, wherein the large model includes the first adapter, the main model and the second adapter; The second node is used to receive the first adapter and the second adapter sent by the first node, perform format transformation processing on the sample service data features according to the second adapter to obtain the first characterization data, and send the first characterization data to the first node. The first node is configured to receive first representation data sent by the second node, input the first representation data into the large model for model training, output second representation data, and send the second representation data to the second node. The second node is configured to receive the second representation data sent by the first node, perform format transformation processing on the second representation data according to the first adapter to obtain a first prediction result, determine the target gradient value according to the first prediction result and the service tag, and send the target gradient value to the first node. The first node is used to receive the target gradient value sent by the second node, and adjust the large model according to the target gradient value until the large model adjustment is completed.

11. A large model data protection device, characterized in that, The large model data protection device includes: a memory, a processor, and a large model data protection program stored in the memory and executable on the processor. When the large model data protection program is executed by the processor, it implements the steps of the large model data protection method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a large model data protection program, which, when executed by a processor, implements the steps of the large model data protection method as described in any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the large model data protection method as described in any one of claims 1-9.

Images