Model processing method and device and computer readable storage medium

Through risk assessment and dynamic update of the model parameters of the adversarial sample generator, combined with the multi-objective optimization algorithm, the problem of insufficient security in the application process of generative large language models is solved, and effective response to dynamic threats and optimization of model parameters is achieved. .

CN119089439APending Publication Date: 2024-12-06SHANGHAI JIKRYPTON BLUE NEW ENERGY TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411249993.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Generative large language models have security problems during application, which may generate harmful content, leak sensitive information, or be maliciously exploited, and the existing technology is difficult to effectively deal with dynamically changing security threat environments.

Method used

The risk score of the generative large language model is determined through the risk assessment model, dynamically update the model parameters of the adversarial sample generator, generate diverse adversarial samples, and use these adversarial samples to update the parameters of the generative large language model, combining with the multi-objective optimization algorithm Optimize model parameters to improve the security and adaptability of the model.

Benefits of technology

It effectively improves the security of the generative large language model, enables it to cope with dynamically changing security threat environments, improves the model's identification and resistance to attacks, and ensures continuous adaptation to new security threats.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089439A_ABST
    Figure CN119089439A_ABST
Patent Text Reader

Abstract

The invention provides a model processing method and device and a computer readable storage medium. The method comprises the steps of determining a risk score of a generative large language model according to a risk assessment model; updating model parameters of an adversarial sample generator according to the risk score; generating an adversarial sample according to the updated adversarial sample generator; and updating model parameters of the generative large language model according to the adversarial sample to generate an updated generative large language model. Through the method, the generative large language model can cope with a dynamically changing security threat environment, and the security of the generative large language model is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of large language models, and more specifically, to a method, device and computer-readable storage medium for processing models. Background Art

[0002] With the widespread application of generative large language models in various fields, their security issues have become increasingly prominent. Such models may generate harmful content, leak sensitive information, or be maliciously exploited, posing a threat to user and system security. Therefore, how to improve the security of generative large language models is a problem that needs to be solved. Summary of the invention

[0003] The present application provides a method, device and readable storage medium for processing a model to cope with a dynamically changing security threat environment and improve the security of a generative large language model.

[0004] In a first aspect, a method for processing a model is provided, comprising: determining a risk score of a generative large language model according to a risk assessment model; updating model parameters of an adversarial sample generator according to the risk score; generating an adversarial sample according to the updated adversarial sample generator; and updating model parameters of the generative large language model according to the adversarial sample to generate an updated generative large language model.

[0005] In a possible implementation, the method further includes: updating a model parameter of a security defense model according to the risk score; and performing security defense on the generative large language model according to the updated security defense model.

[0006] In one possible implementation, the security defense model includes at least one of the following: a first model for filtering input information of the generative large language model; a second model for detecting whether there is abnormal information in the intermediate information generated by the generative large language model; and a third model for detecting whether there is abnormal information in the output information generated by the generative large language model.

[0007] In a possible implementation, the method further includes: optimizing model parameters of the generative large language model using a multi-objective optimization algorithm to generate an optimized generative large language model.

[0008] In a possible implementation, the objective function of the multi-objective optimization algorithm includes at least one of the following functions: a first function for indicating a mapping relationship between model parameters of the generative large language model and the efficiency of the generative large language model; a second function for indicating a mapping relationship between model parameters of the generative large language model and the security of the generative large language model; a third function for indicating a mapping relationship between model parameters of the generative large language model and the prediction accuracy of the generative large language model.

[0009] In a possible implementation, the adversarial sample generator is an adversarial sample generator based on a WGAN structure, and the adversarial sample generator performs adversarial training based on the following objective function:

[0010]

[0011] Where L represents the adversarial sample generator, G represents the generator in the adversarial sample generator, D represents the discriminator in the adversarial sample generator, x represents the real sample, z represents random noise, and p data (x) represents the data distribution of real data, p z (z) represents the data distribution of random noise, λ represents the gradient penalty coefficient, is a random interpolation point between x and G(z), express data distribution.

[0012] In a possible implementation, the input sequence of the generative large language model is x=[x1,...,x T ], the risk score is determined based on the following formula:

[0013] S(x)=∑ t a t ·f(h t );

[0014] h t =LSTM(x t ,h t―1 );

[0015] a t =softmax(v T tanh(W h h t +W x x t + b));

[0016] f(h t )=σ(W f h t +b f);

[0017] Where S(x) represents the risk score, LSTM represents the long short-term memory network, and h t represents the hidden state of LSTM, softmax represents the activation function, a t represents the attention weight, f(h t ) represents the risk assessment function corresponding to the risk assessment model, W f , W x , W h ,v,b,b f represents learnable parameters.

[0018] In a second aspect, a device for processing a model is provided, including: a determination module, used to determine a risk score of a generative large language model according to a risk assessment model; a first update module, used to update a model parameter of an adversarial sample generator according to the risk score; a generation module, used to generate an adversarial sample according to the updated adversarial sample generator; a second update module, used to update the model parameters of the generative large language model according to the adversarial sample to generate an updated generative large language model.

[0019] In a possible implementation, the device also includes: a third updating module, used to update the model parameters of the security defense model according to the risk score; and a security defense module, used to perform security defense on the generative large language model according to the updated security defense model.

[0020] In a possible implementation, the security defense model includes at least one of the following: a first model for filtering input information of the generative large language model; a second model for detecting whether the intermediate information generated by the generative large language model contains abnormal information; a third model for detecting whether the output information generated by the generative large language model contains abnormal information;

[0021] In a possible implementation, the device further includes: an optimization module, configured to optimize model parameters of the generative large language model using a multi-objective optimization algorithm to generate an optimized generative large language model.

[0022] In a possible implementation, the objective function of the multi-objective optimization algorithm includes at least one of the following functions: a first function for indicating a mapping relationship between model parameters of the generative large language model and the efficiency of the generative large language model; a second function for indicating a mapping relationship between model parameters of the generative large language model and the security of the generative large language model; a third function for indicating a mapping relationship between model parameters of the generative large language model and the prediction accuracy of the generative large language model.

[0023] In a possible implementation, the adversarial sample generator is an adversarial sample generator based on a WGAN structure, and the adversarial sample generator performs adversarial training based on the following objective function:

[0024]

[0025] Wherein, L represents the adversarial sample generator, G represents the generator in the adversarial sample generator, D represents the discriminator in the adversarial sample generator, x represents the real sample, z represents random noise, pdata(x) represents the data distribution of the real data, pz(z) represents the data distribution of random noise, λ represents the gradient penalty coefficient, is a random interpolation point between x and G(z), express data distribution.

[0026] In a possible implementation, the input sequence of the generative large language model is x=[x1, ..., xT], and the risk score is determined based on the following formula:

[0027] S(x)=∑ t a t ·f(h t );

[0028] h t =LSTM(h t ,h t―1 );

[0029] a t =softmax(v T tanh(W h h t +W x x t + b));

[0030] f(h t )=σ(W f h t +b f );

[0031] Where S(x) represents the risk score, f(h t ) represents the risk assessment function corresponding to the risk assessment model, LSTM represents the long short-term memory network, h t represents the hidden state of LSTM, softmax represents the activation function, a t represents the attention weight, W f , W x , W h ,v,b,b f represents learnable parameters.

[0032] In a third aspect, a device for processing a model is provided, comprising: a memory for storing code; and a processor for executing the code stored in the memory to execute the method described in the first aspect or any possible implementation method of the first aspect.

[0033] According to a fourth aspect, a computer-readable storage medium is provided, on which is stored a code for executing the method described in the first aspect or any possible implementation of the first aspect.

[0034] In the embodiment of the present application, since the model parameters of the adversarial sample generator are dynamically updated according to the risk score of the generative large language model, the adversarial samples generated by the updated adversarial sample generator are also diversified adversarial samples. Updating the parameters of the generative large language model according to the newly generated diversified adversarial samples can enable the generative large language model to cope with the dynamically changing security threat environment and improve the security of the generative large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 It is a flow chart of a method for processing a generative large language model based on adversarial samples provided in an embodiment of the present application.

[0037] Figure 2 This is an example diagram of security defense for a generative large language model based on a security defense model provided in an embodiment of the present application.

[0038] Figure 3 This is an example diagram of a method for processing a model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present application belong to the scope of protection of this application.

[0040] To facilitate understanding, the following first introduces relevant concepts involved in the embodiments of the present application.

[0041] A generative large language model is a model in natural language processing whose main task is to generate natural language text; this model can learn and understand the inherent structure and regularities of language, and then generate text that conforms to grammatical and semantic rules based on this knowledge and given context or prompts. The working principle of a generative large language model is to learn the statistical laws of language by analyzing large amounts of text data. During the training process, the model tries to predict the probability distribution of the next word or character in a given context. Once the model is trained, it can be used to generate new texts that may be similar to the training data in style, structure, and content.

[0042] Generative large language models have a wide range of applications, including but not limited to machine translation, text summarization, dialogue systems, creative writing, code generation, etc.; these applications require the model to be able to generate reasonable, natural and coherent text output based on given input.

[0043] With the widespread application of generative large language models in various fields, their security issues have become increasingly prominent. These models may generate harmful content, leak sensitive information, or be maliciously exploited, posing a threat to user and system security. Therefore, improving the security of large language models has become an important research topic in academia and industry.

[0044] At present, related technologies mainly use static adversarial training or regular defense strategies to defend the generative large language model. The static adversarial training method enhances the robustness of the model by introducing predefined adversarial samples in the training phase. Adversarial samples refer to input samples formed by deliberately adding subtle interference to the data set, which will cause the model to give an erroneous output with high confidence. For example, the adversarial training method proposed by Goodfellow et al. uses the fast gradient symbol method to generate adversarial samples and incorporates them into the training process. The core idea of ​​the fast gradient symbol method to generate adversarial samples is to use the gradient information of the model's loss function to the input, determine the direction of the perturbation through the symbolic function, and add tiny perturbations along this direction to generate adversarial samples. This method performs well in specific attack modes, but it is difficult to cope with new attacks that evolve rapidly.

[0045] Related technologies also propose rule-based defense strategies. For example, OpenAI's GPT series models use content filtering and output review mechanisms to filter out potentially harmful content through preset rules and sensitive word lists. Although this method is simple and direct, it lacks flexibility and is difficult to handle complex contexts and obscure harmful information.

[0046] In addition, researchers have also tried various security enhancement techniques, such as the trigger detection method proposed by Wallace et al., which aims to identify input patterns that may cause the model to generate harmful content. However, these methods often focus on specific types of security issues and lack comprehensiveness.

[0047] The above methods are static defense methods. Although these static defense methods have improved the security of the model to a certain extent, they still have many limitations. They generally lack dynamic adaptability and are difficult to effectively respond to dynamically changing security threat environments.

[0048] For the above questions, see Figure 1 , an embodiment of the present application proposes a method for processing a model. That is, the score of the generative large language model is determined according to the risk assessment model (see step S110), and then the model parameters of the adversarial sample generator are updated based on the risk score (see step S120), and then the adversarial sample is generated by the updated adversarial sample generator (see step S130), and finally the model parameters of the generative large language model are updated according to the adversarial sample to generate an updated generative large language model (see step S140). Since the model parameters of the adversarial sample generator are dynamically updated according to the risk score of the generative large language model, the adversarial samples generated by the updated adversarial sample generator are also diversified adversarial samples. Updating the parameters of the generative large language model according to the newly generated diversified adversarial samples can enable the generative large language model to cope with the evolving attack strategies and improve the security of the generative large language model.

[0049] See also Figure 1 , in step S110, a risk score of a generative large language model is determined according to the risk assessment model.

[0050] The risk assessment model is a model used to analyze and assess the security risks and potential problems that may be caused by the generative large language model. The risks mentioned here may include, for example, one or more of harmful output content, leakage of sensitive information, malicious use of the model, etc. The risk score output by the risk assessment model may, for example, indicate the probability that the generative large language model has security risks or potential hazards.

[0051] The embodiments of the present application do not specifically limit the type of risk assessment model. For example, the risk assessment model may adopt a rule-based risk assessment model. Alternatively, the risk assessment model may also adopt an attention-based risk assessment model. Taking the attention-based risk assessment model as an example, for the input sequence x=[x1,...,xT], its risk score S(x) can be determined based on the following formula:

[0052]

[0053] h t =LSTM(x t ,h t―1 )

[0054] a t =softmax(v T tanh(W h h t +W x x t +b))

[0055] f(h t )=σ(W f h t +b f )

[0056] Where S(x) represents the risk score, LSTM represents the long short-term memory network, which can remember information over a long time interval when processing sequence data, and h t represents the hidden state of LSTM, softmax represents the activation function, which can normalize a numerical vector into a probability distribution vector, and the sum of each probability is 1, a t represents the attention weight, f(h t ) represents the risk assessment function corresponding to the risk assessment model, W f , W x , W h ,v,b,b f represents learnable parameters.

[0057] In steps S120 to S130, the model parameters of the adversarial sample generator are updated based on the risk score, and the updated adversarial sample generator generates an adversarial sample.

[0058] There are many ways to update the parameters of the adversarial sample generator and generate new adversarial samples. For example, the adversarial sample generator with a traditional generative adversarial network (GAN) structure can be trained through an objective function, or the adversarial sample generator with a Wasserstein generative adversarial network (WGAN) structure can be trained through an objective function. Take the adversarial training of the adversarial sample generator with a WGAN structure through the following objective function as an example:

[0059]

[0060] Wherein, L represents the adversarial sample generator, G represents the generator in the adversarial sample generator, D represents the discriminator in the adversarial sample generator, x represents the real sample, z represents random noise, pdata(x) represents the data distribution of the real data, pz(z) represents the data distribution of random noise, λ represents the gradient penalty coefficient, is a random interpolation point between x and G(z), express data distribution.

[0061] Based on the dynamically updated risk score, the adversarial sample generator of the WGAN structure can generate diverse and high-quality adversarial samples through adversarial training with the above objective function.

[0062] In step S140, the model parameters of the generative large language model are updated according to the adversarial sample to generate an updated generative large language model.

[0063] The generative large language model is trained with newly generated diverse and high-quality adversarial samples. Through training, the ability of the generative large language model to identify and resist attacks can be improved, and the model parameters of the generative large language model can be updated. The updated generative large language model can cope with evolving attack strategies and continuously adapt to new security threats, thereby improving the security of the model.

[0064] In the above, we use the updated adversarial sample generator to generate new adversarial samples, and then use the newly generated adversarial samples to train the generative large language model to drive the model to update parameters, thereby improving the model's ability to cope with dynamically changing security threats. In addition to this method, the ability of the generative large language model to cope with security threats can also be improved by updating the security defense strategy. For example, the model parameters of the security defense model can be updated according to the risk score, and then the generative large language model can be defended according to the updated security defense model.

[0065] The security defense model may include a model for filtering the input information of the generative large language model (for ease of description, the model is referred to as the first model below). The first model can be used to filter the input information of the generative large language model. The input information mentioned here may refer to the initial data provided to the generative large language model. These data may include one or more of text, instructions, images, sounds, etc., and the embodiments of the present application do not specifically limit this.

[0066] The first model can be any model that can filter the input information of the generative large language model. For example, the first model can be a Transformer model. Alternatively, the first model can be a model based on a self-attention mechanism. Taking the first model as a model based on a self-attention mechanism as an example, the self-attention mechanism model can detect the security of the input information of the generative large language model based on the following detection algorithm:

[0067] Q=W Q X,K=W K X,V=W V X

[0068]

[0069] Where Q is the query vector, K is the key vector, V is the value vector, X is the input matrix, and W Q ,W K ,W V is the weight matrix, K T Used to calculate the similarity score, d k It is the dimension of attention.

[0070] The model with the self-attention mechanism can filter out content with security issues in the input information of the large language model, thereby improving the security of the generative large language model.

[0071] The security defense model may also include a model for detecting whether the intermediate information generated by the generative large language model contains abnormal information (for ease of description, the model is referred to as the second model below). The second model can be used to analyze or detect the intermediate information of the generative large language model to determine the security of the intermediate information. The intermediate information mentioned here may refer to a series of intermediate data or states generated by the generative large language model in the process of processing input information and generating output.

[0072] The second model can be any model that can detect whether there are abnormalities in the intermediate information generated by the generative large language model. For example, the second model can be a model based on the attention mechanism. Or, the second model can be a Transformer model. Taking the Transformer model as an example, the Transformer model can use the following algorithm to capture abnormal information in the intermediate information:

[0073] h l =MultiHead(h l―1 )+h l―1

[0074] h l =LayerNorm(FFN(h l ))+h l

[0075] Among them, h l represents the output of layer l, h l―1 represents the output of the (l-1)th layer, MultiHead is the multi-head attention mechanism, FFN is the feedforward neural network, and LayerNorm is the layer normalization.

[0076] The Transformer model can be used to analyze and detect the intermediate information generated by the generative large language model, thereby improving the security of the generative large language model.

[0077] The security defense model can also be a model for detecting whether the output information generated by the generative large language model contains abnormal information (for the sake of ease of description, the model is referred to as the third model below). The third model can be used to audit the output information of the generative large language model. The output information mentioned here can refer to the result generated by the generative large language model after a series of processing of the input model data.

[0078] The third model may include any model that can detect whether the output information generated by the generative large language model has anomalies. For example, the third model may include a Transformer model. For another example, the third model may include a graph neural network model. Taking the graph neural network model as an example, the graph neural network model can detect whether the output information generated by the generative large language model has anomalies based on the following algorithm:

[0079] R(y)=σ(W o [GNN(G y )||R rule (y)]+b o )

[0080] Where y is the output, R(y) is the safety score, σ is the function, and W ois the weight matrix of the output layer, GNN(·) is the graph neural network operation, G y is a graph structure constructed based on the output y, || represents the concatenation operation, R rule (y) is the rule-based scoring, b o is the bias term of the output layer.

[0081] The graph neural network model can be used to detect whether the output information generated by the large language model contains abnormal information, thereby improving the security of the generative large language model.

[0082] The first model, the second model, and the third model in the security defense model are introduced above. The security defense model may include one of the three models, or may include multiple of the three models.

[0083] As mentioned above, the model parameters of the above security defense model can be dynamically updated according to the risk score. The updated security defense architecture can provide security defense for the generative large language model. Since dynamically updating the model parameters of the security defense model according to the risk score is an incremental learning and rapid adaptation mechanism, the security defense architecture of the generative large language model can be continuously optimized without interrupting the service. This enables the generative large language model to cope with continuously changing security threats, greatly improving the security of the model.

[0084] Combine the following Figure 2 , taking the security defense model including the first model, the second model and the third model mentioned above as an example, the process of defending a large language model is explained.

[0085] The security defense model includes a first model, a second model and a third model. The generative large language model inputs information to the first model, and the first model can use a model with a self-attention mechanism to filter the content with security issues in the input information. The filtered information (also referred to as the intermediate information generated by the generative large language model) is transmitted to the second model. The second model can use a transformer model to detect whether there are abnormalities in the intermediate information generated by the generative large language model. The output information (also referred to as the output information generated by the generative large language model) after detection by the second model will be detected by the third model before the final output. The third model can use a graph neural network model to detect whether there are abnormalities in the output information generated by the generative large language model.

[0086] Through the layers of defense of the first model, the second model, and the third model, the risk of security vulnerabilities in the generative large language model can be comprehensively reduced. In addition, the model parameters of the security defense model can be dynamically updated according to the risk score. This security defense model enables the generative large language model to dynamically adjust and optimize the defense strategy in real time without interrupting the service, thereby improving the ability of the generative large language model to deal with dynamic security threats.

[0087] In summary, the method for processing a generative large language model can improve the security of a generative large language model. However, in practical applications, improving the security of the model may reduce other performance of the model. In order to find a balance between the security and other performance of the model, the embodiment of the present application proposes to optimize the parameters of the large language model using a multi-objective optimization algorithm.

[0088] The multi-objectives may include: accuracy, efficiency, security, interpretability, robustness, etc. A multi-objective optimization algorithm is an optimization algorithm that considers multiple objective functions at the same time. The multi-objective optimization algorithm needs to take into account multiple objectives at the same time, and try to optimize the satisfaction of each objective as much as possible while ensuring that each objective is satisfied to a certain extent. The purpose of the multi-objective optimization algorithm is to find a set of solutions that can achieve good performance on multiple objectives. There can be many types of objective functions. For example, the objective function may be a function for indicating the mapping relationship between the model parameters of the generative large language model and the efficiency of the generative large language model (for the convenience of description, the function is referred to as the first function below). The objective function may also be a function for indicating the mapping relationship between the model parameters of the generative large language model and the security of the generative large language model (for the convenience of description, the function is referred to as the second function below). The objective function may also be a function for indicating the mapping relationship between the model parameters of the generative large language model and the accuracy of the generative large language model (for the convenience of description, the function is referred to as the third function below). The objective function of the multi-objective optimization algorithm may include at least one of the above-mentioned multiple objective functions.

[0089] The following example uses a multi-objective optimization algorithm in which the objective function includes the first function, the second function, and the third function as an example to illustrate the optimization of the model parameters of the generative large language model using the multi-objective optimization algorithm. There can be many types of multi-objective optimization algorithms, which are not limited in this application. For example, it can be a multi-objective algorithm based on Pareto optimization. The specific algorithm is as follows:

[0090] min θ F(θ)=[f A (θ),f E (θ),f S (θ)]

[0091] stg i (θ)≤0,i=1,...,m

[0092] h j (θ)=0,j=1,...,n

[0093] Among them, θ represents the decision variable, f A ,f E ,f S They represent the objective functions of accuracy, efficiency and safety respectively, and g i and h j is a constraint.

[0094] By using the NSGA-II algorithm to solve this multi-objective optimization problem, we can find the Pareto optimal solution set. The Pareto optimal solution set here represents the set of solutions that find the optimal trade-off between accuracy, efficiency, and security. By optimizing the model parameters of the generative large language model through the multi-objective optimization algorithm, the optimized generative large language model can find the best balance between security, accuracy, and efficiency, and achieve excellent performance in security, accuracy, and efficiency.

[0095] Combine the following Figure 3 , the embodiments of the present application are described in more detail. In the example below, the risk assessment model adopts a risk assessment model of an attention mechanism. The adversarial sample generator adopts an improved WGAN structure. For the defense architecture, the first model adopts a model of a self-attention mechanism, the second model adopts a Transformer model, and the third model adopts a graph neural network model. The multi-objective optimization algorithm adopts a multi-objective algorithm based on Pareto optimization. It should be noted that this example is only to help those skilled in the art understand the embodiments of the present application, and is not intended to limit the embodiments of the present application to the specific types or scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given below, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0096] Using the risk assessment model of the attention mechanism, for the input sequence x = [x1, ..., xT], its risk score S(x) can be determined based on the following formula:

[0097]

[0098] h t =LSTM(x t ,h t―1 )

[0099] a t =softmax(v Ttanh(W h h t +W x x t +b))

[0100] f(h t )=σ(W f h t +b f )

[0101] Where S(x) represents the risk score, LSTM represents the long short-term memory network, which can remember information over a long time interval when processing sequence data, and h t represents the hidden state of LSTM, softmax represents the activation function, which can normalize a numerical vector into a probability distribution vector, and the sum of each probability is 1, a t represents the attention weight, f(h t ) represents the risk assessment function corresponding to the risk assessment model, W f , W x , W h ,v,b,b f represents learnable parameters.

[0102] The risk score of the generative large language model can be determined based on the risk assessment model of the above attention mechanism. The model parameters of the adversarial sample generator can be updated based on the risk score of the generative large language model. The updated adversarial sample generator can generate new adversarial samples. The model parameters of the adversarial sample generator are updated based on the risk score and a new adversarial sample is generated. The adversarial sample generator with a WGAN structure can be used for adversarial training through the following objective function:

[0103]

[0104] Wherein, L represents the adversarial sample generator, G represents the generator in the adversarial sample generator, D represents the discriminator in the adversarial sample generator, x represents the real sample, z represents random noise, pdata(x) represents the data distribution of the real data, pz(z) represents the data distribution of random noise, λ represents the gradient penalty coefficient, is a random interpolation point between x and G(z), express data distribution.

[0105] The adversarial sample generator of the WGAN structure can generate diverse and high-quality adversarial samples through adversarial training with the above objective function. The generative large language model is trained with the newly generated diverse and high-quality adversarial samples. Through training, the ability of the generative large language model to identify and resist attacks can be improved, and the model parameters of the generative large language model can be updated. The updated generative large language model can cope with new security threats and improve the security of the large language model.

[0106] Furthermore, the model parameters of the security defense model are updated according to the risk score of the generative large language model determined by the risk assessment model of the attention mechanism.

[0107] The security defense model includes a first model, a second model and a third model.

[0108] The first model uses a model based on the self-attention mechanism. The detection algorithm of the self-attention mechanism model is as follows:

[0109] Q=W Q X,K=W K X,V=W V X

[0110]

[0111] Where Q is the query vector, K is the key vector, V is the value vector, X is the input matrix, and W Q ,W K ,W V is the weight matrix, K T Used to calculate the similarity score, d k It is the dimension of attention.

[0112] The model of the self-attention mechanism filters the content with security issues in the input information of the generative large language model.

[0113] The input information of the generative large language model is filtered and transmitted to the second model, which is used as the intermediate information generated by the large language model. The second model uses the Transformer model. The Transformer model detects whether there are security issues in the intermediate information generated by the generative large language model through the following algorithm:

[0114] h l =MultiHead(h l―1 )+h l―1

[0115] h l =LayerNorm(FFN(h l ))+h l

[0116] Among them, h l represents the output of layer l, h l―1 represents the output of the (l-1)th layer, MultiHead is the multi-head attention mechanism, FFN is the feedforward neural network, and LayerNorm is the layer normalization.

[0117] The detected intermediate information is transmitted to the third model, which is used as the output information generated by the generative large language model. The third model uses a graph neural network model. The graph neural network model uses the following algorithm to detect whether the output information generated by the generative large language model contains information about security issues:

[0118] R(y)=σ(W o [GNN(G y )||R rule (y)]+b o )

[0119] Where y is the output, R(y) is the safety score, σ is the function, and W o is the weight matrix of the output layer, GNN(·) is the graph neural network operation, G y is a graph structure constructed based on the output y, || represents the concatenation operation, R rule (y) is the rule-based scoring, b o is the bias term of the output layer.

[0120] The output information after detection by the graph neural network model is finally output by the generative large language model.

[0121] In the security defense model of the generative large language model, the parameters of the self-attention mechanism model, Transformer model, and graph neural network model are dynamically adjusted according to the risk score determined by the risk assessment model.

[0122] By updating the model parameters of the adversarial sample generator and the model parameters of the security defense model, the security of the generative large language model can be improved, enabling it to cope with dynamically changing security threats.

[0123] While the security of the generative large language model is improved, the accuracy and efficiency may be reduced. In order to make the generative large speech model have good performance in terms of security, accuracy and efficiency, the Pareto optimization multi-objective algorithm is used to optimize the generative large language model. The Pareto optimization multi-objective algorithm is as follows:

[0124] min θ F(θ)=[f A (θ),f E (θ),f S(θ)]

[0125] stg i (θ)≤0,i=1,...,m

[0126] h j (θ)=0,j=1,...,n

[0127] Among them, θ represents the decision variable, f A ,f E ,f S They represent the objective functions of accuracy, efficiency and safety respectively, and g i and h j is a constraint.

[0128] By using the NSGA-II algorithm to solve this multi-objective optimization problem, we can find the Pareto optimal solution set, which is the set of solutions that the generative large language model finds to find the optimal trade-off between accuracy, efficiency, and security.

[0129] Optionally, in some embodiments, interpretable analysis tools can be used to generate explanatory information for the generative large language model. Interpretable analysis tools refer to tools that can provide insight and understanding of model prediction results. The explanatory information generated by these tools can help users understand the inner workings of the generative large language model, how to make pending predictions, and the degree of influence of various features on the output of the generative large language model. The transparency of the generative large language model can be improved by using interpretable analysis tools to generate the explanatory language of the generative large language model. This not only helps users to deeply understand the potential risks of the generative large language model, but also facilitates the optimization of the model, improves the defense strategy, and improves the security of the model.

[0130] Optionally, in some embodiments, the generative large language model can be trained using a federated learning framework. The federated learning framework is a framework in which multiple data holders perform data training on local devices, and then upload model parameter updates to a central server for aggregation, thereby building a global model without directly sharing the data itself. Since there is no need to share the data itself, training the generative large language model using a federated learning framework can protect user data privacy. In addition, since the federated learning framework can aggregate a wide range of model parameters, the security of the model can be improved.

[0131] Optionally, in some embodiments, the interpretability analysis tool can be used to generate interpretation information of the generative large language model while the federated learning framework is used to train the generative large language model. This can protect the privacy of user data while greatly improving the security of the generative large language model.

[0132] Combination of the above Figures 1 to 3 , described the method embodiment of the present application in detail, and the device embodiment of the present application is described in detail below. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the part not described in detail can refer to the previous method embodiment.

[0133] An embodiment of the present application also provides a device for processing a model, including: a determination module, used to determine the risk score of a generative large language model according to a risk assessment model; a first update module, used to update the model parameters of an adversarial sample generator according to the risk score; a generation module, used to generate an adversarial sample according to the updated adversarial sample generator; a second update module, used to update the model parameters of the generative large language model according to the adversarial sample to generate an updated generative large language model.

[0134] Optionally, in some embodiments, the device further includes: a third updating module, used to update the model parameters of the security defense model according to the risk score; and a security defense module, used to perform security defense on the generative large language model according to the updated security defense model.

[0135] Optionally, in some embodiments, the security defense model includes at least one of the following: a first model for filtering input information of the generative large language model; a second model for detecting whether the intermediate information generated by the generative large language model contains abnormal information; a third model for detecting whether the output information generated by the generative large language model contains abnormal information;

[0136] Optionally, in some embodiments, the device further comprises: an optimization module, configured to optimize model parameters of the generative large language model using a multi-objective optimization algorithm to generate an optimized generative large language model.

[0137] Optionally, in some embodiments, the objective function of the multi-objective optimization algorithm includes at least one of the following functions: a first function for indicating a mapping relationship between model parameters of the generative large language model and the efficiency of the generative large language model; a second function for indicating a mapping relationship between model parameters of the generative large language model and the security of the generative large language model; a third function for indicating a mapping relationship between model parameters of the generative large language model and the prediction accuracy of the generative large language model.

[0138] Optionally, in some embodiments, the adversarial sample generator is an adversarial sample generator based on a WGAN structure, and the adversarial sample generator performs adversarial training based on the following objective function:

[0139]

[0140] Wherein, L represents the adversarial sample generator, G represents the generator in the adversarial sample generator, D represents the discriminator in the adversarial sample generator, x represents the real sample, z represents random noise, pdata(x) represents the data distribution of the real data, pz(z) represents the data distribution of random noise, λ represents the gradient penalty coefficient, is a random interpolation point between x and G(z), express data distribution.

[0141] Optionally, in some embodiments, the input sequence of the generative large language model is x=[x1, ..., xT], then the risk score is determined based on the following formula:

[0142] S(x)=∑ t a t ·f(h t );

[0143] h t =LSTM(x t ,h t―1 );

[0144] a t =softmax(v T tanh(W h h t +W x x t + b));

[0145] f(h t )=σ(W f h t +b f );

[0146] Where S(x) represents the risk score, LSTM represents the long short-term memory network, and h t represents the hidden state of LSTM, softmax represents the activation function, a t represents the attention weight, f(h t ) represents the risk assessment function corresponding to the risk assessment model, W f , W x , W h ,v,b,b f represents learnable parameters.

[0147] An embodiment of the present application also provides a device for processing a model, including: a memory for storing code; a processor for executing the code stored in the memory to execute the method for processing a generative large language model in any of the above embodiments.

[0148] An embodiment of the present application also provides a computer-readable storage medium on which is stored a code for executing the method for processing a generative large language model in any of the foregoing embodiments.

[0149] It should be understood that in the embodiment of the present application, determining B based on A does not mean determining B only based on A, but B can also be determined based on A and / or other information.

[0150] It should be understood that in the embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0151] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0152] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0156] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can read or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. Available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital video discs (DVD)), or semiconductor media (eg, solid state disks (SSD)).

[0157] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing models, It is characterized in that include: Determining a risk score of the generative large language model according to the risk assessment model; Updating model parameters of the adversarial sample generator according to the risk score; Generate adversarial samples based on the updated adversarial sample generator; The model parameters of the generative large language model are updated according to the adversarial sample to generate an updated generative large language model.

2. The method according to claim 1, It is characterized in that The method further comprises: Updating model parameters of the security defense model according to the risk score; According to the updated security defense model, security defense is performed on the generative large language model.

3. The method according to claim 2, It is characterized in that The security defense model includes at least one of the following: A first model, used to filter input information of the generative large language model; A second model is used to detect whether there is abnormal information in the intermediate information generated by the generative large language model; The third model is used to detect whether the output information generated by the generative large language model contains abnormal information.

4. The method according to any one of claims 1 to 3, It is characterized in that The method further comprises: The model parameters of the generative large language model are optimized using a multi-objective optimization algorithm to generate an optimized generative large language model.

5. The method according to claim 4, It is characterized in that The objective function of the multi-objective optimization algorithm includes at least one of the following functions: A first function, used to indicate a mapping relationship between a model parameter of the generative large language model and an efficiency of the generative large language model; A second function, used to indicate a mapping relationship between a model parameter of the generative large language model and a security of the generative large language model; A third function is used to indicate a mapping relationship between a model parameter of the generative large language model and a prediction accuracy of the generative large language model.

6. The method according to any one of claims 1 to 3, It is characterized in that The adversarial sample generator is an adversarial sample generator based on the WGAN structure, and the adversarial sample generator performs adversarial training based on the following objective function: Where L represents the adversarial sample generator, G represents the generator in the adversarial sample generator, D represents the discriminator in the adversarial sample generator, x represents the real sample, z represents random noise, and p data (x) represents the data distribution of real data, p z (z) represents the data distribution of random noise, λ represents the gradient penalty coefficient, is a random interpolation point between x and G(z), express data distribution.

7. The method according to any one of claims 1 to 3, It is characterized in that The input sequence of the generative large language model is x=[x 1 ,...,x T ], the risk score is determined based on the following formula: S(x)=∑ t a t ·f(h t ); h t =LSTM(x t ,h t―1 ); a t =softmax(v T tanh(W h h t +W x x t +b)); f(h t )=σ(W f h t +b f ); Where S(x) represents the risk score, LSTM represents the long short-term memory network, and h t represents the hidden state of LSTM, softmax represents the activation function, a t represents the attention weight, f(h t ) represents the risk assessment function corresponding to the risk assessment model, W f , W x , W h ,v,b,b f represents learnable parameters.

8. A device for processing a model, It is characterized in that include: A determination module, for determining a risk score of a generative large language model according to a risk assessment model; A first updating module, configured to update a model parameter of an adversarial sample generator according to the risk score; A generation module, used to generate adversarial samples according to the updated adversarial sample generator; The second updating module is used to update the model parameters of the generative large language model according to the adversarial sample to generate an updated generative large language model.

9. A device for processing a model, It is characterized in that include: A memory for storing codes; A processor, configured to execute the code stored in the memory to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that Code for executing the method according to any one of claims 1 to 7 is stored thereon.