Confrontation training method and system for electric power industry large model sample optimization

Abstract

The invention discloses an adversarial training method and system for electric power industry large model sample optimization, and belongs to the technical field of natural language processing, and the method specifically comprises the steps: collecting error samples, marking corresponding first answers, adjusting a large language model to generate second answers, outputting the second answers, and constructing a training data set # imgabs0 #; disturbance is added to an embedded layer and a Transform layer of the large language model, and sensitive Token is dynamically selected to apply disturbance based on a two-norm # imgabs1 # of Token-level disturbance; on the basis of the obtained training data set # imgabs2, forward propagation is carried out on the input of the large language model, and the loss of the first answer and the loss of the second answer are calculated; updating the disturbance direction through back propagation, and dynamically adjusting the applied Token-level disturbance according to the loss of the first answer and the second answer; and cross entropy is calculated according to correct input of the large language model for adversarial training.

Description

Background Art

[0002] Currently, there are mainly the following solutions for optimizing error samples in the operation of large models in the power industry: Solution 1: Optimization method based on identifying error examples This method uses semantic representation models such as traditional BERT models or BGE models based on large models to encode the input of error samples, map them to a high-dimensional semantic space, obtain the vectors of the input and store them in a vector library; then a filtering mechanism is established outside the model service, and it is judged whether the user's input hits the error examples in the vector library by means of similarity threshold retrieval; if it hits, a fixed correct reply is returned as the answer; the advantage of this method is that in the case of a small number of error samples, the optimization cost is small, it has a certain degree of interpretability and can ensure the accuracy of the answer after hitting. It is applicable to the scenario of optimizing a small number of special samples that are difficult to correct.

[0003] Solution 2: Optimization method based on large language model prompting engineering This method activates the capabilities of the large language model related to certain error examples by constructing prompts (prompt information) provided to the large language model. Then, combined with the instructions input by the user, the knowledge of the large language model itself is used to optimize specific error examples. The advantage of this method is that it can utilize the extensive knowledge of the large model in the learning process and stimulate the correction ability for specific examples. In addition, the optimization cost of this method is small and the scope of optimization is relatively wide, which is applicable to the optimization of some simple error examples.

[0004] Solution 3: Optimization method based on fine-tuning of error examples This method incorporates batches of error examples and their corresponding correct replies into the parameters of the model as training through full-scale fine-tuning or efficient partial parameter fine-tuning, and updates the model service. The advantage of this method is that it is applicable to all error samples, only the model needs to be updated, and no other modules need to be designed, which is applicable to the optimization of a large number of error samples.

[0005] The above three solutions have different usage scenarios, and the problems and deficiencies brought about are also different: Solution 1 based on the retrieval module is a general optimization approach in the industry for special samples. However, considering that in the field of power natural language processing, the user's input not only contains general questions but is very likely to be power professional questions. Therefore, this method has high requirements for the retrieval module and needs to ensure excellent retrieval effects in both general and power domains. In addition, the reply content of this method is relatively fixed and not suitable for solving error examples in large quantities.

[0006] The second solution, a method based on large language model prompting engineering, has two main drawbacks. One is that the capabilities of the large language model activated only through prompt information are limited in terms of optimizing for incorrect examples. On the other hand, since adding prompt words as input affects all input examples, the method of optimizing using prompt words requires fine-tuning of the prompt words and is more suitable for some simple general problems.

[0007] The third solution, an optimization method based on fine-tuning with incorrect examples, has a wide range of applications. However, since it is optimized based on model training, first, it is difficult to ensure that the model can answer all incorrect examples during large-scale optimization. If the proportion of incorrect examples is increased during the training process, the model is prone to overfitting to the samples, similar to the first solution. Second, the model training only utilizes the information of the correct answers and ignores the additional information provided by the incorrect examples output by the model. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide, in view of the deficiencies in the above-mentioned prior art, an adversarial training method and system for optimizing examples of large models in the power industry, which uses positive and negative samples of examples for adversarial training, thereby optimizing the output of the large model for incorrect examples and solving the technical problems of targeted optimization of incorrect examples and enhanced targeted robustness.

[0009] The present invention adopts the following technical solutions: An adversarial training method for optimizing examples of large models in the power industry, comprising the following steps: Collect incorrect examples and label the corresponding first answers, adjust the large language model to generate second answers, and construct a training data set based on the obtained first answers and second answers ; Add dynamic perturbations to the embedding layer of the large language model, add random perturbations to the Transformer layer, calculate the loss through forward propagation, then obtain the gradient of each perturbation through backpropagation, and dynamically select to apply perturbations to the embedding layer based on the second norm of the Token-level perturbation gradient through threshold determination; Based on the obtained training data set , perform forward propagation on the input of the large language model, calculate the loss between the first answer and the second answer; update the perturbation direction through backpropagation, dynamically select to apply perturbations based on the second norm of the Token-level perturbation gradient through threshold determination; add the updated direction and the dynamically screened token-level perturbations and the random perturbations applied to the Transformer layer during the training process; perform adversarial training for full parameter update by calculating the cross-entropy loss based on the input of the large language model and the first answer.

[0010] Preferably, collecting incorrect examples and labeling the corresponding first answers specifically is: Collect error example inputs of user feedback and label the corresponding first answers , by adjusting the Temperature parameter to increase the randomness of the large language model output, generate a list of second answer outputs that are inconsistent with the first answer.

[0011] Preferably, the training dataset is as follows:

[0012] Among them, , represents the th data,[[]] represents the input of the th data, represents the first answer of the th data, represents the xth second answer of the th data.

[0013] Preferably, adding dynamic perturbations to the embedding layer and Transformer layer of the large language model is specifically as follows: Apply dynamic perturbations to the Embedding layer of the large language model, and the perturbation direction is driven by the gradient difference between the correct output and the wrong output; inject random noise into the hidden representation of the Transformer layer in the middle and early stages of the large language model, and the noise intensity satisfies .

[0014] Preferably, set a threshold , and apply perturbations to the tokens that satisfy , and the rest of the tokens remain in their original representations.

[0015] Preferably, design perturbations of the same size according to the dimensions (l, d) of the input representation, and add random perturbations to the representation X of the example input to obtain the output of the large language model as . Calculate the gradient of the perturbation with respect to multiple second answer outputs W using the cross-entropy loss function .

[0016] Preferably, the update of the Token-level perturbation is as follows:

[0017] Among them, is the gradient of the first answer output R, is the average gradient of the perturbation with respect to the second answer, is the The gradient of the second answer output W, where is the number of second answers.

[0018] Preferably, the optimization objective for adversarial training is: After the input representation is perturbed, minimize the cross-entropy loss of the large language model's output of the first answer.

[0019] In a second aspect, an embodiment of the present invention provides an adversarial training system for optimizing large model examples in the power industry, including: A data module that collects incorrect examples and annotates the corresponding first answers, adjusts the large language model to generate second answers, and constructs a training data set based on the obtained first answers and second answers ; A perturbation module that adds dynamic perturbations to the embedding layer of the large language model, adds random perturbations to the Transformer layer, performs forward propagation to calculate the loss, and then obtains the gradient of each perturbation through backpropagation. Based on the two-norm of the Token-level perturbation gradient dynamically selects to apply perturbations to the embedding layer; An adversarial module that, based on the training data set obtained by the data module , performs forward propagation on the input of the large language model to calculate the losses of the first answer and the second answer; based on the two-norm of the Token-level perturbation gradient, dynamically selects to apply perturbations through threshold determination; adds an update direction during training and dynamically filters the token-level perturbations and the random perturbations applied to the Transformer layer; performs adversarial training for full parameter update by calculating the cross-entropy loss based on the input of the large language model and the first answer.

[0020] Preferably, in the perturbation module, adding dynamic perturbations to the embedding layer of the large language model specifically means: Applying dynamic perturbations to the Embedding layer of the large language model, where the perturbation direction is driven by the gradient difference between the correct output and the incorrect output; injecting random noise into the hidden representation of the Transformer layer in the middle and early stages of the large language model, and the noise intensity satisfies .

[0021] Preferably, set a threshold , and apply perturbations to the tokens that satisfy , and the remaining tokens remain in their original representations.

[0022] Preferably, design perturbations of the same size according to the dimensions (l, d) of the input representation, and add random perturbations to the representation X of the example input to obtain the output of the large language model as , and use the loss function of cross-entropy to calculate the perturbation Gradient of multiple second answer outputs W 。

[0023] Preferably, the Token-level perturbation is updated as follows:

[0024] wherein, is the gradient of the first answer output R, is the average gradient of the perturbation with respect to the second answer, is the gradient of the th second answer output W, is the number of second answers.

[0025] In a third aspect, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned adversarial training method for optimizing the power industry large model example are implemented.

[0026] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium including a computer program. When the computer program is executed by a processor, the steps of the above-mentioned adversarial training method for optimizing the power industry large model example are implemented.

[0027] In a fifth aspect, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned adversarial training method for optimizing the power industry large model example are implemented.

[0028] In a sixth aspect, an embodiment of the present invention provides an electronic device including a computer program. When the computer program is executed by the electronic device, the steps of the above-mentioned adversarial training method for optimizing the power industry large model example are implemented.

[0029] Compared with the prior art, the present invention has at least the following beneficial effects: An adversarial training method for optimizing the power industry large model example, By actively collecting error samples and adjusting Temperature parameters to generate adversarial data (S1), the model's ability to identify professional errors in the power field is significantly improved; a dynamic adjustment mechanism is used to ensure that the disturbance intensity is adaptively matched to the training process to avoid model distortion caused by excessive disturbance; adversarial training under cross-entropy constraints balances robustness and generation quality to ensure the professionalism of power text output; the present invention uses an innovative dynamic adversarial mechanism to significantly improve the defense capability against complex adversarial samples while maintaining the professional generation capability of large power models, reducing the error rate by more than 60%, and providing a highly reliable language model solution for key scenarios such as smart grids and power dispatching.

[0030] Furthermore, a data set is constructed based on error samples actually fed back by users to accurately locate high-frequency error patterns in power business scenarios. By organically combining real error capture with intelligent generation, a more comprehensive decision boundary is constructed while ensuring the accuracy of power knowledge. Experiments have shown that the error recognition accuracy of the model in tasks such as power grid fault diagnosis and operation ticket generation can be improved by 42%, while the misuse rate of professional terminology can be reduced to below 1.3%.

[0031] Furthermore, real-time alignment of the adversarial direction and the model decision boundary changes is achieved, and vulnerable semantic space areas are automatically identified through gradient symbol analysis to enhance robustness in a targeted manner.

[0032] Furthermore, the perturbation is ensured to be aligned with the original input space to avoid semantic damage caused by dimension mismatch. The embedding vector of each Token is perturbed independently to achieve accurate positioning of local sensitive areas while maintaining the overall structure of the sequence.

[0033] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0034] In summary, the present invention aims to improve the robustness and generalization ability of the large preview model in the power industry, especially its performance in dealing with erroneous samples.

[0035] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1It is the step diagram of the adversarial training of the present invention; Figure 2 It is the schematic flow diagram of the error sample optimization of the present invention; Figure 3 It is the schematic diagram of the computer device provided by an embodiment of the present invention.

[0038] Figure 4 It is the block diagram of an electronic device provided according to an embodiment of the present invention.

[0039] Among them, 60. Computer device; 61. Processor; 62. Memory; 63. Computer program; 600. Electronic device; 610. Processing unit; 620. Storage unit; 6201. Random access storage unit; 6202. Cache storage unit; 6203. Read-only storage unit; 6204. Program / utilities; 6205. Program module; 630. Bus; 640. Display unit; 650. Input / output interface; 660. Network adapter; 700. External device. Detailed implementation manners

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] In the description of the present invention, it should be understood that the terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0042] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0043] It should be further understood that the term " / and" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present invention generally represents an "or" relationship between the contextually related objects.

[0044] It should be understood that although terms such as first, second, and third may be used in the embodiments of the present invention to describe preset ranges, etc., these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0045] Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detected (stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detected (stated condition or event)" or "in response to detecting (stated condition or event)".

[0046] Various schematic structural diagrams according to the disclosed embodiments of the present invention are shown in the drawings. These figures are not drawn to scale, where for the purpose of clear expression, some details are enlarged and some details may be omitted. The shapes of various regions and layers shown in the figures and their relative sizes and positional relationships are merely exemplary, and may actually deviate due to manufacturing tolerances or technical limitations, and those skilled in the art can design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0047] The present invention provides an adversarial training method optimized for power industry large model examples, which collects incorrect examples and annotates the first answer, adjusts the large language model to generate the second answer, and constructs a training data set D based on the two; adds dynamic perturbations to the embedding layer, calculates the loss through forward propagation, obtains the perturbation gradient through backpropagation, and dynamically selects perturbations based on the second norm and threshold; uses the training data set D to perform forward propagation on the input of the large language model, calculates and updates the answer loss, optimizes the perturbation direction through backpropagation, and dynamically screens and applies perturbations at the same time; combines Token-level perturbations and self-attention mechanism perturbations, and performs full-parameter adversarial training according to cross-entropy calculation. Through dynamic perturbations, gradient updates, and adversarial training, the performance of the power industry large model is optimized, especially for enhanced training when dealing with incorrect examples.

[0048] Embodiment 1 An adversarial training method optimized for power industry large model examples according to the present invention includes the following steps: S1. Construct data; Collect incorrect examples: Using the input of each incorrect example fed back by the user, obtain the corresponding correct answer through manual annotation or other annotation methods As the first answer; Generate error output: By changing the Temperature of the original large language model, obtain the error outputs of multiple large language models as the second answers, and then obtain the data set required for training. , where , represents the th data, represents the input of the th data, represents the first answer of the th data, represents the x-th second answer of the

[0049] S2. Design perturbations; S201. Add perturbations Embedding layer perturbations: For large language models with a Decoder-only architecture, referring to the methods of Neftune and FreeLB, add a dynamic perturbation to the embedding layer (Embedding layer) of the large language model to alleviate the sensitivity of the large language model to some rare power-related professional terms.

[0050] Transformer layer perturbations: In addition, considering that it may be necessary to oversample error examples during training, in order to reduce the overfitting of the model to the input of error examples, draw on the research experience of early models with a Transformer architecture, that is, the representations of the first few layers of the Transformer architecture model are more biased towards the syntactic structure of the sentence, while the representations of the middle layers have shallow semantic information.

[0051] Therefore, random perturbations are added to the middle and early Transformer layers of the large language model, aiming to perturb the syntactic structure and early semantics of the example input, increase the robustness of the large language model to the shallow semantics of the input, and improve the generalization ability for error examples.

[0052] S202. Design Token-level perturbations Perturbation size setting: In terms of setting the perturbation size, perturbations of the same shape are designed according to the dimensions (l, d) of the input representation, while ensuring that the Token-level perturbations and hierarchical perturbations

[0053] Perturbation direction setting: In terms of setting the direction of perturbation, specifically design adversarial perturbations for the second answer (wrong example) and the first answer (correct example); specifically, add random perturbations to the representation X of the example input to obtain the output representation of the large language model and use the loss function of cross entropy to calculate the perturbation with respect to the gradients of the multiple second answer outputs W ; at the same time, the perturbation with respect to the gradients of the first answer output R will also be calculated ; finally, update the direction of the perturbation as follows:

[0054] It is worth mentioning that when calculating the perturbation direction, the large language model only calculates the gradients and does not perform parameter updates.

[0055] S203. Select dynamic token perturbation Sensitive Token Judgment: Considering that the large model has excellent effects on general questions, it is considered that the reasons for wrong examples are more due to insufficient learning of some domain-related tokens and being too sensitive to professional questions, rather than a general problem of all tokens in the overall model Embedding space.

[0056] Therefore, judge the sensitivity degree of the token to the correctness of the output according to the L2 norm of the token-level perturbation .

[0057] For tokens that are not sensitive to the second answer, there is no need to perform unnecessary perturbations on them, and the remaining tokens remain the original representation.

[0058] S3. Training algorithm.

[0059] Forward propagation: Based on the training data set , for each input, first perform a forward propagation to calculate the losses for the first answer and multiple second answers.

[0060] Backward propagation: Based on the losses, backpropagate to calculate the gradients with respect to the perturbations, and use the gradient directions and magnitudes to confirm the dynamic perturbations for each token.

[0061] Add perturbation training: Finally, add this perturbation to the representation of the input, and add random-level perturbations to the Transformer layer, and perform training by calculating the cross entropy using the data set of the first answer part.

[0062] In addition, for hierarchical perturbations, no specific perturbation direction design is considered.

[0063] The optimization objective for adversarial training is as follows: After perturbing the input representation, minimize the cross-entropy loss of the large language model's output of the first answer.

[0064] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, method, or program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "platform" here.

[0065] Example 2 The present invention provides an adversarial training system for optimizing large model examples in the power industry, which can be used to implement the above-mentioned adversarial training method for optimizing large model examples in the power industry. Specifically, the adversarial training system for optimizing large model examples in the power industry includes a data module, a perturbation module, and an adversarial module.

[0066] Among them, the data module collects error examples and annotates the corresponding first answers, adjusts the large language model to generate second answers, and constructs a training data set based on the obtained first answers and second answers ; The perturbation module adds dynamic perturbations to the embedding layer of the large language model, adds random perturbations to the Transformer layer, calculates the loss through forward propagation, then obtains the gradient of each perturbation through backpropagation, and dynamically selects to apply perturbations to the embedding layer based on the two-norm of the Token-level perturbation gradient ; The adversarial module, based on the training data set obtained by the data module , performs forward propagation on the input of the large language model, and calculates the loss between the first answer and the second answer; dynamically selects to apply perturbations through threshold determination based on the two-norm of the Token-level perturbation gradient; adds an update direction during training and dynamically screens the token-level perturbations and the random perturbations applied to the Transformer layer; performs adversarial training for full parameter update by calculating the cross-entropy loss based on the input of the large language model and the first answer.

[0067] Example 3 The present invention provides a terminal device, which includes a processor and a memory. The memory is used to store a computer program, and the computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Graphics Processing Units (GPU), Tensor Processing Units (TPU), Digital Signal Processors (DSP), Application Specific Integrated Circuits (ASIC), Field-Programmable Gate Arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function. The processor described in the embodiments of the present invention can be used for the operations of the adversarial training method optimized for the large model samples in the power industry, including: Collecting error samples and annotating the corresponding first answers, adjusting the large language model to generate second answers, and constructing a training data set based on the obtained first answers and second answers ; adding dynamic perturbations to the embedding layer of the large language model, adding random perturbations to the Transformer layer, performing forward propagation to calculate the loss, then obtaining the gradient of each perturbation through backpropagation, and dynamically selecting to apply perturbations to the embedding layer based on the threshold determination of the second norm of the Token-level perturbation gradient; based on the obtained training data set Performing forward propagation on the input of the large language model to calculate the loss between the first answer and the second answer; updating the perturbation direction through backpropagation, and dynamically selecting to apply perturbations based on the threshold determination of the second norm of the Token-level perturbation gradient; adding the updated direction and the dynamically screened token-level perturbations and the random perturbations applied to the Transformer layer of the large language model during the training process, and performing adversarial training for full parameter update by calculating the cross-entropy loss based on the input of the large language model and the first answer.

[0068] Please refer to Figure 3, the terminal device is a computer device. The computer device 60 in this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, it implements the adversarial training method for optimizing the large model samples in the power industry in the embodiment. To avoid repetition, it will not be elaborated here one by one. Alternatively, when the computer program 63 is executed by the processor 61, it implements the functions of each model / unit in the adversarial training system for optimizing the large model samples in the power industry in the embodiment. To avoid repetition, it will not be elaborated here one by one.

[0069] The computer device 60 can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art can understand that Figure 3 These are merely examples of the computer device 60 and do not constitute a limitation on the computer device 60. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0070] The so-called processor 61 may be a central processing unit (CPU), or may also be other general-purpose processors, a graphics processing unit (GPU), a tensor processing unit (TPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0071] The memory 62 may be an internal storage unit of the computer device 60, such as the hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk equipped on the computer device 60, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0072] Further, the memory 62 may also include both the internal storage unit of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0073] Please refer to Figure 4 , the terminal device is the electronic device 600, and the electronic device 600 is presented in the form of a general computing device. The components of the electronic device may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0074] Among them, the storage unit stores program codes, and the program codes can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above method part of this specification. For example, the processing unit 610 may execute the steps as shown in Figure 1 .

[0075] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0076] The storage unit 620 may also include a program / utilities 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0077] The bus 630 may represent one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any one of the multiple bus structures.

[0078] The electronic device 600 can also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem). Such communication can be carried out through the input / output interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network, a wide area network, and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0079] Embodiment 4 The present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device, and of course can also include the extended storage medium supported by the terminal device. It can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. The computer-readable storage medium provides a storage space that stores the operating system of the terminal. And, in this storage space, one or more instructions suitable for being loaded and executed by the processor are also stored. These instructions can be one or more computer programs (including program codes). It should be noted that more specific examples of the computer-readable storage medium here include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0080] The computer-readable storage medium also includes a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the computer-readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, radio frequency, etc., or any suitable combination of the above.

[0081] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0082] One or more instructions stored in the computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the adversarial training method for optimizing the large model sample in the power industry in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps: Collect error samples and label the corresponding first answers, adjust the large language model to generate second answers, and construct a training data set based on the obtained first answers and second answers ; Add dynamic perturbations to the embedding layer of the large language model, add random perturbations to the Transformer layer, perform forward propagation to calculate the loss, and then obtain the gradient of each perturbation through backpropagation. Based on the second norm of the Token-level perturbation gradient, dynamically select to apply perturbations to the embedding layer through threshold determination; based on the obtained training data set Perform forward propagation on the input of the large language model to calculate the losses of the first answer and the second answer; update the perturbation direction through backpropagation, and dynamically select the perturbation to be applied based on the second norm of the Token-level perturbation gradient through threshold determination; during training, add the updated direction and the dynamically screened Token-level perturbation and the random perturbation applied to the Transformer layer of the large language model, and perform adversarial training for full parameter update by calculating the cross-entropy loss based on the input of the large language model and the first answer.

[0083] In the embodiments provided in this application, the databases involved may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., and is not limited thereto. The processors involved in the embodiments provided in this application may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., and are not limited thereto.

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0085] Adversarial Training Method for Optimizing Industry Large Model Examples Input: Training sample D, perturbation boundary , learning rate lr; Initialization: Model parameters ; for epoch = 1...N do for miniBatch B D do # Perturbation direction calculation ← # Initialize perturbation # Calculate perturbation direction for incorrect example # Calculate perturbation direction for correct example

[0086] #Dynamic perturbation & truncation based on perturbation size # Calculate loss ← # Initialize layer perturbations #Calculate gradients #Update model parameters end for end for See also Figure 1 , where orange represents the perturbation process (calculating the size and direction of the perturbation, and selecting the perturbation token), and blue represents the model training process. is the selected perturbation, and the dotted line is the unselected disturbance.

[0087] For the inference stage, the method of the present invention does not add disturbances during the inference stage, and thus does not affect the inference process of the large language model.

[0088] See also Figure 2 , the blue part is the perturbation calculation process, and the orange part is the large language model parameter update process.

[0089] In summary, the adversarial training method and system for optimizing large model samples in the power industry of the present invention have the following characteristics: 1. Targeted sample error optimization Compared with the traditional fine-tuning optimization method, this invention makes more use of the erroneous results of the model output, and adopts an adversarial mindset by adding a directional perturbation that is biased towards the wrong sample and away from the correct sample on the Embedding layer of the model to confront the model during the training process. This method can not only increase the probability of the model outputting the correct answer during the training process, but also reduce the probability of the model outputting the wrong answer, thereby achieving the goal of targeted optimization.

[0090] 2. Improve the generalization of the model to error examples Related work has found that the representation of the early and middle Transformer layers contains shallow semantic information. In addition, in the process of corrective training, oversampling is usually used to train error samples in a targeted manner. Therefore, referring to previous experience, random perturbations are added to the representation obtained in the early and middle Transformer layers to try to generalize the shallow semantics of the input to a certain extent, thereby reducing the phenomenon of deliberate memorization.

[0091] 3. Dynamic Domain Token Perturbation Not all tokens in the domain model are sensitive to incorrect examples because there are differences in the training sufficiency of professional domain vocabulary during pre-training and continued pre-training. Therefore, performing adversarial training on all tokens without thinking will not only increase the computational cost but also have a negative impact on the performance. So, dynamically screen the tokens to be perturbed according to the perturbation gradient, which can reduce the perturbation of the well-trained part of the original model while ensuring an increase in the model's robustness.

[0092] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0093] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present invention can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0095] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal and method can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0096] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0099] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus, and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one or more flows and / or one or more blocks Figure 1 in one or more flows and / or one or more blocks Figure 1 in one or more blocks.

[0100] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one or more flows and / or one or more blocks Figure 1 in one or more flows and / or one or more blocks Figure 1 in one or more blocks.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows and / or one or more blocks Figure 1 in one or more flows and / or one or more blocks Figure 1 in one or more blocks.

[0102] The above content is only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any modification made on the basis of the technical solution according to the technical idea proposed by the present invention falls within the protection scope of the claims of the present invention.

Claims

1. An adversarial training method for optimizing large model examples in the power industry, characterized in that: The following steps are involved: Collect incorrect examples and label the corresponding first answers, adjust the large language model to generate the second answer, and build a training dataset based on the first and second answers. ; Dynamic perturbations are added to the embedding layer of the large language model, random perturbations are added to the Transformer layer, and forward propagation is performed to calculate the loss. Then, the gradient of each perturbation is obtained through backpropagation. Based on the binary norm of the perturbation gradient at the token level, the perturbation applied to the embedding layer is dynamically selected through threshold determination. Based on the obtained training data set The input of the large language model is forward propagated to calculate the loss of the first answer and the second answer; the perturbation direction is updated through backpropagation, and the perturbation is dynamically selected based on the second norm of the Token-level perturbation gradient through threshold judgment; during the training process, the update direction is increased, the token-level perturbation after dynamic screening and the random perturbation applied to the Transformer layer of the large language model are added, and the cross-entropy loss is calculated based on the input of the large language model and the first answer to perform adversarial training with full parameter update.

2. The adversarial training method for optimizing large model examples in the power industry according to claim 1 is characterized in that: Collect the wrong examples and mark the corresponding first answers: Collect incorrect sample inputs from user feedback and mark the corresponding first answer By adjusting the Temperature parameter, the randomness of the large language model output is increased to generate the same answer as the first answer. The inconsistent second answer outputs a list.

3. The adversarial training method for optimizing large model examples in the power industry according to claim 2 is characterized in that: Training Dataset The details are as follows: in, , Representative Items of data, Indicates The input of data, Indicates The first answer to the data, Indicates The xth second answer to the data item.

4. The adversarial training method for optimizing large model examples in the power industry according to claim 1 is characterized in that: The specific steps of adding dynamic perturbations to the embedding layer of the large language model are: Dynamic perturbations are applied to the Embedding layer of the large language model, and the perturbation direction is driven by the gradient difference between the correct output and the wrong output; random noise is injected into the hidden representation of the Transformer layer in the early and middle stages of the large language model, and the noise intensity satisfies .

5. The adversarial training method for optimizing large model examples in the power industry according to claim 4 is characterized in that: Setting Thresholds , to satisfy The tokens are perturbed and the other tokens remain in their original representation.

6. The adversarial training method for optimizing large model examples in the power industry according to claim 4 is characterized in that: Design perturbations of the same size according to the dimensions (l, d) of the input representation, by adding random perturbations to the representation X of the sample input , and the output of the large language model is , using the cross entropy loss function to calculate the perturbation Gradients of the output W of multiple second answers .

7. The adversarial training method for optimizing large model examples in the power industry according to claim 6 is characterized in that: Token-level disturbance The updates are as follows: in, is the gradient of the correct sample output R, is the mean gradient of the perturbation to the second answer, For the The second answer outputs the gradient of W, is the number of second answers.

8. The adversarial training method for optimizing large model examples in the power industry according to claim 1 is characterized in that: The optimization goal of adversarial training is: Minimize the cross entropy loss of the first answer output by the large language model after the input representation is perturbed.

9. An adversarial training system optimized for large model examples in the power industry, characterized in that: include: Data module, collects error samples and marks the corresponding first answers, adjusts the large language model to generate the second answer, and builds a training data set based on the first and second answers ; The perturbation module adds dynamic perturbations to the embedding layer of the large language model and random perturbations to the Transformer layer. It performs forward propagation to calculate the loss, and then obtains the gradient of each perturbation through back propagation. The perturbation gradient at the token level is based on the binary norm. Dynamically choose to apply perturbations to the embedding layer; Adversarial module, based on the training data set obtained by the data module , forward propagate the input of the large language model, calculate the loss of the first answer and the second answer; based on the second norm of the Token-level perturbation gradient, dynamically select the perturbation to be applied through the threshold judgment; during the training process, increase the update direction and dynamically screen the token-level perturbation and the random perturbation applied to the Transformer layer; calculate the cross-entropy loss based on the input of the large language model and the first answer to perform adversarial training with full parameter update.

10. The adversarial training system for optimizing large model examples in the power industry according to claim 9, characterized in that: In the perturbation module, dynamic perturbations are added to the embedding layer and Transformer layer of the large language model as follows: Dynamic perturbations are applied to the Embedding layer of the large language model, and the perturbation direction is driven by the gradient difference between the correct output and the wrong output; random noise is injected into the hidden representation of the Transformer layer in the early and middle stages of the large language model, and the noise intensity satisfies .

11. The adversarial training system for optimizing large model examples in the power industry according to claim 10, characterized in that: Setting Thresholds , to satisfy The tokens are perturbed and the other tokens remain in their original representation.

12. The adversarial training system for optimizing large model examples in the power industry according to claim 11, characterized in that: Design perturbations of the same size according to the dimensions (l, d) of the input representation, by adding random perturbations to the representation X of the sample input , and the output of the large language model is , using the cross entropy loss function to calculate the perturbation Gradient of the output W with respect to multiple error samples .

13. The adversarial training method for optimizing large model examples in the power industry according to claim 12 is characterized in that: Token-level disturbance The updates are as follows: in, is the gradient of the correct sample output R, is the mean gradient of the perturbation to the error output, For the The gradient of the output W of the error sample is, is the number of error samples.

14. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 8.

15. A computing device, characterized in that: include: One or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the method according to any one of claims 1 to 8.

Images