Power industry large model continuous pre-training method and system based on dynamic self-constraint

Abstract

The invention discloses a power industry large model continuous pre-training method and system based on dynamic self-constraint, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining industry pre-training corpora and instruction training corpora, dynamically adjusting the mixing ratio of the industry pre-training corpora and the instruction training corpora through a curriculum-type strategy, and obtaining an industry pre-training corpora and an instruction training corpora; obtaining a dynamic mixed data set; configuring a reference model based on the dynamic mixed data set, and training a target power industry large model by adopting a differential loss function and the reference model for different types of data in the dynamic mixed data set; according to the differential loss function, self-adaptive KL divergence is calculated according to inter-partition optimization logic, and the self-adaptive KL divergence is adopted to construct a loss function; and obtaining the probability of the reference model through an online reasoning framework, and enabling the training to be continuously carried out based on the probability of the reference model. According to the method, the knowledge conflict problem in professional field training is effectively solved, and the generality of the model is kept while the professional property of the power field is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a power industry large model continued pre-training method and system based on dynamic self-constraint. BACKGROUND

[0002] Large language models (LLMs) have made significant progress in the field of natural language processing (NLP), and super-large scale pre-training models have gradually become the mainstream direction of artificial intelligence research. However, most current language models are still general-purpose models, and there are problems such as insufficient understanding of industry knowledge and lack of professionalism. This makes the importance of industry large language model training increasingly prominent. Industry large models can learn complex knowledge systems, professional terms and business processes in specific fields, thereby providing more accurate and efficient solutions for industry users.

[0003] The training method of large models is to use unsupervised data to let the model learn to predict the content of the next context based on the previous context. Through cross-entropy loss, the probability value of the model predicting the next correct token is improved, and the model parameters are updated. In the training process of industry large models, this method is still used for training.

[0004] In order to inject power knowledge and retain general-purpose capabilities, traditional training schemes adjust the proportion of industry data and general data in the training data to achieve this. By changing the proportion of general data, the model still learns general knowledge during training, thereby maintaining the general-purpose capabilities of the industry model. The traditional method uses hard labels for learning during unsupervised training, ignoring the relevance between words. Intuitively, the prediction of the next token is not absolute, but a probability distribution. Therefore, rich distribution information between words is missing, and overfitting phenomenon is easy to occur, causing conflicts between industry knowledge and general knowledge.

[0005] In the process of continued pre-training of the power industry large model, a multi-stage training method can be used, and annealing instructions are used for training. After the injection of industry knowledge is completed, high-quality instruction data is collected for annealing training. Specifically, by gradually reducing the learning rate from the highest value to zero, the model parameters are gradually fine-tuned in finer granularity, thereby improving the general-purpose capabilities of the model. The annealing training method also uses hard labels for learning, ignoring the relevance between words. In addition, the training data distribution in the annealing stage and the continued pre-training process is quite different, which is easy to cause instability in training and forgetting of industry knowledge. SUMMARY

[0006] The purpose of the present application is to provide a dynamic self-constraint based power industry large model continuous pre-training method and system, which effectively solves the knowledge conflict problem in professional field training, improves the professionalism of the power field while maintaining the general performance of the model, and is particularly suitable for deployment scenarios with limited computing power architecture.

[0007] To achieve the above purpose, the present application provides the following technical solutions: The present application proposes a dynamic self-constraint based power industry large model continuous pre-training method, which comprises: Obtain industry pre-training corpus and instruction training corpus, dynamically adjust the mixing ratio of industry pre-training corpus and instruction training corpus through course strategy, and obtain dynamic mixed dataset; Based on the dynamic mixed dataset, configure a reference model, use a differential loss function and a reference model for different types of data in the dynamic mixed dataset, and train a target power industry large model; The differential loss function is calculated according to the adaptive KL divergence of the partition optimization logic, and the adaptive KL divergence is used to construct the loss function; During the training process, an online inference framework is constructed based on vLLM, the probability of the reference model is obtained through the online inference framework, and the training is continuously performed based on the probability of the reference model.

[0008] As a further improvement of the present application, the industry pre-training corpus contains general knowledge sub-corpus and power industry professional sub-corpus, and the scale is logarithmic B tokens.

[0009] As a further improvement of the present application, the course strategy is specifically: Let the number of training samples of the industry pre-training corpus be , the number of training samples of the instruction training corpus be , and the training batch size be B, then the instruction training sample in the i-th step Batch is:

[0010] Where a, b, L are hyperparameters, L defines the number of steps for starting to inject instruction corpus, and a, b jointly control the injection rate and amplitude of instruction corpus.

[0011] As a further improvement of the present application, the reference model comprises a constraint model for maintaining the general ability of the model and a teacher model for optimizing the instruction following ability of the model; The constraint model is a general field pre-training large language model, and the teacher model is a dialog large language model fine-tuned by instructions.

[0012] As a further improvement of the present application, the differential loss function is specifically: The general knowledge sub-corpus and the instruction training corpus in the industry pre-training corpus are increased with a loss function based on adaptive KL divergence training The power industry professional sub-corpus in the industry pre-training corpus is trained with a language model-based loss function training Total training loss of the target model + Wherein, α is the weight hyperparameter of the KL divergence loss function.

[0013] As a further improvement of the present application, the adaptive KL divergence is determined by the partition optimization logic, and the loss function is constructed by using the adaptive KL divergence, comprising: Using adaptive KL divergence to fit the model and the data of the reference model The adaptive KL divergence uses forward KL divergence FKL and reverse KL divergence RKL to process the probability of the head and tail respectively:

[0014]

[0015] Wherein, h, m, t are the number of head, middle and tail tokens respectively, are the token sets of the head, middle and tail respectively; for the judgment of token x, the mean and the standard deviation are used for judgment, specifically: Head token: token; Tail token: token; Middle token: other tokens; the tokens are sorted according to size, and part of the tokens are selected for calculation.

[0016] As a further improvement of the present application, when fitting the data of the model and the reference model , the probability value is accurate to the thousandth, and the accuracy below the thousandth is discarded.

[0017] As a further improvement of the present application, the construction method of the online inference framework is specifically: Deploy the reference model to the inference cluster independent of the training cluster by using vLLM, and configure the load balancing mechanism Passing token information by the ID of the token, a fixed decoding stage only outputs a single predicted token, and a reverse token processing stage is removed. Using an asynchronous data loader of a training process, a reference model probability is obtained in advance from an inference cluster, and the training process and the reference model probability obtaining process are asynchronously parallel.

[0018] The application also proposes a power industry large model continued pre-training system based on dynamic self-constraint, based on the power industry large model continued pre-training method based on dynamic self-constraint, comprising: A dynamic data set module is used to obtain industry pre-training corpus and instruction training corpus, dynamically adjust the mixing ratio of the industry pre-training corpus and the instruction training corpus through a course strategy, and obtain a dynamic mixed data set. A dynamic self-constraint training module is used to configure a reference model based on the dynamic mixed data set, use a differential loss function and a reference model for different types of data in the dynamic mixed data set, and train a target power industry large model. An adaptive KL divergence calculation module is used to calculate adaptive KL divergence according to partition optimization logic, and construct a loss function using adaptive KL divergence. An online inference deployment module is used to construct an online inference framework based on vLLM during training, obtain the probability of the reference model through the online inference framework, and make the training continue based on the probability of the reference model.

[0019] The application also proposes an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned power industry large model continued pre-training method based on dynamic self-constraint when executing the computer program.

[0020] The application also proposes a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the above-mentioned power industry large model continued pre-training method based on dynamic self-constraint.

[0021] The application also proposes a computer program product comprising computer instructions instructing a computer to execute the above-mentioned power industry large model continued pre-training method based on dynamic self-constraint.

[0022] Compared with the prior art, the application has the following advantages: The application provides a power industry large model dynamic self-constraint continued pre-training method, which is trained by fusing power and general corpus. The reference model and the teacher model provide knowledge guidance, and the dynamic constraint mechanism balances the improvement of professional ability and the retention of general knowledge. The training process and knowledge calculation are decoupled, additional knowledge is provided through online calculation, which reduces the storage pressure and adapts to the heterogeneous computing power environment. This method effectively solves the knowledge conflict problem in professional field training, improves the professionalism of the power field while maintaining the general performance of the model, and is particularly suitable for deployment scenarios with limited computing power architecture. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The figure is a schematic diagram of an industry continued pre-training algorithm design; Figure 2 The figure is a schematic diagram of dynamic updating of instruction following data proportion in Batch with training step number; Figure 3 The figure is a schematic diagram of the overall training framework. DETAILED DESCRIPTION

[0024] The technical solutions in the application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. The components of the application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. Based on the embodiments of the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0025] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the application, the terms first, second, etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0026] Term explanation: LLMs: Large Language Models; large language models are neural network models based on deep learning.

[0027] NLP: Natural Language Processing; natural language processing builds a communication bridge between human natural language and computer, enabling computers to have the ability to process, analyze, understand and generate human language.

[0028] vLLM: An efficient large-scale language model inference engine based on the Transformer architecture KL divergence: Kullback-Leibler divergence.

[0029] The power industry is accelerating the transformation towards intelligence, and large language models (LLM) are increasingly being applied in grid dispatching, equipment maintenance, customer service interactions, and other scenarios. However, existing general-purpose large models have insufficient professional knowledge in the power industry, high training costs, and high computational power requirements.

[0030] The first object of the present application is to provide a method for dynamic self-constrained power industry large model continued pre-training based on dynamic self-constraint technology, comprising the following steps: Obtain industry pre-training corpus and instruction training corpus, dynamically adjust the mixing ratio of industry pre-training corpus and instruction training corpus through course strategy, and obtain dynamic mixed dataset; Configure a reference model based on the dynamic mixed dataset, use a differentiated loss function and reference model for different types of data in the dynamic mixed dataset, and train a target power industry large model; The differentiated loss function is to calculate adaptive KL divergence according to the partition optimization logic, and to construct a loss function using adaptive KL divergence; During the training process, an online inference framework is constructed based on vLLM, the probability of the reference model is obtained through the online inference framework, and the training is continuously performed based on the probability of the reference model.

[0031] The dynamic self-constrained continued pre-training method proposed in the present application can be applied to the continued pre-training of power industry large models, and can inject power industry knowledge while alleviating the decline in general capabilities. The knowledge calculation decoupling can also be applied to heterogeneous training power environments to alleviate the shortage of computing resources. This method ensures the model's instruction following ability while injecting power industry knowledge through dynamic constraints and mixed training stages, and maintains the general knowledge and instruction following ability, which is beneficial for subsequent instruction training and reinforcement learning.

[0032] This method dynamically fuses industry pre-training corpus and instruction corpus in the industry stage to ensure the model's instruction following ability and alleviate data distribution differences; at the same time, the reference model and teacher model provide additional knowledge for the training process, constrain and guide the model's update, and ensure the model's general knowledge and instruction following ability.

[0033] In scenarios where the mixing ratio of industry pre-training corpora and instruction training corpora is dynamically adjusted, the curriculum-based strategy is akin to curriculum learning in the field of large-scale model training. It mimics the human learning path from easy to difficult, from basic to advanced, breaking down model training into multiple stages, each with a clearly defined training objective. Then, it dynamically adjusts the mixing ratio of industry pre-training corpora and instruction training corpora based on the model's training status, thereby efficiently building model capabilities, reducing training costs, and improving training effectiveness.

[0034] Furthermore, the framework implemented by this method employs online computation during the additional knowledge provision phase, significantly reducing the pressure on storage space caused by the additional knowledge. Moreover, the framework uses a decoupling approach between training and knowledge computation, alleviating the pressure on computing power and memory, as well as the requirement for homogeneous computing power, enabling it to support scenarios with mixed and constrained computing power architectures.

[0035] This application proposes a dynamic self-constraint-based method for continued pre-training of a large-scale power industry model. By dynamically fusing industry pre-training corpora and instruction corpora during the continued pre-training phase, the method ensures the model's instruction compliance capability. Simultaneously, it utilizes reference and teacher models to provide additional knowledge to the training process, guaranteeing the model's generalizability. The main technical implementations are as follows: Figure 1 As shown, specifically: Step 1, Dynamic Dataset Design Steps: In this application, the training dataset incorporates industry pre-trained corpora. and instruction training corpus Among them, the industry pre-training corpus Includes general parts and the power industry The size is typically several gigabytes (B) of tokens, with a high density of industry knowledge. Meanwhile, the instruction training corpus... Typically, there are only tens of thousands of data points, with a focus on optimizing instruction compliance. The size and distribution of the two datasets differ significantly.

[0036] This application provides a method for dynamically increasing the proportion of instruction data using a curriculum-based strategy, assuming... The number of training samples is , The number of training samples is If the training batch size is B, then the training samples for the instructions in the i-th batch are:

[0037] Where a, b, and L are hyperparameters, representing the injection of instruction training samples from step L, and parameters a and b correspond to the variations in the IT training sample injection. Overall, maintain The training corpus has a small amount of repetition, such asFigure 2 As shown. This avoids the drastic changes in data distribution that traditional methods bring.

[0038] Step 2, Dynamic Self-Constraint Design Steps: This application designs a dynamic self-constraint method to provide additional knowledge for constraint and guidance during the industry's continued pre-training process. The loss function of traditional industry continued pre-training is:

[0039] Where L is the length of sample X, For the model The output probability value.

[0040] The dynamic self-constraint method designed in this application constrains and guides the model's general capabilities and instruction compliance capabilities, respectively. First, the base model and the dialogue model are designed as constraint models. Teacher Model (Hereafter referred to as the reference model) ), and use this to train a large-scale model for the power industry. .

[0041] Specifically, for industry pre-training corpora The data in the general part adopts Constrain the model's updates to maintain its generalizability. For example... Figure 3 The diagram shown illustrates the overall training framework. For the instruction data in the training set, this application will use the teacher model... The ability to follow instructions is used to train a large-scale power industry model through knowledge distillation. .

[0042] In both of the above sections, this application adds adaptive KL divergence (Kullback-Leibler Divergence) as a loss function to train the model. The specific training loss function is as follows:

[0043] +

[0044] in Dynamically adopt based on data category and , The hyperparameter for the KL divergence loss ratio is... This is a novel algorithm proposed in this application. T is the length of a single training example. This represents the token probability distribution output by the target model. (Based on the industry pre-training corpus.) The industry data in the middle adopts Train the model as a loss function.

[0045] Step 3, adaptive KL divergence design step: Because the calculation of KL divergence has directionality, when using KL divergence to fit the model and the data distribution of the reference model , this application proposes an adaptive KL divergence. The head and tail probabilities are processed using forward KL divergence FKL and reverse KL divergence RKL respectively to balance the optimization of head and tail probabilities, as shown in the following formula:

[0046]

[0047] Where h, m, t are the number of head, middle and tail tokens, respectively, the set of head, middle and tail tokens. For the token x, this application uses the mean and standard deviation of the probability to judge, specifically: Head token: token whose probability is greater than the mean value plus one standard deviation; Tail token: token whose probability is less than the mean value minus one standard deviation; Middle token: other tokens.

[0048] It is worth noting that considering the excessive length of the vocabulary of large language models, this application sorts the tokens by size, and selects the top 1000 tokens for calculation.

[0049] Because the reference model implemented in this application framework is based on vLLM[3] (an efficient large-scale language model inference engine based on Transformer architecture) online inference, there is a slight accuracy difference between the training model, so this application rounds all probabilities P to the thousandth place, and the accuracy at the thousandth place is discarded.

[0050] Step 4, training framework design step: The method of this application needs to add reference probabilities to the training process. If all reference probabilities are loaded offline, compared with traditional methods, it will cause more than 1000 times storage pressure. If it is deployed for online inference of reference probabilities on the same GPU, it will reduce the storage pressure, but it will increase the video memory pressure of a single GPU due to the addition of multiple reference models, which may cause insufficient video memory.

[0051] Therefore, the present application selects to use online reasoning to exchange communication pressure for memory pressure, and uses vLLM to deploy multiple model services in another cluster and configure load balancing. In this way, the problem of reference model and training model competing for memory is solved, the problem of storage pressure is solved, and the demand pressure for homogeneous computing power in the scene with limited computing power is relieved.

[0052] The method has certain requirements for reasoning efficiency. The present application simplifies the reasoning process of vLLM, passes tokens through ID, only outputs a single token in the fixed decoding stage, and removes the inverse token stage of vLLM to improve efficiency. Because the data loader of the training process is asynchronous with the training process, the method can use the data loader to request data in advance to ensure that the training process and the data acquisition process are asynchronous to reduce the impact on training efficiency. Details are shown in the following. Figure 3

[0053] Based on the above analysis, the present application can obtain a dynamic self-constrained power industry large model continued pre-training method, which has the following advantages: The present application dynamically injects instruction training data in the industry model continued pre-training stage to ensure training stability in the form of course learning and improve the ability of the industry model to follow instructions, which is convenient for subsequent dialogue and reasoning ability training.

[0054] Meanwhile, maintain general ability: the present application dynamically maintains the basic general ability of the original model through dynamic self-constraint in the industry model continued pre-training stage, and optimizes the instruction following ability of the model guided by the teacher model. In addition, the effectiveness of the reference model for model training is improved through the adaptive KL algorithm design.

[0055] The present application uses communication cost to replace memory cost, uses vLLM to deploy reasoning service to reduce the demand for computing power, which requires low computing power and improves the reasoning speed. In addition, in the form of service communication, the same architecture restriction of reference model and training model computing power is solved.

[0056] The dynamic self-constrained power industry large model continued pre-training method and framework proposed by the present application does not depend on the model architecture. In addition, the deployment of the reference model is also based on vLLM, so the architecture of the computing power also has certain applicability.

[0057] The second object of the present application is to provide a dynamic self-constrained power industry large model continued pre-training system based on the above dynamic self-constrained power industry large model continued pre-training method, comprising: ​​​A dynamic dataset module is configured to obtain an industry pre-training corpus and an instruction training corpus, dynamically adjust a mixing ratio of the industry pre-training corpus and the instruction training corpus through a course strategy, and obtain a dynamic mixed dataset. A dynamic self-constrained training module is configured to configure a reference model based on the dynamic mixed dataset, adopt a differential loss function and the reference model for different types of data in the dynamic mixed dataset, and train a target power industry large model. An adaptive KL divergence calculation module is configured to calculate adaptive KL divergence according to a partition optimization logic, and construct a loss function using the adaptive KL divergence. An online inference deployment module is configured to construct an online inference framework based on a vLLM during training, obtain a probability of the reference model through the online inference framework, and make the training continue based on the probability of the reference model.

[0058] A third object of the embodiments of the present application is to provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned power industry large model continued pre-training method based on dynamic self-constraint when executing the computer program.

[0059] When the processor executes the computer program, the power industry large model continued pre-training method based on dynamic self-constraint is implemented, and specifically includes: An industry pre-training corpus and an instruction training corpus are obtained, a mixing ratio of the industry pre-training corpus and the instruction training corpus is dynamically adjusted through a course strategy, and a dynamic mixed dataset is obtained. A reference model is configured based on the dynamic mixed dataset, a differential loss function and the reference model are adopted for different types of data in the dynamic mixed dataset, and a target power industry large model is trained. The differential loss function is calculated according to a partition optimization logic to obtain adaptive KL divergence, and a loss function is constructed using the adaptive KL divergence. During training, an online inference framework is constructed based on a vLLM, a probability of the reference model is obtained through the online inference framework, and the training is made to continue based on the probability of the reference model.

[0060] A fourth object of the embodiments of the present application is to provide a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the power industry large model continued pre-training method based on dynamic self-constraint.

[0061] When the computer program is executed by the processor, the power industry large model continued pre-training method based on dynamic self-constraint is implemented, and specifically includes: The industry pre-training corpus and the instruction training corpus are acquired, a mixed ratio of the industry pre-training corpus and the instruction training corpus is dynamically adjusted through a course strategy, and a dynamic mixed dataset is obtained; A reference model is configured based on the dynamic mixed dataset, a differential loss function and the reference model are used for different types of data in the dynamic mixed dataset, and a target power industry large model is trained; The differential loss function is adaptive KL divergence calculated according to a partition optimization logic, and the adaptive KL divergence is used to construct a loss function. During the training process, an online inference framework is constructed based on the vLLM, the probability of the reference model is obtained through the online inference framework, and the training is continuously performed based on the probability of the reference model.

[0062] A fifth object of the embodiments of the application is to provide a computer program product comprising computer instructions instructing a computer to execute the power industry large model continued pre-training method based on dynamic self-constraint.

[0063] The computer instructions instruct the computer to execute the power industry large model continued pre-training method based on dynamic self-constraint, and specifically include: The industry pre-training corpus and the instruction training corpus are acquired, a mixed ratio of the industry pre-training corpus and the instruction training corpus is dynamically adjusted through a course strategy, and a dynamic mixed dataset is obtained; A reference model is configured based on the dynamic mixed dataset, a differential loss function and the reference model are used for different types of data in the dynamic mixed dataset, and a target power industry large model is trained; The differential loss function is adaptive KL divergence calculated according to a partition optimization logic, and the adaptive KL divergence is used to construct a loss function. During the training process, an online inference framework is constructed based on the vLLM, the probability of the reference model is obtained through the online inference framework, and the training is continuously performed based on the probability of the reference model.

[0064] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction devices, which realize the functions specified in the flow Figure 1 One flow or multiple flows and / or blocks Figure 1 One block or multiple blocks.

[0065] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0066] The present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, Read-Only Memory (ROM), programmable memory, and the like) embodying computer program instructions.

[0067] The present application is described in terms of flowcharts and / or block diagrams in accordance with embodiments of the present application. It will be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0068] Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should belong to the scope of protection of the present application.

[0069] Finally, it should be noted that the above embodiments are merely used to describe but not limit the technical solutions of the present application; although the present application is described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A dynamic self-constrained-based power industry large model continuation pre-training method, characterized in that, The method comprises the following steps: acquire industry pre-training corpus and instruction training corpus, dynamically adjust the mixing ratio of the industry pre-training corpus and the instruction training corpus through a course strategy, and obtain a dynamic mixed dataset; configure a reference model based on the dynamic mixed dataset, and train a target power industry large model by using a differential loss function and the reference model for different types of data in the dynamic mixed dataset; the differential loss function is calculated by adaptive KL divergence according to the partition optimization logic, and the loss function is constructed by using the adaptive KL divergence; during the training process, an online inference framework is constructed based on vLLM, the probability of the reference model is obtained through the online inference framework, and the training is continuously performed based on the probability of the reference model.

2. The dynamic self-constrained based power industry large model continued pre-training method of claim 1, wherein, The industry pre-training corpus includes general knowledge sub-corpus and power industry professional sub-corpus, and the scale is logarithmic B tokens.

3. The dynamic self-constrained based power industry large model continued pre-training method of claim 1, wherein, The course strategy is specifically: Suppose the number of training samples of the industry pre-training corpus is , the number of training samples of the instruction training corpus is , and the training batch size is B, then the samples of the instruction training in the i-th step Batch are: wherein a, b and L are hyperparameters, L defines the number of steps at which the instruction corpus starts to be injected, and a and b jointly control the injection rate and amplitude of the instruction corpus.

4. The method of claim 1, wherein the method is performed by a computer system. The reference model includes a constraint model for maintaining the general ability of the model and a teacher model for optimizing the instruction following ability of the model; The constraint model is a general domain pre-training large language model, and the teacher model is a dialog large language model fine-tuned by instructions.

5. The method of claim 1, wherein the method is performed by a computer system. The differential loss function is specifically: adding a loss function based on adaptive KL divergence to general knowledge subcorpora and instruction training corpora in industry pre-training corpora training; For the power industry professional sub corpus in the industry pre training corpus, the language model based loss function is adopted training; total training loss of the target model where a is a weight hyperparameter of the KL divergence loss function.​ 6. The method of claim 1, wherein the method is performed by a computer system. The adaptive KL divergence is determined according to the partition optimization logic, and the loss function is constructed by using the adaptive KL divergence, The method comprises the following steps: Fitting a model using adaptive KL divergence and reference model The adaptive KL divergence uses forward KL divergence FKL and reverse KL divergence RKL to handle the head and tail probabilities, respectively: Wherein, h, m, t are the number of head, middle and tail tokens respectively, The sets of head, middle and tail tokens respectively; for the discrimination of token x, the mean and standard deviation are used to judge, specifically: Head token: Token of the token; Tail token: Token of the tail; Mid tokens: other tokens; sort tokens by size and select some tokens for computation. from which to select some tokens for computation.

7. The method of claim 1, wherein the method is performed by a computer system. Fitting a model using adaptive KL divergence and reference models When converting probability values to data, the probability values are rounded to the nearest centile, discarding precision below the centile.

8. The method of claim 1, wherein the method is performed by a computer system. The online inference framework is constructed by using vLLM to deploy the reference model on an inference cluster independent of the training cluster and configuring a load balancing mechanism; token information is transmitted through the ID identification of the token, a fixed decoding stage only outputs a single predicted token, and an inverse token processing stage is removed; The asynchronous data loader of the training process is used to obtain the reference model probability from the inference cluster in advance, and the training process and the reference model probability acquisition process are asynchronously parallel. The method comprises the following steps:

9. A system for dynamic self-constrained power industry large model continued pre-training, used to implement the method of dynamic self-constrained power industry large model continued pre-training according to any one of claims 1 to 8, characterized in that, Dynamic dataset module: used for acquiring industry pre-training corpus and instruction training corpus, dynamically adjusting the mixing ratio of the industry pre-training corpus and the instruction training corpus through a course strategy, and obtaining a dynamic mixed dataset; Dynamic self-constrained training module: used for configuring a reference model based on the dynamic mixed dataset, and training a target power industry large model by using a differential loss function and the reference model for different types of data in the dynamic mixed dataset; Adaptive KL divergence calculation module: used for calculating adaptive KL divergence according to partition optimization logic, and constructing a loss function by using adaptive KL divergence; Online inference deployment module: during the training process, an online inference framework is constructed based on vLLM, the probability of the reference model is obtained through the online inference framework, and the training is continuously performed based on the probability of the reference model. ​ 10. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the dynamic self-constraint based power industry large model continued pre-training method in any one of claims 1-7.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the dynamic self-constraint based power industry large model continued pre-training method in any one of claims 1-7.

12. A computer program product comprising computer instructions, characterized in that, The computer instructions instruct the computer to execute the dynamic self-constraint based power industry large model continued pre-training method in any one of claims 1-7.

Images