Electric power large language model construction method based on knowledge enhancement and adaptive fine tuning

Abstract

The invention relates to an electric power large language model construction method based on knowledge enhancement and self-adaptive fine tuning, which comprises the following steps: S1, collecting text data related to the electric power field, and preprocessing the text data; s2, using SeqGAN to generate diversified data samples, and adding a confrontation mechanism in the generation step; s3, performing text labeling and basic NLP processing by using SpaCy, and identifying and labeling electric power related entities; s4, using Neo4j to design and construct a knowledge graph of the power field; s5, designing a hybrid expert model architecture, processing different types of power data through different expert modules, and constructing a power big language model; and S6, combining the knowledge graph with the electric power large language model, and enhancing the reasoning ability of the model through a prompt and attention mechanism. Structured information of the knowledge graph can be effectively combined, and the reasoning ability of the model and the application ability of the model in complex electric power tasks can be enhanced.

Description

Technical Field

[0001] This invention relates to the field of power large language model construction, and in particular to a method for constructing power large language models based on knowledge enhancement and adaptive fine-tuning. Background Technology

[0002] With the increasing complexity and volume of data in the power industry, the management and optimization of power systems face unprecedented challenges. The power industry encompasses a wide range of areas, including power equipment management, system status monitoring, process optimization, and compliance with standards and regulations. To improve efficiency and decision-making accuracy, more and more companies are seeking to leverage artificial intelligence and natural language processing technologies to process and analyze massive amounts of power data.

[0003] Large language models have shown great potential in processing natural language text, enabling them to perform prediction and generation tasks by learning patterns in data. However, traditional methods face challenges such as data scarcity, insufficient domain knowledge, and inadequate model generalization ability when dealing with domain-specific data. Therefore, the power industry urgently needs a new approach that combines domain knowledge and adaptive capabilities to improve model performance in complex tasks. Summary of the Invention

[0004] To address the aforementioned issues, the present invention aims to provide a method for constructing a large language model for power systems based on knowledge enhancement and adaptive fine-tuning. This method enables the constructed large language model to effectively integrate the structured information of knowledge graphs, thereby strengthening the model's reasoning ability and its application in complex power tasks.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for constructing a large-scale language model for power systems based on knowledge enhancement and adaptive fine-tuning includes the following steps: S1: Collect text data related to the power industry, and preprocess it to remove redundant, irrelevant or duplicate information to obtain preprocessed text data; S2: Based on the preprocessed text data, SeqGAN is used to generate diverse data samples, and an adversarial mechanism is added to the SeqGAN generation step; S3: Use SpaCy for text annotation and basic NLP processing to identify and annotate power-related entities; S4: Use Neo4j to design and build a knowledge graph in the power industry, covering the concepts of equipment, systems, processes and their relationships; S5: Based on a pre-trained large language foundation model, and designed a hybrid expert model architecture, different expert modules are used to process different types of power data, thereby improving the model's performance on diverse tasks and constructing a large power language model. S6: Combine knowledge graphs with the power big language model to enhance the model's reasoning ability through prompting and attention mechanisms, as detailed below: S61: Use knowledge graph embedding methods to convert entities and relations in knowledge graphs into vector representations; S62: Knowledge graph vectors are embedded and integrated into the input of the power big language model as supplementary information to enhance the model’s context awareness and reasoning ability, and the extracted graph features are injected into specific layers of the corresponding expert model to enhance domain knowledge. S63: Construct specific natural language prompts, combined with graph embeddings, as contextual start parameters for the language model; S64: Generate dynamic prompts based on the current input content and relevant knowledge graph node information; S65: Add a cross-attention layer inside the power large language model to focus on important information obtained through graph embedding and cues.

[0006] Furthermore, S1 specifically refers to: S11: Collect text data related to the power industry, convert data of different formats into a unified text format, and perform text extraction; S12: Remove redundant information from the text, including HTML tags, footnotes, headers and footers; use regular expressions to clean up special characters, blank lines and irrelevant tags; S13: Use the Jaccard similarity algorithm to detect and remove duplicate documents and paragraphs, and filter out content related to the power industry through keyword filtering and topic modeling LDA.

[0007] Furthermore, S13 specifically refers to: Each document or paragraph is segmented into a set of words, and the Jaccard similarity is calculated for each pair of documents or paragraphs. Set a similarity threshold; text pairs exceeding this threshold are considered duplicates, and one of them is removed. The text is trained using an LDA model, with a set number of topics; Analyze the topic distribution of each document, filter out topics related to the power industry, retain documents related to the target topic, and filter out other content; Create a list of keywords related to the power industry, iterate through each document or paragraph, check if it contains words from the keyword list, keep the text containing the keywords, and filter out irrelevant content.

[0008] Furthermore, the LDA model was used to train the text, with a set number of topics; the topic distribution of each document was analyzed, topics related to the power industry were selected, documents related to the target topic were retained, and other content was filtered out, as follows: The text data is segmented into words or phrases to form a bag of words, and stop words are removed. Stemming is used to normalize the same words in different forms, and the frequency of each word in the document is counted to form a word frequency matrix. Based on prior knowledge of the domain, the number of topics is set to K, and the LDA model is used to train documents to extract topics. The goal of the LDA model is to find the topic distribution by maximizing the likelihood. ; ; in, The topic distribution of document d; The distribution of keywords for topic k; Dirichlet priors for document topic distribution; Dirichlet priors for the distribution of subject terms; Represents the Dirichlet distribution; Each document generation process: ; in, It is the topic of the nth word in document d; Let be the nth word in document d; Multinomial represents the multinomial distribution; Analyze the LDA model training results and label the specific content reflected by each topic. Identify topics directly related to the power sector; For each document, calculate its topic distribution and identify the weights of topics related to electricity; Save documents whose weight in the power-related topic is greater than a threshold, and filter out other content that does not match the target topic.

[0009] Furthermore, based on the preprocessed text data, SeqGAN is used to generate diverse data samples, specifically: The preprocessed text data is converted into a format suitable for SeqGAN input, and SeqGAN is trained to generate diverse synthetic data. The generator G is an RNN used to generate text sequences. The goal of the generator is to generate realistic text sequences to fool the discriminator. The generator loss is: ; in, The loss function for the generator; Y represents the text sequence generated from generator G; E represents the output of the discriminator to the generated text sequence Y; E represents the expectation function. Discriminator D is a classifier used to distinguish between real text sequences and generated text sequences. The goal of the discriminator is to accurately identify generated text sequences. ; in, The loss function of the discriminator; Indicates the distribution of real data p data The real text sequence X sampled in the middle; Let X be the logarithm of the discriminator's output on the real text sequence X.

[0010] Furthermore, an adversarial mechanism is added to the SeqGAN generation step, as follows: Construct a knowledge base for the power sector, including key terms, standards, regulations, and typical data patterns, and extract key features and patterns from the knowledge base as constraints in the generation process; By introducing a conditional generation mechanism into the generator to ensure that the generated data conforms to domain requirements, the generator formula becomes: ; in, is the word preceding the generated sequence, and c is the condition vector containing specific constraints in the power sector; These are generated words; If specific characteristics of the power sector are used as input conditions to guide the generation process, then a domain constraint term needs to be added to the loss function to penalize generated data that does not conform to the domain criteria. ; in, This is the domain constraint loss, where λ is the weighting coefficient. A domain feature detection module is added to the discriminator to identify features that do not conform to the standards of the power industry, and the discriminator is fine-tuned using a pre-trained domain feature detection module.

[0011] Furthermore, the power big language model includes a base model, a routing layer, and an expert module. The base model adopts a pre-trained big language model to handle common language tasks, including semantic understanding, sentence generation, and general feature representation, which are then further processed by the expert module. The routing layer determines the optimal expert module based on the characteristics of the input data: ; in, It represents the probability of the input being assigned to the i-th expert; This is the routing weight matrix; The feature representation of the input; The expert module includes several expert sub-modules, each focusing on a specific type of power data or task. These include an equipment expert sub-module for processing equipment-related data, a system expert sub-module for processing system-level data, and a process expert sub-module for processing process-related data. Each expert sub-module is a neural network, trained using specific task data. Overall loss function: ; Where N is the number of expert submodules; α i The weights of the loss for the i-th expert module; D i This is the dataset prepared for the i-th expert; It is the loss function of the i-th expert on its specific task; It is a global regularization term; It is a regularization parameter. Let be the expected function.

[0012] Furthermore, the training process of the power large language model is as follows: We chose the pre-trained large language model BERT as the base model; Initialize the base model and expert modules by initializing parameters using a pre-trained large language model; add specialized layers to each expert module to handle its task, with initial parameters selected from similar domain modules of the base model; The router is trained to learn the mapping relationship between task characteristics and expert modules. A full dataset containing various tasks and data types is used to enable the router to learn to distinguish the characteristics of different task features. The cross-entropy loss function is used to optimize routing decisions so that it can correctly map task characteristics to appropriate expert modules. Joint training is performed across the entire architecture, enabling routing decisions and expert modules to be optimized together. In each training step, the parameters of the base model, expert modules, and routers are updated synchronously.

[0013] In another embodiment, a power big language model construction system based on knowledge enhancement and adaptive fine-tuning is provided, including a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the power big language model construction method based on knowledge enhancement and adaptive fine-tuning as described above.

[0014] In another embodiment, a computer storage medium is provided that stores a plurality of instructions adapted for loading by a processor and executing the method steps described above.

[0015] The present invention has the following beneficial effects: 1. The large language model constructed in this invention can effectively combine the structured information of knowledge graphs to enhance the model's reasoning ability and its application in complex power tasks; 2. This invention effectively analyzes and filters large-scale text by combining Jaccard similarity and LDA topic modeling, accurately locating documents closely related to the power industry, enabling the model to focus more on key content within the field. 3. This invention utilizes SeqGAN to generate high-quality synthetic text data, enhancing the richness and coverage of datasets in the power sector. Furthermore, by introducing specific constraints from the power sector into the SeqGAN generation process, the authenticity and compliance of the generated data are ensured. 4. This invention constructs a hybrid expert large language model for the power field, which utilizes the general features of the pre-trained large language model and processes diverse tasks in the power field through the specialization capabilities of expert modules, thereby improving the overall model's performance and adaptability. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of a text processing method in one embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the process of combining knowledge graphs and power big data language models in one embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: refer to Figure 1 In this implementation, a method for constructing a large-scale power language model based on knowledge enhancement and adaptive fine-tuning is provided, including the following steps: S1: Collect text data related to the power industry, including technical documents, standards, operation manuals, and academic papers, and preprocess them to remove redundant, irrelevant, or duplicate information to obtain preprocessed text data. S2: Based on the preprocessed text data, SeqGAN is used to generate diverse data samples to enhance the richness of the dataset, ensure that the data covers key changes in the power sector, and add an adversarial mechanism in the SeqGAN generation step to ensure the authenticity and compliance of the generated data. S3: Use SpaCy for text annotation and basic NLP processing to identify and annotate power-related entities; S4: Use Neo4j to design and build a knowledge graph for the power industry, covering device, system, process concepts and their relationships: S5: Based on a pre-trained large language foundation model, and designed a hybrid expert model architecture, different expert modules are used to process different types of power data, thereby improving the model's performance on diverse tasks and constructing a large power language model. S6: Combine knowledge graphs with a large-scale language model for electrical systems, and enhance the model's reasoning ability through prompts or attention mechanisms. (Refer to...) Figure 3 The details are as follows: S61: Use knowledge graph embedding methods (such as TransE, Node2Vec, GraphSAGE, etc.) to transform entities and relations in the knowledge graph into vector representations; S62: Knowledge graph vectors are embedded and integrated into the input of the power big language model as supplementary information to enhance the model’s context awareness and reasoning ability, and the extracted graph features are injected into specific layers of the corresponding expert model to enhance domain knowledge. S63: Construct specific natural language prompts, combined with graph embeddings, as contextual start parameters for the language model; S64: Generate dynamic prompts based on the current input content and relevant knowledge graph node information; S65: Add a cross-attention layer inside the power large language model to focus on important information obtained through graph embedding and cues.

[0018] refer to Figure 2 In this embodiment, the text processing specifically includes: S11: Collect text data related to the power industry, convert data in different formats (such as PDF, Word, HTML) into a unified text format, and use tools such as Apache Tika or PDFMiner to extract the text. S12: Remove redundant information from the text, including HTML tags, footnotes, headers and footers; use regular expressions to clean up special characters, blank lines and irrelevant tags; S13: Use the Jaccard similarity algorithm to detect and remove duplicate documents and paragraphs, and filter out content related to the power industry through keyword filtering and topic modeling LDA.

[0019] In this embodiment, S13 specifically refers to: Each document or paragraph is segmented into a set of words, and the Jaccard similarity is calculated for each pair of documents or paragraphs. Set a similarity threshold (e.g., 0.8). Text pairs with similarity values ​​above this threshold are considered duplicates, and one of them is removed. The text is trained using an LDA model, with a set number of topics; Analyze the topic distribution of each document, filter out the topics related to the power field, retain the documents related to the target topic, and filter out other content; Create a keyword list related to the power field, traverse each document or paragraph, check whether it contains the words in the keyword list, retain the text containing the keywords, and filter out the irrelevant content.

[0020] In this embodiment, use the LDA model to train the text, set the number of topics; analyze the topic distribution of each document, filter out the topics related to the power field, retain the documents related to the target topic, and filter out other content, specifically as follows: Segment the text data into words or phrases to form a bag of words, and remove stop words (such as "of", "is", etc.); Use stemming to normalize the same vocabulary in different forms, and count the frequency of each word in the document to form a term frequency matrix; According to the domain prior knowledge, set the number of topics as K, and use the LDA model to train the documents to extract topics. The goal of the LDA model is to find the topic distribution by maximizing the likelihood: ; ; Among them, is the topic distribution of document d; is the topic word distribution of topic k; is the Dirichlet prior of the document topic distribution; is the Dirichlet prior of the topic word distribution; represents the Dirichlet distribution; The generation process of each document: ; Among them, is the topic of the nth word in document d; is the nth word in document d; Multinomial represents the multinomial distribution; Analyze the training results of the LDA model, and mark the specific content reflected by each topic. Identify the topics directly related to the power field; For each document, calculate its topic distribution and identify the weights of the topics related to power; Save the documents with the weights of the power-related topics greater than the threshold, and filter out other content that does not match the target topic.

[0021] In this embodiment, based on the preprocessed text data, use SeqGAN to generate diverse data samples, specifically as follows: The preprocessed text data is converted into a format suitable for SeqGAN input, and SeqGAN is trained to generate diverse synthetic data. The generator G is an RNN used to generate text sequences. The goal of the generator is to produce realistic text sequences to fool the discriminator. The generator loss is: ; in, The loss function for the generator; Y represents the text sequence generated from generator G; E represents the output of the discriminator to the generated text sequence Y; E represents the expectation function. Discriminator D is a classifier used to distinguish between real text sequences and generated text sequences. The goal of the discriminator is to accurately identify generated text sequences. ; in, The loss function of the discriminator; Indicates the distribution of real data p data The real text sequence X sampled in the middle; Let X be the logarithm of the discriminator's output on the real text sequence X.

[0022] In this embodiment, an adversarial mechanism is added to the SeqGAN generation step, as follows: A knowledge base for the power sector is constructed, including key terms, standards, regulations, and typical data patterns. Key features and patterns are extracted from the knowledge base as constraints in the generation process. Features include the parameter range of the power system, typical operating modes, and compliance standards. By introducing a conditional generation mechanism into the generator to ensure that the generated data conforms to domain requirements, the generator formula becomes: ; in, is the preceding word in the generated sequence, and c is the condition vector, which contains specific constraints in the power field (such as terminology and parameter range). These are generated words; If specific characteristics of the power sector are used as input conditions to guide the generation process, then a domain constraint term needs to be added to the loss function to penalize generated data that does not conform to the domain criteria. ; in, This is the domain constraint loss, where λ is the weighting coefficient. A domain feature detection module is added to the discriminator to identify features that do not conform to the standards of the power industry, and the discriminator is fine-tuned using a pre-trained domain feature detection module.

[0023] In this embodiment, the power big language model includes a base model, a routing layer, and an expert module. The base model adopts a pre-trained big language model to handle common language tasks, including semantic understanding, sentence generation, and general feature representation, which are then further processed by the expert module. The routing layer determines the optimal expert module based on the characteristics of the input data: ; in, It represents the probability of the input being assigned to the i-th expert; This is the routing weight matrix; The feature representation of the input; The expert module includes several expert sub-modules, each focusing on a specific type of power data or task. These include an equipment expert sub-module for processing equipment-related data (such as sensor data and maintenance information); a system expert sub-module for processing system-level data (such as load forecasting and system status monitoring); and a process expert sub-module for processing process-related data (such as power generation processes and transmission issues). Each expert sub-module is a neural network, trained using specific task data. Overall loss function: ; Where N is the number of expert submodules; α i The weights of the loss for the i-th expert module; D i This is the dataset prepared for the i-th expert; It is the loss function of the i-th expert on its specific task; It is a global regularization term; It is a regularization parameter. Let be the expected function.

[0024] In this embodiment, the training process of the power big language model is as follows: We chose the pre-trained large language model BERT as the base model; Initialize the base model and expert modules by initializing parameters using a pre-trained large language model; add specialized layers (e.g., several neural network layers) to each expert module to handle its task, with initial parameters selected from similar domain modules of the base model; The router is trained to learn the mapping relationship between task characteristics and expert modules. A full dataset containing various tasks and data types is used to enable the router to learn to distinguish the characteristics of different task features. The cross-entropy loss function is used to optimize routing decisions so that it can correctly map task characteristics to appropriate expert modules. Joint training is performed across the entire architecture, enabling routing decisions and expert modules to be optimized together. In each training step, the parameters of the base model, expert modules, and routers are updated synchronously.

[0025] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0026] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0027] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0028] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0029] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for constructing a large power language model based on knowledge enhancement and adaptive fine-tuning, characterized in that: The following steps are involved: S1: Collect text data related to the power field, and pre-process to remove redundant, irrelevant or repeated information to obtain pre-processed text data; S2: Based on the preprocessed text data, SeqGAN is used to generate diverse data samples, and an adversarial mechanism is added to the SeqGAN generation step; S3: Use SpaCy for text annotation and basic NLP processing to identify and annotate power-related entities; S4: Use Neo4j to design and build a knowledge graph in the power field, covering equipment, system, process concepts and their relationships; S5: Based on the pre-trained large language basic model, a hybrid expert model architecture is designed to process different types of power data through different expert modules, improve the performance of the model on diversified tasks, and build a large power language model; S6: Combine the knowledge graph with the power language model and enhance the reasoning ability of the model through prompts and attention mechanisms; The power large language model includes a basic model, a routing layer and an expert module. The basic model uses a pre-trained large language model to process common language tasks, including semantic understanding, sentence generation, and general feature representation for further processing by the expert module; The routing layer determines the best expert module based on the input data characteristics: ； in, is the probability of the input being assigned to the i-th expert; is the routing weight matrix; is the feature representation of the input; The expert module includes several expert submodules, each of which focuses on a specific power data type or task, including an equipment expert submodule for processing equipment-related data; a system expert submodule for processing system-level data; and a process expert submodule for processing process-related data. Each expert submodule is a neural network trained with specific task data. Overall loss function: ; Where N is the number of expert submodules; α i is the weight of the loss of the i-th expert module; D i is the dataset prepared for the i-th expert; is the loss function of the i-th expert on his specific task; is the global regularization term; is the regularization parameter, is the expected function.

2. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 1 is characterized in that: The S1 is specifically: S11: Collect text data related to the power field, convert data in different formats into a unified text format, and perform text extraction; S12: Remove redundant information from the text, including HTML tags, footnotes, headers and footers; use regular expressions to clean up special characters, blank lines and irrelevant tags; S13: Use the text similarity algorithm Jaccard similarity to detect and remove duplicate documents and paragraphs, and filter out content related to the power field through keyword filtering and topic modeling LDA.

3. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 2 is characterized in that: The S13 is specifically: Split each document or paragraph into a set of words and calculate the Jaccard similarity between each pair of documents or paragraphs; Set a similarity threshold. Text pairs above this threshold are considered duplicates and one of them is removed. Use the LDA model to train the text and set the number of topics; Analyze the topic distribution of each document, filter out topics related to the power field, retain documents related to the target topic, and filter out other content; Create a list of keywords related to the power sector, traverse each document or paragraph, check whether it contains words in the keyword list, retain the text containing the keywords, and filter out irrelevant content.

4. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 3 is characterized in that: The LDA model is used to train the text and set the number of topics; the topic distribution of each document is analyzed, topics related to the power field are screened out, documents related to the target topic are retained, and other content is filtered out, as follows: Split text data into words or phrases, form bags of words, and remove stop words; Use stem extraction to normalize different forms of the same word, count the frequency of each word in the document, and form a word frequency matrix; According to the prior knowledge of the domain, the number of topics is set to K, and the LDA model is used to train documents to extract topics. The goal of the LDA model is to find the topic distribution by maximizing the likelihood: ； ； in, is the topic distribution of document d; is the topic word distribution of topic k; Dirichlet prior for document topic distribution; is the Dirichlet prior of the keyword distribution; Represents Dirichlet distribution; each document generation process: ； in, is the topic of the nth word in document d; is the nth word in document d; Multinomial represents multinomial distribution; Analyze the LDA model training results and mark the specific content reflected by each topic; identify topics directly related to the power field; For each document, calculate its topic distribution and identify the weight of electricity-related topics; Documents with power-related topic weights greater than the threshold are saved, and other content that does not match the target topic is filtered out.

5. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 1 is characterized in that: Based on the preprocessed text data, SeqGAN is used to generate diversified data samples, specifically: Convert the preprocessed text data into a format suitable for SeqGAN input and train SeqGAN to generate diverse synthetic data; The generator G is an RNN that generates text sequences. The goal of the generator is to generate realistic text sequences to deceive the discriminator. The generator loss is: ； in, is the loss function of the generator; represents the text sequence Y generated from the generator G; is the output of the discriminator for generating the text sequence Y; E represents the expected function; The discriminator D is a classifier used to distinguish between real text sequences and generated text sequences. The goal of the discriminator is to accurately identify the generated text sequences; ; in, is the loss function of the discriminator; Represents the distribution of real data p data The real text sequence X sampled in is the logarithm of the discriminator's output for the true text sequence X.

6. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 5 is characterized in that: The adversarial mechanism is added to the SeqGAN generation step, as follows: Build a knowledge base in the power field, including key terms, standards, regulations and typical data patterns, and extract key features and patterns from the knowledge base as constraints in the generation process; A conditional generation mechanism is introduced into the generator to ensure that the generated data meets the domain requirements. The generator formula is: ； in, is the previous word of the generated sequence, c is the conditional vector containing specific constraints in the power field; is a generated word; The specific characteristics of the power field are input into the generator as conditions to guide the generation process. It is necessary to add domain constraints to the loss function to penalize the generated data that does not meet the domain standards: ； in, is the domain constraint loss, λ is the weight coefficient; A domain feature detection module is added to the discriminator to identify features that do not meet the standards of the power field, and the pre-trained domain feature detection module is used to fine-tune the discriminator.

7. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 1 is characterized in that: The S6 is specifically as follows: S61: Use the knowledge graph embedding method to convert entities and relations in the knowledge graph into vector representations; S62: embed the knowledge graph vector into the input of the power large language model as supplementary information to enhance the model's context perception and reasoning capabilities, and inject the extracted graph features into the specific layer of the corresponding expert model to enhance domain knowledge; S63: Construct specific natural language prompts, combined with graph embedding, as contextual activation parameters for the language model; S64: Generate dynamic prompts based on current input content and related knowledge graph node information; S65: Add a cross-attention layer inside the power large language model to focus on important information obtained through graph embedding and prompts.

8. The method for constructing a large electric power language model based on knowledge enhancement and adaptive fine-tuning according to claim 1 is characterized in that: The training process of the power language model is as follows: Select the pre-trained large language model BERT as the base model; Initialize the basic model and expert modules, and initialize the parameters through the pre-trained large language model; add specialized layers to each expert module to handle its task, and the initial parameters of these layers are selected from the modules in similar fields of the basic model; Train the router to learn the mapping relationship between task characteristics and expert modules. Use a full dataset containing various tasks and data types to enable the router to learn to distinguish the characteristics of different task characteristics. Use the cross entropy loss function to optimize routing decisions so that it can correctly map task characteristics to appropriate expert modules. Joint training is performed on the entire architecture so that routing decisions and expert modules are optimized together. In each training step, the parameters of the basic model, expert module and router are updated synchronously.

9. A system for constructing a large power language model based on knowledge enhancement and adaptive fine-tuning, characterized in that: It includes a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the method for constructing a large power language model based on knowledge enhancement and adaptive fine-tuning as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 8.

Images