Autonomous controllable power text large model construction system based on fine-grained knowledge enhancement

Abstract

The invention discloses an autonomous controllable power text large model construction system based on fine-grained knowledge enhancement, and relates to the technical field of electronic control. The data acquisition and storage module acquires and stores user electricity charge, electric quantity and power grid electricity price information through the marketing data middle platform; the data preprocessing and anomaly detection module is used for preprocessing data, monitoring power consumption data by a dynamic threshold method, and identifying, analyzing and correcting or marking abnormal points; the data analysis and modeling module adopts a comprehensive prediction model to establish a prediction model of future electricity consumption, load change and electric charge expenditure of the user, and analyzes association between electricity consumption behaviors and electric charges; the energy-saving suggestion generation module is used for generating energy-saving suggestions according to analysis results and actual power And the interaction module displays suggestions for power supply company personnel and adjusts suggestion priorities according to feedback and use frequency. According to the invention, the intelligent level of power data processing is improved, abnormal data is accurately processed, a user is assisted to control cost and optimize power consumption, and reasonable utilization of power resources, energy conservation and emission reduction are promoted.

Description

Technical Field

[0001] This invention relates to the field of power control system technology, and in particular to an autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement. Background Technology

[0002] Textual data in the power industry contains a large amount of professional terminology, procedures, and equipment operation and maintenance knowledge, forming a crucial foundation for the intelligent operation of power systems. With the digital transformation of power businesses, textual data such as dispatch logs, maintenance reports, and customer inquiries are experiencing explosive growth, urgently requiring automated processing and knowledge mining through large-scale modeling technology. However, general-purpose large-scale models suffer from a lack of professional knowledge and misunderstandings of terminology in the power sector, making it difficult to meet the precise business needs.

[0003] Existing power text processing models are mostly based on fine-tuning of general pre-trained models, lacking deep integration of fine-grained knowledge in the power field, resulting in poor performance in professional tasks.

[0004] With the advancement of new power system construction, higher demands are being placed on the professional depth, response speed, and security and controllability of large-scale text models. Existing technologies have significant shortcomings in areas such as fine-grained knowledge integration, domestic adaptation, and dynamic knowledge updates, making them unable to support the intelligent upgrade of core businesses such as power dispatching and customer service. Therefore, building a large-scale power text model system that integrates fine-grained domain knowledge and possesses independent controllability has become crucial to overcoming current technological bottlenecks. Summary of the Invention

[0005] The present invention proposes an autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement to solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an autonomous and controllable large-scale power text model construction system based on fine-grained knowledge enhancement, comprising: The knowledge graph construction module for the power industry: By crawling power industry standards, technical documents, and equipment manual text data, it uses entity recognition and relation extraction algorithms to extract power professional entities and their relationships, constructs a graph containing fine-grained knowledge of equipment parameters, fault types, and operation and maintenance processes, and stores it in a distributed knowledge database; Text preprocessing module: Cleans, segments, and tags the texts of power dispatch logs, maintenance reports, and user inquiries. It uses a power industry thesaurus for mapping and conversion of technical terms, processes low-frequency words through a dynamic masking mechanism, and generates standardized text corpus. Knowledge-enhanced pre-training module: Based on the Transformer architecture, knowledge graph information is integrated into the pre-training process. Words in the text are associated with entities in the knowledge graph through entity links. A knowledge-aware attention mechanism is designed to strengthen the guidance of professional knowledge on the model. A hybrid loss function is used to optimize the model parameters. Autonomous and controllable adaptation module: Based on the chip architecture, the model calculation logic is optimized, the model compression technology is used to reduce the deployment resource requirements, and the collaborative training of each node is realized through the federated learning framework to ensure data privacy and model autonomy and controllability; Model Evaluation and Iteration Module: Constructs a dedicated evaluation dataset for the power industry, evaluates model performance from the dimensions of accuracy, recall, and consistency of technical terminology, and dynamically updates model parameters based on evaluation results and new knowledge.

[0007] Furthermore, in the power domain knowledge graph construction module, entity recognition adopts the BERT-CRF model, introduces power domain dictionary features to enhance entity boundary recognition, and extracts hierarchical relationships between entities through annotation strategies in each round. The knowledge graph is stored in RDF format and supports SPARQL queries. Entity attributes include name, type, parameter range, and a list of associated entities.

[0008] Furthermore, in the text preprocessing module, the dynamic masking mechanism uses a combination of synonym substitution and partial masking to process terms that appear less frequently than a preset threshold. The standardized text corpus is divided into a training set, a validation set, and a test set in an 8:1:1 ratio, where the training set contains 100 million labeled samples in the power industry.

[0009] Furthermore, in the knowledge-enhanced pre-training module, the calculation method for the knowledge-aware attention mechanism is as follows: Let the word vectors of the text sequence be... The entity vector of the knowledge graph is Through entity link matrix Establishing connections, attention weights W Q W K For querying the key matrix, M i,k A value of 0-1 indicates the strength of the association between the word wi and the entity ek.

[0010] Furthermore, in the autonomous and controllable adaptation module, the model compression adopts a non-uniform quantization strategy, the attention layer weights are quantized with 8 bits, and the hidden layer outputs are quantized with 16 bits; the federated learning framework adopts a layered encryption mechanism, the local model parameters are uploaded to the aggregation server through homomorphic encryption, and the aggregation algorithm adopts a weighted average.

[0011] Furthermore, the evaluation metrics of the model evaluation and iteration module include professional question answering accuracy, entity linking accuracy, and the professionalism of the generated text. When any metric falls below a preset threshold, incremental training is triggered, introducing new knowledge graph data and labeled samples to update the model.

[0012] Furthermore, the formula for the hybrid loss function is as follows: L LM For bidirectional language model loss, L KG For knowledge matching loss, =0.7 is the weighting coefficient, which is adjusted based on the performance of the validation set.

[0013] Furthermore, it also includes: Domain fine-tuning module: For scenarios such as power dispatching, customer service, and equipment maintenance, it uses labeled data for fine-tuning, introduces scenario-specific vocabularies to expand the word embedding matrix, sets the fine-tuning learning rate to 1 / 10 of the pre-training stage, and dynamically adjusts the number of iterations according to the amount of scenario data.

[0014] Furthermore, the knowledge graph construction module supports incremental updates. It processes new text data through an incremental entity recognition model, automatically discovers new entities and relationships, and integrates them into the existing graph after manual review. The update frequency can be set to once a week.

[0015] Furthermore, the self-controllable adaptation module also includes a domestic deployment interface, which supports compatibility with domestic operating systems and databases, and provides a one-click script for model deployment, including environment detection, dependency installation, and performance testing functions.

[0016] Compared with existing technologies, the beneficial effects of this invention are: Multi-dimensional innovations have significantly improved the professionalism and reliability of power industry text processing. The power industry knowledge graph construction module integrates fine-grained knowledge such as equipment parameters and fault types, providing precise domain support for the model. This effectively solves the problem of misunderstanding of professional terminology in general models, enabling the model to accurately distinguish similar concepts in professional tasks and improve semantic understanding accuracy.

[0017] The multi-source heterogeneous text preprocessing module optimizes the processing flow for the characteristics of power industry texts. A dynamic masking mechanism ensures effective learning of low-frequency professional vocabulary, and standardized corpora lay a high-quality data foundation for model training, enhancing the model's adaptability to various text types such as dispatch logs and maintenance reports. The knowledge-enhanced pre-training module achieves deep integration of text and knowledge graphs through a knowledge-aware attention mechanism. A hybrid loss function balances language modeling and knowledge matching, ensuring that the model-generated content conforms to both linguistic rules and power industry standards.

[0018] The self-controllable adaptation module optimizes computational logic based on domestically produced chips. The federated learning framework ensures data privacy while enabling collaborative training. Model compression technology reduces deployment resource requirements, meeting the power industry's requirements for technological self-control. The model evaluation and iteration module dynamically monitors performance through a dedicated evaluation dataset and continuously incorporates new knowledge through an incremental update mechanism, ensuring the model adapts to the evolving power business. Overall, this application's system provides a professional, secure, and sustainable solution for intelligent processing of power-related text, contributing to the power industry's digital transformation. Attached Figure Description

[0019] Figure 1 This is a schematic block diagram of the autonomous and controllable large power text model construction system based on fine-grained knowledge enhancement proposed in this invention. Figure 2 This is a schematic diagram of the distribution of entity relationships in the knowledge graph of the power field in the autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement proposed in this invention. Figure 3 This is a schematic diagram of the loss function change curve during the model pre-training process of the autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement proposed in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0022] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The invention will now be described in further detail with reference to the accompanying drawings.

[0023] Reference Figures 1 to 3 A system for constructing an autonomous and controllable large-scale power text model based on fine-grained knowledge enhancement, comprising: The knowledge graph construction module for the power industry: By crawling text data such as power industry standards, technical documents, and equipment manuals, it uses entity recognition and relation extraction algorithms to extract power professional entities and their relationships, constructs a graph containing fine-grained knowledge such as equipment parameters, fault types, and operation and maintenance processes, and stores it in a distributed knowledge database; Multi-source heterogeneous text preprocessing module: Cleans, segments, and tags parts of speech for various types of text such as power dispatch logs, maintenance reports, and user inquiries. It uses a power field thesaurus for mapping and conversion of professional terms, processes low-frequency professional words through a dynamic masking mechanism, and generates standardized text corpus. Knowledge-enhanced pre-training module: Based on the Transformer architecture, knowledge graph information is integrated into the pre-training process. Words in the text are associated with entities in the knowledge graph through entity links. A knowledge-aware attention mechanism is designed to enhance the guidance of professional knowledge on the model. A hybrid loss function (language model loss + knowledge matching loss) is used to optimize the model parameters. Independent and controllable adaptation module: Based on the domestic chip architecture, the model calculation logic is optimized, the model compression technology (quantization, pruning) is adopted to reduce the deployment resource requirements, and multi-node collaborative training is realized through the federated learning framework to ensure data privacy and model autonomy and controllability; Model Evaluation and Iteration Module: Constructs a dedicated evaluation dataset for the power industry (including professional question answering, fault diagnosis, and text generation tasks), evaluates model performance from dimensions such as accuracy, recall, and consistency of professional terminology, and dynamically updates model parameters based on evaluation results and newly added knowledge.

[0024] In this invention, the power field knowledge graph construction module uses an improved BERT-CRF model for entity recognition, introduces power field dictionary features to enhance entity boundary recognition, and extracts hierarchical relationships between entities (such as "equipment-fault-cause" and "operation-step-standard") through a multi-round annotation strategy. The knowledge graph is stored in RDF format and supports SPARQL query. Entity attributes include name, type, parameter range, and a list of associated entities. In the multi-source heterogeneous text preprocessing module, the dynamic masking mechanism is used to process professional terms that appear less than a preset threshold (e.g., 5 times / million words) by combining synonym substitution (based on a knowledge graph thesaurus) with partial masking. The standardized text corpus is divided into training set, validation set and test set in an 8:1:1 ratio, where the training set contains at least 1 million labeled samples in the power field. In the knowledge enhancement pre-training module, the calculation method of the knowledge perception attention mechanism is as follows: Let the word vector of the text sequence be... The entity vector of the knowledge graph is Through entity link matrix Establishing connections, attention weights W Q W K For querying the key matrix, M i,k A value of 0-1 indicates the strength of the association between the word wi and the entity ek.

[0025] In this invention, the autonomous and controllable adaptation module employs a non-uniform quantization strategy for model compression, using 8-bit quantization for the attention layer weights and 16-bit quantization for the hidden layer outputs, with the quantization error controlled within 3%. The federated learning framework uses a layered encryption mechanism, with local model parameters uploaded to the aggregation server via homomorphic encryption. The aggregation algorithm uses a weighted average (weights are positively correlated with the amount of node data), ensuring that the original data is not leaked during the training process. The evaluation metrics of the model evaluation and iteration module include: professional question answering accuracy (correct answering accuracy for power regulations and fault handling), entity linking accuracy (correct matching accuracy between text entities and knowledge graph), and generated text professionalism (the degree of standardization of terminology marked by power experts, 0-10 points). When any metric is lower than the preset threshold (such as accuracy of 80%), incremental training is triggered, and new knowledge graph data and labeled samples are introduced to update the model. In this invention, the formula for the hybrid loss function is as follows: L LM For bidirectional language model loss (predicting mask word probabilities), L KG This is the knowledge matching loss (calculating the embedding distance loss between text entities and knowledge graph entities). =0.7 is the weighting coefficient, which is adjusted based on the performance of the validation set; This invention also includes a domain fine-tuning module: for subdivided scenarios such as power dispatching, customer service, and equipment maintenance, a small amount of labeled data (at least 10,000 data points for each scenario) is used for fine-tuning, a scenario-specific vocabulary is introduced to expand the word embedding matrix, the fine-tuning learning rate is set to 1 / 10 of the pre-training stage, and the number of iterations is dynamically adjusted according to the amount of scenario data (5-20 rounds).

[0026] In this invention, the knowledge graph construction module supports incremental updates. It processes new text data through an incremental entity recognition model (based on an incremental learning algorithm), automatically discovers new entities and relationships, and integrates them into the existing graph after manual review. The update frequency can be set to once a week to ensure the timeliness of knowledge. The self-controllable adaptation module also includes a domestic deployment interface, which supports compatibility with domestic operating systems and databases, provides a one-click script for model deployment, and includes functions such as environment detection, dependency installation, and performance testing. The deployment time is no more than 30 minutes, and the model inference latency is controlled within 500ms.

[0027] This invention discloses an autonomous and controllable large-scale power text model construction system based on fine-grained knowledge enhancement. By integrating fine-grained knowledge in the power field with domestically developed adaptation technologies, it achieves intelligent processing of power text. The following provides a detailed description of each module of the system in the context of power dispatching and customer service scenarios.

[0028] I. System Hardware and Data Configuration The system is deployed on a domestically produced server cluster (equipped with a Kunpeng 920 chip, with a computing power of 512 TOPS), including: Distributed knowledge database (using DM database, supporting RDF format storage, concurrent access ≥1000 times / second); Text preprocessing node (8-core CPU, 16GB memory, processing speed ≥5000 texts / minute); Model training nodes (4 Ascend 910 chips, 32GB of VRAM / chip, supporting mixed precision training); Federated learning aggregation server (encryption computing power ≥200TOPS, data transmission encryption level meets the national level 3 security protection standard).

[0029] Data sources include: power industry standards (DL / T series regulations), equipment manuals (10kV-500kV transformers, circuit breakers, etc.), dispatch logs (2018-2023, a total of 1.2 million entries), maintenance reports (800,000 documents), and user consultation records (5 million entries).

[0030] II. Implementation of Core Modules By crawling text data such as power industry standards, technical documents, and equipment manuals, entity recognition and relation extraction algorithms are used to extract power-related entities and their relationships, construct a graph containing fine-grained knowledge such as equipment parameters, fault types, and operation and maintenance processes, and store it in a distributed knowledge database.

[0031] 1. Power Sector Knowledge Graph Construction Module: Data Acquisition and Entity Recognition. This module acquires publicly available power sector literature through web crawling and combines it with internal company documents. An improved BERT-CRF model is used for entity recognition. The model adds a power sector dictionary (containing 32,000 technical terms) as a feature layer to the general BERT model, improving entity recognition accuracy by 15%. Recognized entity types include 12 categories such as: equipment (transformers, switchgear), faults (short circuits, insulation aging), and operations (switching, inspections).

[0032] Relation extraction and graph construction: A multi-round annotation strategy is used to extract entity relations, such as "equipment-fault" (transformer-abnormal oil level), "fault-cause" (abnormal oil level-poor sealing), and "operation-standard" (switching operation-repeated announcement). The knowledge graph is stored in RDF format, with 8 million triples, and supports SPARQL queries (response time ≤ 0.5 seconds).

[0033] Incremental update mechanism: New text is processed weekly by an incremental entity recognition model (based on EWC incremental learning algorithm), which automatically discovers new entities (such as new smart meter models) and relationships. After manual review (pass rate ≥90%), the data is integrated into the graph, and the update time is ≤4 hours.

[0034] 2. Multi-source heterogeneous text preprocessing module The system cleans, segments, and tags parts of speech for various types of text, including power dispatch logs, maintenance reports, and user inquiries. It uses a power-related thesaurus for mapping and conversion of technical terms, and processes low-frequency technical words through a dynamic masking mechanism to generate standardized text corpus.

[0035] Text cleaning and standardization: Processing various types of text: Removing redundant codes from scheduling logs (e.g., “#2 Unit Shutdown Record [20230615]” → “Unit 2 Shutdown Record 20230615”); Standardizing the fault description format in maintenance reports (“liquid leakage” and “oil leakage” are both mapped to “liquid leakage”); Removing emoticons and irrelevant small talk from user inquiries.

[0036] Dynamic masking: For low-frequency technical terms (occurring less than 5 times per million words, such as "GIS combined electrical appliances"), a combination of synonym substitution (based on a knowledge graph thesaurus, such as "GIS" → "gas-insulated switchgear") and partial masking ("gas-insulated switchgear") is used. The masking ratio increases from 10% to 30% as the frequency decreases.

[0037] Corpus partitioning and storage: Standardized texts are divided into training set (10 million records), validation set (1.25 million records), and test set (1.25 million records) in an 8:1:1 ratio, and stored in TFRecord format, supporting parallel reading (throughput ≥200MB / s).

[0038] 3. Knowledge Enhancement Pre-training Module Based on the Transformer architecture, knowledge graph information is incorporated during the pre-training process. Words in the text are associated with entities in the knowledge graph through entity links. A knowledge-aware attention mechanism is designed to enhance the guidance of professional knowledge on the model. A hybrid loss function (language model loss + knowledge matching loss) is used to optimize the model parameters.

[0039] Model architecture: Based on the Transformer architecture (12-layer encoder, 768-dimensional hidden layers, 12 attention heads), knowledge graph entity embeddings (300-dimensional) are added after the word embedding layer, using an entity link matrix. Establishing associations between text words and entities (M i,k =1 indicates that the word wi is associated with the entity ek, otherwise it is 0).

[0040] Knowledge perception attention mechanism: Attention weights are calculated as follows: in, Let represent the attention weights between the i-th and j-th words; wi and wj are the word vectors of the i-th and j-th words in the text sequence, respectively; M i,k is the element in the i-th row and k-th column of the entity link matrix (0 or 1, representing the association between a word and an entity); ek is the embedding vector of the k-th entity in the knowledge graph; W Q W K These are the query matrix and the key matrix (768×768 dimensions); dk=64 is the dimension of the attention head. This mechanism strengthens the association between key knowledge such as "fault-handling steps".

[0041] Hybrid loss function optimization: The formula for the hybrid loss function is as follows: The formula L = 0.7LLM + 0.3LKG is used, where L is the total loss function; L LM For bidirectional language model loss (used to optimize mask word prediction); L KG 0.7 represents the knowledge matching loss (used to optimize the matching degree between text entities and knowledge graph entities); 0.7 and 0.3 are weight coefficients. Pre-training iterations are performed for 1 million steps, with the learning rate linearly decreasing from 5e-5 to 1e-5, and batch size = 256.

[0042] 4. Independently controllable adaptation module The model computation logic is optimized based on domestic chip architecture, and model compression technology (quantization and pruning) is used to reduce deployment resource requirements. Multi-node collaborative training is achieved through a federated learning framework to ensure data privacy and model autonomy and controllability.

[0043] Model compression: Non-uniform quantization strategy is adopted: attention layer weights are 8-bit quantized (quantization range [-10,10], step size 0.039), hidden layer outputs are 16-bit quantized, and quantization error ≤2.5%; pruning removes connections with absolute weight values ​​<1e-4 (pruning rate 30%), reducing the model size from 1.2GB to 500MB.

[0044] Federated learning training: The power company headquarters acts as the aggregation server, and five regional branches act as clients. The local model is trained on the client terminals (20 iterations per round). Parameters are uploaded via homomorphic encryption (Paillier algorithm, 2048-bit key length). The aggregation weight is positively correlated with the amount of data on the client (the client with 30% of the data has a weight of 0.3). The risk of data leakage during training is zero.

[0045] Domestic Deployment: Provides interfaces for integration with Kylin operating system and Kingbase database. The deployment script includes environment detection (checking dependency library versions), dependency installation (automatically downloading domestic Python libraries), and performance testing (generating inference latency reports). The entire process takes ≤25 minutes, and the inference latency is ≤400ms.

[0046] 5. Model Evaluation and Iteration Module Evaluation Dataset and Metrics: A power sector-specific evaluation dataset was constructed, including: professional Q&A (5000 questions, such as "Specific Contents of the Two-Ticket Three-System"), fault diagnosis (3000 cases, such as "Analysis of the Causes of Transformer Gas Protection Actions"), and text generation (2000 entries, such as "Writing Line Maintenance Work Orders"). Evaluation metrics include: accuracy of professional Q&A, accuracy of entity linking, and professionalism of generated text (scored by power experts, 0-10 points).

[0047] Iteration mechanism: When any indicator falls below the threshold (accuracy 80%, professionalism 7 points), incremental training is triggered: new knowledge graph data (new triples added in the past week) and labeled samples (5000 items / time) are introduced, the learning rate is 1 / 10 of the pre-training stage (5e-6), iterates for 10 rounds, and the model update time is ≤8 hours.

[0048] 6. Domain Fine-tuning Module For scenarios such as dispatching, customer service, and maintenance, scenario-specific data fine-tuning is adopted: Dispatch scenario: Input 10,000 dispatch instructions (such as "order #3 line to cold standby"), expand the vocabulary to include terms such as "dispatch order number" and "operation ticket status", and fine-tune 15 rounds; Customer service scenario: Input 20,000 user inquiries (such as "residential peak and off-peak electricity pricing periods"), optimize the question and answer generation logic, and make 20 rounds of fine-tuning; Maintenance scenario: Input 15,000 maintenance reports, strengthen the association between fault description and handling steps, and make 12 rounds of fine-tuning.

[0049] III. Implementation Results Data

[0050] Due to a lack of knowledge in the power industry, the accuracy of general large-scale models in professional question answering and entity linking is less than 60%, and the generated text contains terminological errors (such as writing "zero-phase protection" instead of "zero-sequence protection"). This application's system, through fine-grained knowledge enhancement, improves accuracy to over 90%, with a professionalism score approaching expert level. Autonomous and controllable adaptation reduces inference latency by nearly half, meeting real-time business needs, and achieving the highest industry standards for data privacy protection. In actual business operations such as dispatch instruction parsing and user inquiry responses, the system's processing efficiency is 5-10 times higher than manual methods, significantly reducing operating costs and providing reliable support for intelligent text processing in the power industry.

[0051] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A system for constructing an autonomous and controllable large-scale power text model based on fine-grained knowledge enhancement, characterized in that, include: The knowledge graph construction module for the power industry: By crawling power industry standards, technical documents, and equipment manual text data, it uses entity recognition and relation extraction algorithms to extract power professional entities and their relationships, constructs a graph containing fine-grained knowledge of equipment parameters, fault types, and operation and maintenance processes, and stores it in a distributed knowledge database; Text preprocessing module: Cleans, segments, and tags the texts of power dispatch logs, maintenance reports, and user inquiries. It uses a power industry thesaurus for mapping and conversion of technical terms, processes low-frequency words through a dynamic masking mechanism, and generates standardized text corpus. Knowledge-enhanced pre-training module: Based on the Transformer architecture, knowledge graph information is integrated into the pre-training process. Words in the text are associated with entities in the knowledge graph through entity links. A knowledge-aware attention mechanism is designed to strengthen the guidance of professional knowledge on the model. A hybrid loss function is used to optimize the model parameters. Autonomous and controllable adaptation module: Based on the chip architecture, the model calculation logic is optimized, the model compression technology is used to reduce the deployment resource requirements, and the collaborative training of each node is realized through the federated learning framework to ensure data privacy and model autonomy and controllability; Model Evaluation and Iteration Module: Constructs a dedicated evaluation dataset for the power industry, evaluates model performance from the dimensions of accuracy, recall, and consistency of technical terminology, and dynamically updates model parameters based on evaluation results and new knowledge.

2. The autonomous and controllable large-scale power text model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, In the power domain knowledge graph construction module, entity recognition adopts the BERT-CRF model, and power domain dictionary features are introduced to enhance entity boundary recognition. Relationship extraction extracts the hierarchical relationship between entities through the annotation strategy of each round. The knowledge graph is stored in RDF format and supports SPARQL query. The entity attributes include name, type, parameter range, and a list of associated entities.

3. The autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, In the text preprocessing module, the dynamic masking mechanism uses a combination of synonym substitution and partial masking to process terms that appear less frequently than a preset threshold. The standardized text corpus is divided into a training set, a validation set, and a test set in an 8:1:1 ratio, with the training set containing 100 million labeled samples in the power industry.

4. The autonomous and controllable large-scale power text model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, In the knowledge enhancement pre-training module, the calculation method of the knowledge perception attention mechanism is as follows: Let the word vector of the text sequence be... The entity vector of the knowledge graph is Through entity link matrix Establishing connections, attention weights W Q W K For querying the key matrix, M i,k A value of 0-1 indicates the strength of the association between the word wi and the entity ek.

5. The autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, In the autonomous and controllable adaptation module, the model compression adopts a non-uniform quantization strategy, the attention layer weights are quantized with 8 bits, and the hidden layer outputs are quantized with 16 bits; the federated learning framework adopts a layered encryption mechanism, the local model parameters are uploaded to the aggregation server through homomorphic encryption, and the aggregation algorithm adopts a weighted average.

6. The autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, The evaluation metrics of the model evaluation and iteration module include professional question answering accuracy, entity linking accuracy, and the professionalism of the generated text. When any metric falls below a preset threshold, incremental training is triggered, introducing new knowledge graph data and labeled samples to update the model.

7. The autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement according to claim 4, characterized in that, The formula for the hybrid loss function is: L LM For bidirectional language model loss, L KG For knowledge matching loss, =0.7 is the weighting coefficient, which is adjusted based on the performance of the validation set.

8. The autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, Also includes: Domain fine-tuning module: For scenarios such as power dispatching, customer service, and equipment maintenance, it uses labeled data for fine-tuning, introduces scenario-specific vocabularies to expand the word embedding matrix, sets the fine-tuning learning rate to 1 / 10 of the pre-training stage, and dynamically adjusts the number of iterations according to the amount of scenario data.

9. The autonomous and controllable large-scale power text model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, The knowledge graph construction module supports incremental updates. It processes new text data through an incremental entity recognition model, automatically discovers new entities and relationships, and integrates them into the existing graph after manual review. The update frequency can be set to once a week.

10. The autonomous and controllable power text large model construction system based on fine-grained knowledge enhancement according to claim 1, characterized in that, The independently controllable adaptation module also includes a domestic deployment interface, which supports compatibility with domestic operating systems and databases, and provides a one-click script for model deployment, including environment detection, dependency installation, and performance testing functions.

Images