Plug-in function test method for power system pre-training model based on NLP

Abstract

This invention relates to the field of NLP technology and discloses a plug-in function testing method for a pre-trained power system model based on NLP. The method includes: receiving a technical supervision query question and performing vector encoding and retrieval to obtain an initial retrieval result set; obtaining reference text with standard citation annotations through weighted sorting and concatenating it with multi-level prompt word templates to obtain domain-structured prompt words; performing a bundle search using a large technical supervision domain model fine-tuned with LoRA low-rank adaptation to generate candidate answer texts; and conducting quality assessments including citation integrity detection, logical consistency detection, and entity illusion detection to obtain a qualified technical supervision standard answer. This method solves the technical problems of low standard retrieval accuracy, poor answer accuracy, lack of citation tracing, and insufficient domain customization capabilities in existing technologies, providing a complete technical solution for intelligent applications in the field of power technical supervision.

Description

Technical Field

[0001] This invention relates to the field of NLP technology, and in particular to a plug-in functionality testing method for a pre-trained power system model based on NLP. Background Technology

[0002] Power technology supervision involves the management of standards and regulations throughout the entire lifecycle of thermal power equipment, requiring frequent searches of national, industry, and enterprise standards. Traditional manual retrieval methods are inefficient and lack accuracy. While existing general-purpose large language models possess natural language understanding capabilities, their training data primarily comes from the publicly available internet, lacking systematic learning of specialized standard documents in the field of power technology supervision. This leads to frequent issues such as irrelevant answers, incorrect standard number identification, and misunderstandings of technical indicators when responding to technical supervision questions.

[0003] Existing technical supervision information systems mostly employ keyword matching for retrieval, which fails to understand the semantic intent of user queries. This results in low accuracy and a lack of contextual relevance, making it difficult to meet the query needs of complex technical issues. Furthermore, general-purpose models are prone to generating fabricated responses (illusion questions), and the generated content lacks authoritative standard citations, compromising the accuracy and traceability of answers and severely impacting the authority and standardization of technical supervision work. In addition, technical supervision standard documents are characterized by dense technical terminology, complex inter-references, and frequent updates. Existing large-scale models cannot achieve real-time knowledge updates and localized secure deployment, and lack customization capabilities specific to the technical supervision field, making it difficult to generate technical documents and testing reports that conform to industry standards. Summary of the Invention

[0004] This invention provides a plug-in function testing method for a pre-trained power system model based on NLP, which effectively guides a large domain model fine-tuned by LoRA low-rank adaptation to understand technical supervision tasks and generate standardized answers. It solves the technical problems of low standard retrieval accuracy, poor answer accuracy, lack of citation tracing and insufficient domain customization capabilities in the prior art, and provides a complete technical solution for intelligent applications in the field of power technology supervision.

[0005] In a first aspect, the present invention provides a plug-in functionality testing method for a pre-trained power system model based on NLP, the plug-in functionality testing method for the pre-trained power system model based on NLP comprising: Receive technical supervision query questions, and perform vector encoding and retrieval on the technical supervision query questions to obtain an initial retrieval result set; The initial search result set is sorted by weight to obtain reference text with standard citation annotations. The reference text with standard citation annotations is then concatenated with a multi-level prompt word template to obtain domain structured prompt words. The domain structured cue words are input into a technically supervised domain large model that has been fine-tuned with LoRA low-rank adaptation for beam search to generate candidate answer text; The candidate answer texts are subjected to quality assessments including citation integrity detection, logical consistency detection, and entity illusion detection to obtain a technically supervised standard answer that meets quality standards.

[0006] In conjunction with the first aspect, in a first implementation of the first aspect of the present invention, a technical supervision query question is received, and the technical supervision query question is vector-encoded and retrieved to obtain an initial retrieval result set, including: The system receives technical supervision query questions, filters the technical supervision query questions for interjections, and replaces colloquial technical terms with corresponding standardized technical terms based on a preset standardized terminology dictionary to obtain the first query text. Obtain the character length in the first query text. When the character length is lower than a preset length threshold, extract extended words from the technical supervision terminology dictionary and co-occurrence word statistics table and add them to the first query text to obtain the second query text. The second query text is input into the BGE pre-trained model for word segmentation and Transformer encoding. The feature vectors of the CLS marker positions are extracted and used as the query semantic vectors. Based on the query semantic vector, cosine similarity calculation and retrieval are performed in the local vector knowledge base to obtain an initial retrieval result set.

[0007] In conjunction with the first aspect, in a second implementation of the first aspect of the present invention, cosine similarity calculation and retrieval are performed in a local vector knowledge base based on the query semantic vector to obtain an initial retrieval result set, including: Based on the query semantic vector, a metric distance is calculated in the IVF inverted file index of the local vector knowledge base. A preset number of cluster centers with the closest metric distance are selected, and the knowledge base vector set corresponding to the cluster centers is obtained. For each knowledge base vector in the knowledge base vector set, perform a dot product operation and vector magnitude normalization on the query semantic vector to calculate the cosine similarity score. Filter out knowledge base vectors with cosine similarity scores higher than a preset similarity threshold, along with their associated text block content, standard number, chapter title, and document update time. Sort them by cosine similarity score from high to low and extract a preset number of search records to obtain the initial search result set.

[0008] In conjunction with the first aspect, in a third implementation of the first aspect of the present invention, the initial search result set is weighted and sorted to obtain reference text with standard citation annotations, and the reference text with standard citation annotations is concatenated with a multi-level prompt word template to obtain domain-structured prompt words, including: Extract the cosine similarity score, document type corresponding to the standard number, document update time, and text block sequence number of the same standard document for each search record from the initial search result set; Based on a preset document type authority mapping table, the document type is converted into a document authority score; based on a preset timeliness decay rule, the document update time is converted into a timeliness score; and based on the text block sequence number, a diversity penalty coefficient is calculated. The cosine similarity score is multiplied by a first preset weight coefficient, the document authority score is multiplied by a second preset weight coefficient, the timeliness score is multiplied by a third preset weight coefficient, and the diversity penalty coefficient is multiplied by a fourth preset weight coefficient, and then weighted and summed to obtain the comprehensive ranking score of each search record. For each search record in the initial search result set, a citation mark is generated based on the comprehensive ranking score. The citation mark is then concatenated with the corresponding text block content to obtain reference text with standard citation marks. By concatenating the reference text with standard citation annotations with the multi-level prompt word template, a domain-structured prompt word is obtained.

[0009] In conjunction with the first aspect, in the fourth implementation of the first aspect of the present invention, the reference text with standard citation annotations is concatenated with a multi-level prompt word template to obtain domain-structured prompt words, including: Construct a multi-level prompt word template that includes a role setting level, a task constraint level, a few-sample example level, and a thought chain reasoning level; The reference text with standard citations is embedded into the reference level position of the multi-level prompt word template according to the preset reference format to obtain the prompt word structure for filling references; By embedding the technical supervision query question into the query hierarchy of the prompt word structure of the fill reference materials, a domain-structured prompt word is obtained.

[0010] In conjunction with the first aspect, in the fifth implementation of the first aspect of the present invention, the role setting level defines the identity description of the technical supervision expert, the task constraint level defines the output specifications and prohibitions, the few sample example level includes a preset number of standard question and answer examples, and the thought chain reasoning level includes a step-by-step reasoning instruction sequence.

[0011] In conjunction with the first aspect, in the sixth implementation of the first aspect of the present invention, the technical supervision query question is embedded in the query hierarchy position of the prompt word structure of the filled reference materials to obtain domain structured prompt words, including: Locate the preset query level position identifier in the prompt word structure of the filled reference materials, replace the query level position identifier with the technical supervision query question, and obtain the fully filled prompt word structure; Following the preset splicing order of role setting level, task constraint level, reference material level, query level, few sample example level, and thought chain reasoning level, the text content of each level in the complete filled prompt word structure is spliced ​​in sequence, and preset separators are inserted between each level to obtain the domain structured prompt words.

[0012] In conjunction with the first aspect, in the seventh implementation of the first aspect of the present invention, the domain-structured cue words are input into a large-scale technically supervised domain model fine-tuned by LoRA low-rank adaptation for beam search to generate candidate answer text, including: The domain structured prompts are input into a large-scale technically supervised domain model fine-tuned by LoRA low-rank adaptation, and preset temperature coefficient, preset kernel sampling probability threshold, preset maximum generation length and preset repetition penalty coefficient are set as generation parameters to obtain the configured answer generation task. Based on the generation parameters, the configured answer generation task is executed with a bundle search algorithm until an end marker is generated or a preset maximum generation length is reached, resulting in multiple candidate answer sequences. The candidate answer sequence with the highest cumulative probability score is selected from the multiple candidate answer sequences as the candidate answer text.

[0013] In conjunction with the first aspect, in the eighth implementation of the first aspect of the present invention, a bundle search algorithm is executed on the configured answer generation task based on the generation parameters until an end marker is generated or a preset maximum generation length is reached, resulting in multiple candidate answer sequences, including: The configured answer generation task is initialized with a preset number of candidate generation paths with a specified bundle width. Each candidate generation path contains an initial token sequence and an initial cumulative probability score, resulting in a first set of candidate paths. For each candidate generation path in the first candidate path set, predict the probability distribution of the next token in the current decoding step. Combine each path with all its possible next tokens to form an extended path set. Calculate the cumulative probability score of each extended path. Select the extended paths with the highest cumulative probability scores (a preset number of bundle widths) to update the first candidate path set, thus obtaining the second candidate path set. The function for calculating the cumulative probability score is: ; Cumulative probability score Indicates the current decoding step number. This represents the length penalty coefficient. This indicates that the i-th token is generated. This represents the sequence of the first i-1 tokens. Indicates input prompt words, Represents conditional probability. Natural logarithm function; Determine whether there is a path in the second candidate path set that generates an end marker or whose token sequence length reaches the preset maximum generation length. If it exists, terminate the beam search and output all paths in the second candidate path set as multiple candidate answer sequences. If it does not exist, repeat the step of predicting the next token.

[0014] In conjunction with the first aspect, in the ninth implementation of the first aspect of the present invention, the candidate answer text is subjected to quality assessment through citation integrity detection, logical consistency detection, and entity illusion detection to obtain a qualified technical supervision standard answer, including: The candidate answer text is syntactically analyzed to identify factual statements. Each factual statement is checked to see if it has a corresponding citation tag. The ratio of the number of statements with citation tags to the total number of statements is calculated to obtain the citation integrity coverage score. The candidate answer text and the reference text with standard citation annotations are input into a natural language reasoning model to calculate the implication relationship. When the implication relationship judgment result is contradictory, the logical consistency score is set to zero. When the implication relationship judgment result is implication or neutral, the logical consistency score is set to one, thus obtaining a normalized logical consistency score. Entities in the candidate answer text are extracted and matched with a preset whitelist of entities in the technical supervision domain and the reference text with standard citation annotations. The number of unmatched entities is counted. The product of the number of unmatched entities and the preset illusion penalty coefficient is subtracted from one to obtain the entity illusion control score, thus obtaining the normalized entity illusion control score. The answer quality comprehensive score is obtained by multiplying the reference integrity coverage score by a first preset quality weight coefficient, the normalized logical consistency score by a second preset quality weight coefficient, and the normalized entity illusion control score by a third preset quality weight coefficient. The comprehensive answer quality score is then determined to be higher than a preset quality threshold. If it is higher, the candidate answer text is taken as the technical supervision standard answer that meets the quality requirements.

[0015] The technical solution provided by this invention achieves accurate semantic representation of technical supervision professional queries by constructing a query understanding mechanism that combines terminology standardization processing with BGE pre-trained model vector encoding. It improves the retrieval accuracy and result quality of technical supervision standard documents by employing cosine similarity calculation based on IVF inverted file index and a four-dimensional weighted ranking strategy (comprehensively considering semantic relevance, document authority, timeliness, and diversity penalties). By designing multi-level prompt word templates that include role settings, task constraints, few-sample examples, and thought chain reasoning, it effectively guides the domain-wide model, fine-tuned by LoRA low-rank adaptation, to understand technical supervision professional tasks and generate standardized answers. The beam search algorithm optimizes the candidate answer generation path, ensuring the fluency and accuracy of the answers. A three-dimensional quality evaluation system is established, encompassing citation integrity detection, logical consistency detection, and entity illusion detection, eliminating the illusion problem of the large model from the source and ensuring that all generated answers are supported by authoritative standard citations and have logical self-consistency. This invention organically integrates general intelligence of a general large model with professional knowledge in the field of technical supervision, and solves the technical problems of low standard retrieval accuracy, poor answer accuracy, lack of citation tracing and insufficient domain customization capabilities in the existing technology, providing a complete technical solution for intelligent applications in the field of power technology supervision.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of an embodiment of the plug-in function testing method for a pre-trained power system model based on NLP in this invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0021] To facilitate understanding of this embodiment, a detailed description of the plug-in functionality testing method for a pre-trained NLP-based power system model disclosed in this embodiment of the invention will be provided first. For example... Figure 1 As shown, this method includes the following steps: 101. Receive technical supervision query questions, perform vector encoding and retrieval on the technical supervision query questions, and obtain an initial retrieval result set; 102. The initial search result set is sorted by weight to obtain reference text with standard citation annotations. The reference text with standard citation annotations is then concatenated with the multi-level prompt word template to obtain domain structured prompt words. 103. Input the domain-structured prompts into a technically supervised domain-large model that has been fine-tuned with LoRA low-rank adaptation and perform beam search to generate candidate answer texts; 104. Conduct quality assessments on the candidate answer texts using citation integrity checks, logical consistency checks, and entity illusion checks to obtain a technically supervised standard answer that meets quality standards.

[0022] In one specific embodiment, a technical supervision query question is received, and the technical supervision query question is vector-encoded and retrieved to obtain an initial retrieval result set, including: The system receives technical supervision query questions, filters them for interjections, and replaces colloquial technical terms with corresponding standardized technical terms based on a pre-set standardized terminology dictionary to obtain the first query text. Obtain the character length of the first query text. When the character length is lower than the preset length threshold, extract extended words from the technical supervision terminology dictionary and co-occurrence word statistics table and add them to the first query text to obtain the second query text. The second query text is input into the BGE pre-trained model for word segmentation and Transformer encoding. The feature vectors of the CLS marker positions are extracted and used as the query semantic vectors. Based on the query semantic vector, cosine similarity calculation and retrieval are performed in the local vector knowledge base to obtain an initial retrieval result set.

[0023] Specifically, the input parsing module receives technical supervision queries from users and performs language-level standardization on these queries. It performs a filtering operation on interjections, using a predefined list of interjections. Interjections are used to improve the naturalness of interaction and are considered redundant in semantic retrieval; therefore, they are removed from the query text through rule matching or regular expressions, retaining only the main technical content of the query. The query text after removing interjections is then processed by the terminology standardization mapping module. This module uses a pre-built terminology standardization dictionary as the mapping basis, converting colloquial technical expressions in the query into standard terminology. For example, "tripping" is replaced with "unplanned shutdown," and "burned out" is replaced with "insulation breakdown" or "fault burnout." After terminology standardization and replacement, the first query text is obtained. The first query text undergoes length judgment processing. By counting the number of characters in the text, the completeness of the query expression is evaluated. If the character length is lower than a set length threshold (e.g., 10 Chinese characters), the query is deemed too brief and unable to fully express the user's true intent, triggering the query expansion module. The query extension module integrates a technical supervision terminology dictionary and a co-occurrence word statistics table constructed from query logs. It selects extension words with high semantic relevance to the keywords in the first query text; for example, "oxygen content" can be expanded into phrases like "oxygen content standard index range" and "oxygen content test method standard number." These extension words are appended to the end of the first query text and separated by spaces or newlines to construct the second query text. The second query text is then input into a pre-loaded BGE pre-trained semantic embedding model, where word segmentation is performed. The model's built-in Chinese word segmenter encodes the input text, mapping natural language into a token sequence that the model can process. This token sequence is then input into the multi-layer Transformer encoder within the BGE model. The Transformer encoder uses a multi-head self-attention mechanism to model the contextual semantics of the input sequence and continuously updates the vector representation at each position in each layer. After the encoding process, the hidden layer vector located at the [CLS] marker is extracted. Since this position is designed as a compressed representation of the entire sentence's semantics in BERT-like models, the vector at this position is used as the query semantic vector to represent the user's query position in the semantic space. The query semantic vector is taken as input, and a locally deployed vector database module is invoked to perform semantic retrieval. The retrieval process measures the similarity between each knowledge fragment vector in the local vector knowledge base and the current query semantic vector by calculating cosine similarity, measuring their similarity in a high-dimensional semantic space. The calculation formula is the inner product of the query vector and the candidate knowledge vector divided by the product of their moduli. The resulting similarity value ranges from -1 to 1, with higher values ​​indicating greater semantic relevance. All candidate text segments are sorted according to their similarity scores, and the highest-scoring segments are selected to form the initial retrieval result set.

[0024] After obtaining the query semantic vector and before performing cosine similarity calculation and retrieval, a query reconstruction step based on named entity recognition and query intent classification is included: The second query text undergoes named entity recognition processing, using a pre-trained technical supervision domain named entity recognition model to extract standard number entities, equipment name entities, and technical indicator entities, and each entity is labeled with an entity type label and confidence score to obtain entity-annotated query text; the entity-annotated query text is then input into a preset query intent classification model for intent recognition. This model is trained based on a technical supervision query type training set and classifies queries into standard query intent, indicator query intent, and fault query intent. The system identifies one of the following intentions: diagnostic intent, document generation intent, or comprehensive analysis intent. Based on the classification results, it retrieves the corresponding retrieval strategy parameters from a preset intent-retrieval strategy mapping table to obtain the intent classification results and retrieval strategy parameters. Based on the entity type weights in the entity-annotated query text and the retrieval strategy parameters, the query semantic vector is weighted and adjusted. The vector dimension weights corresponding to high-confidence standard number entities are increased by a first preset multiple, and the vector dimension weights corresponding to device name entities are increased by a second preset multiple. Entity constraint filtering conditions are generated based on the query intent type to obtain an enhanced query semantic vector. This enhanced query semantic vector is then used for cosine similarity calculation and retrieval.

[0025] After receiving the technical supervision query and before performing terminology standardization, the process includes a step of reference resolution and query enhancement based on dialogue history: This involves obtaining the dialogue history corresponding to the current session identifier, extracting the most recent preset number of historical question-and-answer pairs (each pair containing the user's historical question, generated answer, and key entities involved in the answer), constructing a dialogue history context sequence in chronological order to obtain structured dialogue history data; and performing pronoun detection processing on the technical supervision query, using preset pronoun recognition rules to identify pronoun-type pronouns, nominal pronouns, and ellipsis-type pronouns in the query, and performing reference pairing for each detected pronoun in the structured dialogue history data. Image matching calculates the matching score of candidate referents based on entity type consistency and semantic relevance, selects the historical entity with the highest matching score as the referent, and replaces the referent components in the technical supervision query with the corresponding referent entity to obtain the referent-resolved query text. Historical key entities and technical terms semantically related to the referent-resolved query text are extracted from the structured dialogue historical data. The extracted historical key entities are concatenated with the referent-resolved query text according to the preset context fusion template. The preset context fusion template defines the preceding position of the entities and the format of the separator to obtain the context-enhanced query text. The context-enhanced query text is used as the input for terminology standardization processing.

[0026] The second query text is input into the BGE pre-trained model for word segmentation and Transformer encoding. Feature vectors at the CLS marker positions are extracted and used as the query semantic vector. This process includes: inputting the second query text into the BGE pre-trained model's word segmenter for character-level word segmentation; inserting a special marker CLS at the beginning of the segmentation result and a special marker SEP at the end; querying a pre-defined vocabulary to obtain the corresponding token number for each token in the segmentation sequence; generating positional and fragment codes for each token; and summing the token number sequence, positional code sequence, and fragment code sequence to obtain the initial input embedding vector sequence. The initial input embedding vector sequence is then sequentially input into the twelve-layer Transformer encoder of the BGE pre-trained model for bidirectional context encoding. Each Transformer encoder layer contains a multi-head self-attention mechanism sublayer and a feedforward neural network sublayer. The multi-head self-attention mechanism sublayer calculates the query matrix, key matrix, and value matrix for the input vector sequence and uses scaled dot-products. Attention calculates the attention weight distribution of each token and all tokens in the sequence. Based on the attention weights, the value matrix is ​​weighted and summed to obtain the context-aware representation. The feedforward neural network sub-layer performs a two-layer fully connected transformation and GELU activation function processing on the context-aware representation. Each sub-layer is followed by residual connections and layer normalization operations. After twelve layers of encoding, the final hidden state vector sequence is obtained. The 768-dimensional hidden state vector corresponding to the CLS marker position is extracted from the final hidden state vector sequence. The hidden state vector at the CLS marker position integrates the bidirectional contextual semantic information of the entire query text sequence. The extracted 768-dimensional hidden state vector is normalized by L2 norm to obtain the query semantic vector.

[0027] In one specific embodiment, cosine similarity calculation and retrieval are performed in a local vector knowledge base based on the query semantic vector to obtain an initial retrieval result set, including: The metric distance is calculated in the IVF inverted file index of the local vector knowledge base based on the query semantic vector. The number of cluster centers with the closest metric distance is selected, and the knowledge base vector set corresponding to the cluster center is obtained. For each knowledge base vector in the knowledge base vector set, a dot product operation and vector magnitude normalization are performed with the query semantic vector to calculate the cosine similarity score. Knowledge base vectors with cosine similarity scores higher than the preset similarity threshold, along with their associated text block content, standard number, chapter title, and document update time, are selected. They are then sorted from high to low according to their cosine similarity scores, and a preset number of search records are extracted to obtain the initial search result set.

[0028] Specifically, a vector index structure based on an inverted file mechanism is constructed in the locally deployed vector database, employing the inverted file indexing method. The inverted file index pre-clusters all knowledge base vectors into several cluster centers. In the offline phase, the K-means algorithm is used to train and partition all vectors, forming several representative cluster centroids. Each knowledge vector is then assigned to its corresponding cluster bucket based on its distance to the nearest centroid. In the online phase, upon receiving the query semantic vector encoded by the query text using the BGE model, a vector distance metric is performed. The query vector is sequentially compared with all cluster centers, using Euclidean distance or inner product distance as the initial coarse screening standard. The closest cluster centers are then selected from all cluster centers. The number of clusters is controlled by the parameter `nprobe`, for example, setting it to 64 indicates that further fine-tuning and retrieval operations will be performed within 64 cluster buckets. After obtaining all knowledge base vectors corresponding to these cluster centers, a similarity calculation is performed between each candidate vector and the current query vector. To enhance the numerical stability and comparability of vector similarity calculation, a dot product operation combined with vector magnitude normalization is used to accurately calculate cosine similarity. The formula for calculating cosine similarity is the inner product of two vectors divided by the product of their magnitudes, i.e., Similarity(A, B) = (A·B) / (‖A‖ × ‖B‖), where A is the query semantic vector and B is the knowledge base vector. After calculation, all candidate vectors with cosine similarity scores higher than a set threshold are selected. The threshold is set at 0.65 or 0.7 to control the balance between retrieval precision and recall. For each knowledge base vector that meets the threshold condition, its associated metadata information is extracted synchronously, including the text block content to which the knowledge base vector belongs, the bound standard number, the chapter title, and the update time of the original document. All candidates that meet the condition are sorted from high to low according to the cosine similarity score. The highest-scoring records are selected from the Top-K (e.g., Top-10 or Top-5) according to the preset return quantity threshold, and used as the initial retrieval result set for the current retrieval task.

[0029] In one specific embodiment, the initial search result set is weighted and sorted to obtain reference text with standard citation annotations. This reference text with standard citation annotations is then concatenated with a multi-level prompt term template to obtain domain-structured prompt terms, including: Extract the cosine similarity score, document type corresponding to the standard number, document update time, and text block sequence number of the same standard document for each search record from the initial search result set; Based on a preset document type authority mapping table, document type is converted into document authority score; based on a preset timeliness decay rule, document update time is converted into timeliness score; and based on text block sequence number, diversity penalty coefficient is calculated. The cosine similarity score is multiplied by the first preset weight coefficient, the document authority score is multiplied by the second preset weight coefficient, the timeliness score is multiplied by the third preset weight coefficient, and the diversity penalty coefficient is multiplied by the fourth preset weight coefficient, and then the weighted sum is obtained to obtain the comprehensive ranking score of each search record. For each search record in the initial search result set, a citation mark is generated based on the comprehensive ranking score. The citation mark is then concatenated with the corresponding text block content to obtain the reference text with standard citation marks. By combining reference text with standard citations with multi-level prompt templates, a domain-structured prompt is obtained.

[0030] Specifically, in the initial search result set, each record contains a semantically matched text block and its associated basic information. From this, the cosine similarity score, the document type corresponding to the standard number, the document update time, and the sequential number of the text block within the same standard document (i.e., the text block sequence number) are extracted. The cosine similarity score reflects the semantic relevance between the query vector and the knowledge vector, while the standard number points to the original technical standard document. The document type field distinguishes whether the standard belongs to different levels such as national standards, industry standards, enterprise standards, or case studies. The document update time measures the freshness of the content, and the text block sequence number reflects the relative position of the text content within the same document. The extracted document type field is converted into a document authority score by consulting a pre-defined document type authority mapping table. The mapping table defines the authority levels of different standard types; for example, national standards are mapped to 1.0, industry standards to 0.9, enterprise standards to 0.8, and case study materials to 0.7. Simultaneously, a timeliness decay rule function is invoked based on the document update time field. This function calculates a linear or exponential decay over the number of years; for example, updates within one year score 1.0, updates from two to three years score 0.9, updates from three to five years score 0.8, and updates from five years or more score 0.7. To avoid semantic redundancy caused by multiple search results clustering in adjacent areas of the same standard document, a diversity penalty coefficient is generated using the text block number as input and an exponential decay formula. For example, for each standard document, the score of each search result starting from the second text block is multiplied by 0.9. (n-1)Where n is the sequence number of the current text block in the standard, with the first being 1.0, the second 0.9, and so on. Each scoring factor is calculated using preset weighting coefficients to form a comprehensive ranking score. The cosine similarity score is multiplied by the first preset weighting coefficient α1, the document authority score by the second preset weighting coefficient α2, the timeliness score by the third preset weighting coefficient α3, and the diversity penalty coefficient by the fourth preset weighting coefficient α4. These four factors are then added together to obtain the comprehensive score for the search record: Score = α1×S1 + α2×S2 + α3×S3 - α4×P, where S1 is the semantic similarity, S2 is the document authority score, S3 is the timeliness score, and P is the diversity penalty. All initial search results are reordered from highest to lowest according to this comprehensive score, and the top K results are selected as the final reference content. For each result in the final ranking, a unique citation label is generated based on its standard number and text block position, using a uniform format such as "[Standard Number-Chapter Number]". The citation markers are concatenated with the corresponding original text blocks to form structured reference entries with citation traceability. All entries are then merged into the reference text section. This reference text is embedded into predefined multi-level prompt templates. These templates include character descriptions, task output constraints, few-shot reasoning examples, context embedding areas, and multi-turn dialogue history caches. By inserting the references into these dedicated areas, the prompt content is dynamically customized, resulting in domain-structured prompt text.

[0031] After obtaining the reference text with standard citations and before concatenating multi-level prompt word templates, a dynamic prompt word strategy selection step based on search result features is included: Feature analysis is performed on the reference text with standard citations; the average cosine similarity score of the reference text is calculated as the result confidence index; the number of different standard numbers in the reference text is counted as the coverage index; consistency verification is performed on the technical indicator values ​​and standard clauses in the reference text to calculate the consistency index; the result confidence index, coverage index, and consistency index are combined into a search result quality feature vector; the search result quality feature vector is input into a preset prompt word template selection decision tree model for classification processing. The prompt word template selection decision tree model is trained based on historical question-and-answer performance data. Based on the numerical range of the search result quality feature vector, the query scenario is classified into high-confidence direct citation scenario, medium-confidence inference comprehensive scenario, or low-confidence multi-source verification scenario. The initial prompt word templates are selected from the template library based on the scenario classification results. For high-confidence scenarios that directly cite the original text, the templates for medium-confidence scenarios that incorporate reasoning levels within the thought chain are selected. For low-confidence scenarios that utilize multi-source verification, the templates for these scenarios add multi-angle argumentation requirements, resulting in the initial prompt word templates. Based on the coverage and consistency metrics in the quality feature vector of the search results, the parameters of the initial prompt word templates are adaptively adjusted. When the coverage metric is higher than the first preset coverage threshold, the maximum length parameter of the context in the prompt word template is expanded to the first preset length value. When the coverage metric is lower than the second preset coverage threshold, the maximum length parameter of the context is compressed to the second preset length value. When the consistency metric is lower than the preset consistency threshold, a multi-source cross-validation instruction is added to the task constraint level of the prompt word template, resulting in a multi-level prompt word template with optimized parameters. This optimized multi-level prompt word template is then used to concatenate with the reference text.

[0032] In one specific embodiment, reference text with standard citations is concatenated with a multi-level prompt template to obtain domain-structured prompts, including: Construct a multi-level prompt word template that includes a role setting level, a task constraint level, a few-sample example level, and a thought chain reasoning level; The reference text with standard citations is embedded into the reference hierarchy position of the multi-level prompt word template according to the preset reference format, resulting in the prompt word structure for filling references; By embedding technical supervision query questions into the query hierarchy of the prompt word structure used to fill in reference materials, we obtain domain-structured prompt words.

[0033] Specifically, a hierarchical prompt word template is constructed, consisting of four components: a role setting level, a task constraint level, a few-shot example level, and a thought chain reasoning level. The role setting level specifies the knowledge expert role the model should simulate in the current task, stimulating the relevant semantic patterns learned during the model's pre-training phase. The task constraint level clearly defines the output format, citation conventions, language requirements, and issues to be avoided for the generated content. The few-shot example level embeds two to three question-answer examples that conform to the task specifications, creating a few-shot learning environment and guiding the model to imitate the output format and content logic. The thought chain reasoning level guides the model to think about the problem step-by-step, clarifying the order of solutions. After completing the initial structure of the multi-level prompt word template, the retrieved reference text with standard citations is embedded into the reference level of the template according to a preset format. This position is located between the role setting and the question, and multiple reference texts are organized by number, with all reference items arranged in a structured list. During the reference insertion process, the overall prompt structure is controlled to not exceed the upper limit of the model context window (e.g., 4096 tokens). Any excess is truncated or compressed. Additionally, materials containing table information are converted to Markdown format for embedding to maintain structural consistency. User-submitted technical supervision queries are embedded at the query level within the populated reference prompt structure. This query level follows all guidance information and reference text, and is clearly labeled to form domain-structured prompts.

[0034] In one specific embodiment, the role setting level defines the identity description of the technical supervision expert, the task constraint level defines the output specifications and prohibitions, the few sample examples level contains a preset number of standard question and answer examples, and the thought chain reasoning level contains a step-by-step reasoning instruction sequence.

[0035] Specifically, the role setting level defines the identity description of the technical supervisor expert. This level uses natural language to describe the knowledge role and professional background the model should assume in the current task. Role setting enhances the domain adaptability and terminology matching ability of the generated content by activating semantic distribution patterns related to that role within the larger model, and provides a contextual background of the professional stance. The task constraint level defines output specifications and prohibitions. The output specifications explicitly require that the model's generated content must cite standard clauses, indicator values, or chapter content from the reference materials, and adhere to requirements such as clear answer structure, consistent citation format, and formal language. The prohibitions set boundaries for model generation through negative rules, such as "do not fabricate technical indicators or standard clauses not appearing in the materials," "do not use vague expressions or subjective judgments," and "if there is no clear basis, answer 'the reference materials are insufficient to support the answer to this question.'" To guide the model in learning the output logic and format specifications that conform to the question-and-answer scenarios of power technology supervision, a preset number of standard question-and-answer examples are embedded in the few-shot example level. Two to three representative pairs of real questions and standard answers are selected, each pair including a clear question description, answer content extracted based on specific standard clauses, and citation annotations. This few-shot learning mechanism enhances the model's mastery of task patterns and output templates. In the thought chain reasoning level, an instruction sequence is constructed to guide the large model to complete complex reasoning tasks step by step. This level adopts a clear step division method, such as "Step 1: Identify the core entities and indicator types in the question; Step 2: Locate the relevant chapters or technical clauses in the references; Step 3: Extract key values ​​or regulatory information; Step 4: Organize the answer and annotate the citation sources; Step 5: Verify whether the answer is complete, accurate, and conforms to the task constraints."

[0036] In one specific embodiment, the technical supervision query question is embedded in the query hierarchy of the prompt word structure for filling reference materials to obtain domain-structured prompt words, including: Locate the preset query level location identifier in the prompt word structure for filling in reference materials, replace the query level location identifier with the technical supervision query question, and obtain the complete prompt word structure. Following the preset splicing order of role setting level, task constraint level, reference material level, query level, few sample example level, and thought chain reasoning level, the text content of each level in the complete and filled prompt word structure is spliced ​​in sequence, and preset separators are inserted between each level to obtain the domain structured prompt words.

[0037] Specifically, within the populated reference prompt structure, a template parser locates a pre-defined placeholder identifier for the query level. This placeholder identifier exists in special format such as "[Current Question]" or "<|query|>", marking the position where the user query should be inserted within the structure. The parameter injection module replaces the position identifier with the current round of technical supervision query question, accurately embedding the user input into the target position within the template. This creates contextual coupling between the user input and the reference materials, role settings, and other content, resulting in a populated prompt structure. To meet the formatting requirements of structured reasoning, all components are merged and serialized in a fixed order. The following steps are implemented: Insert the role-setting level text as the first paragraph, defining the model's knowledge identity and expert background; concatenate the task constraint level text, specifying output format requirements and prohibitions; insert the reference level with embedded standard citations, providing authoritative semantic context to support answer generation; insert the query level content that has just undergone a replacement operation, ensuring the question semantics continue from the reference text; concatenate the few-sample example level, containing several standard question-answer pairs to guide the model in mimicking generation logic and structural patterns; and concatenate the thought chain reasoning level content, using guided reasoning sequences to stimulate step-by-step logical processing, improving the interpretability and accuracy of the generated content. Separators with a pre-defined format are uniformly inserted between each level, using markers such as "\n---\n" or "<|section|>" to enhance the hierarchical structure and assist the model in recognizing segment boundaries. The preset assembly order is: role setting → task constraints → reference materials → query question → example → reasoning chain. The model reasoning starts from the identity background, combines the norms and literature semantics to understand the question intent, and then deduces the final answer under the guidance of the case.

[0038] The technically supervised domain-wide model, fine-tuned with LoRA low-rank adaptation, was obtained through the following training method: A pre-trained pedestal large language model was acquired. This pedestal large language model adopted a decoder-only Transformer architecture with 13 billion parameters, containing several Transformer decoder layers. Each Transformer decoder layer included a causal self-attention mechanism module and a feedforward neural network module. LoRA low-rank adapters were injected into the bypass positions of the query projection matrix, key projection matrix, value projection matrix, and output projection matrix of the causal self-attention mechanism module. Each LoRA low-rank adapter consists of a pair of rank decomposition matrices. The dimensionality change of the original weight matrix is ​​decomposed into the multiplication of two low-rank matrices that are first reduced in dimensionality and then increased in dimensionality. The rank parameter of the dimensionality-reduced matrix is ​​set to a preset rank value. The 13 billion original parameters of the base large language model are frozen, and training is performed only on the injected LoRA low-rank adapter parameters. The total number of trainable parameters of the LoRA low-rank adapter is 30 million, resulting in a parameter-frozen LoRA augmented base model. A technical supervision domain instruction fine-tuning training dataset is constructed. The training dataset contains several training samples, each training sample adopting the instruction-input-output triplet format. The technical supervision task type and constraints are defined. The input part includes user questions and retrieved standard references, and the output part is the standard answer and citation annotations. The training dataset is divided into training set, validation set, and test set according to a preset ratio. The training samples in the training set are sequentially input into the LoRA enhanced base model with frozen parameters for forward propagation calculation. For each training sample, the instruction part and the input part are concatenated as the model input. The model generates an output sequence through autoregression. The cross-entropy loss between the output sequence generated by the model and the standard output in the training samples is calculated. The loss is calculated only for the tokens in the output part, and not for the instructions and input parts. The token setting mask is ignored. The gradient of the loss with respect to the LoRA low-rank adapter parameters is calculated through the backpropagation algorithm. The AdamW optimizer is used to update the LoRA low-rank adapter parameters with a preset learning rate. The preset learning rate adopts a cosine annealing strategy to gradually decay from the initial learning rate to the minimum learning rate. The batch size and the number of training epochs are set. After each training epoch, the perplexity of the model is evaluated on the validation set. Training is stopped early when the perplexity of the validation set does not decrease for a preset number of consecutive times. After training is completed, the LoRA low-rank adapter weights are merged with the base model weights to obtain a large-scale domain model for technical supervision that has been fine-tuned by LoRA low-rank adaptation.

[0039] The domain-structured prompts are input into a large-scale supervised domain model fine-tuned with LoRA low-rank adaptation. Preset parameters for temperature coefficient, kernel sampling probability threshold, maximum generation length, and repetition penalty coefficient are set to generate the configured answer. This process includes: tokenizing the domain-structured prompts; using the large-scale supervised domain model's segmenter to convert the prompt text into a token sequence; calculating the total length of the token sequence; verifying if the total length exceeds the maximum context window length of the large-scale supervised domain model; if it does, truncating from the reference level and retaining the preceding content; using the encoded token sequence as the initial input context for the model to obtain the encoded prompt context; configuring the decoding control parameters for answer generation; setting the temperature coefficient parameter value to adjust the smoothness of the output probability distribution (a lower temperature coefficient makes the model output more deterministic); setting the kernel sampling probability threshold parameter to limit the cumulative probability range of candidate tokens; and performing kernel sampling... The sampling probability threshold parameter is set to the second preset value, and only the smallest token candidate set whose cumulative probability reaches the threshold is sampled. The maximum generation length parameter is set to limit the upper limit of the number of tokens generated in the answer, and the maximum generation length parameter is set to the third preset value. The repetition penalty coefficient parameter is set to reduce the probability of repeated occurrence of generated tokens, and the repetition penalty coefficient parameter is set to the fourth preset value. Repetition suppression is achieved by multiplying the logits value of the already occurred tokens by the penalty coefficient, thus obtaining the decoding control parameter configuration. Based on the encoded prompt word context and the decoding control parameter configuration, the answer generation task state is initialized. The answer generation task state includes the currently generated token sequence, the current decoding step counter, the cumulative generation probability score, and the generation end flag. The currently generated token sequence is initialized to the encoded prompt word context, the current decoding step counter is initialized to zero, the cumulative generation probability score is initialized to a unit value, and the generation end flag is initialized to an unfinished state, thus obtaining the configured answer generation task.

[0040] In one specific embodiment, the domain-structured cue words are input into a technically supervised domain-large model fine-tuned by LoRA low-rank adaptation for beam search to generate candidate answer text, including: The domain structured prompts are input into a large-scale technically supervised domain model that has been fine-tuned by LoRA low-rank adaptation. Preset temperature coefficient, preset kernel sampling probability threshold, preset maximum generation length and preset repetition penalty coefficient are set as generation parameters to obtain the configured answer generation task. Based on the generated parameters, the configured answer generation task is executed with a bundle search algorithm until an end marker is generated or the preset maximum generation length is reached, resulting in multiple candidate answer sequences; The candidate answer sequence with the highest cumulative probability score is selected from multiple candidate answer sequences as the candidate answer text.

[0041] Specifically, structured domain prompts are input into a large-scale supervised model fine-tuned using LoRA low-rank adaptation. Based on pre-training, the large model undergoes structured weight adjustments for specific supervised knowledge domains by introducing the LoRA fine-tuning strategy, thereby enhancing its generalization ability and output consistency in tasks such as standard question answering, text comprehension, and specification generation. To ensure that the candidate answer generation process meets specific control conditions, a set of core generation parameters are set during the generation task configuration phase. These include: a temperature coefficient, controlling the smoothness of the model's generation probability distribution to determine the trade-off between output diversity and conservatism; a kernel sampling probability threshold, used to control the cumulative probability quality coverage range in the Top-p sampling mechanism, limiting the random range of sampled words; a maximum generation length, used to set the upper limit of the generated text length to prevent the output content from being verbose or infinitely extended; and a repetition penalty coefficient, used to suppress repetitive segments and enhance the information density and reading quality of the output content. The answer generation task constructed based on the above parameters triggers the execution of the bundle search algorithm. The bundle search mechanism constructs a search tree structure for candidate answers by retaining a preset number of probabilistic optimal sequence paths in each generation step and expanding the next possible output marker on each path. In each generation cycle, the cumulative probability score of the candidate sequences is continuously calculated, and the process terminates with an end marker or reaching the maximum generation length, outputting a set of sequences containing several candidate answers. By comparing the cumulative probability scores of all candidate answers, the sequence with the highest score is selected as the final candidate answer text.

[0042] In one specific embodiment, a beam search algorithm is executed on the configured answer generation task based on the generation parameters until an end marker is generated or a preset maximum generation length is reached, resulting in multiple candidate answer sequences, including: The configured answer generation task is initialized with a preset number of candidate generation paths with a specified bundle width. Each candidate generation path contains an initial token sequence and an initial cumulative probability score, resulting in the first set of candidate paths. For each candidate generation path in the first candidate path set, predict the probability distribution of the next token in the current decoding step. Combine each path with all its possible next tokens to form an extended path set. Calculate the cumulative probability score of each extended path. Select the extended paths with the highest cumulative probability scores (a preset number of bundle widths) to update the first candidate path set, thus obtaining the second candidate path set. The function for calculating the cumulative probability score is: ; Cumulative probability score Indicates the current decoding step number. This represents the length penalty coefficient. This indicates that the i-th token is generated. This represents the sequence of the first i-1 tokens. Indicates input prompt words, Represents conditional probability. Natural logarithm function; Determine if there is a path in the second candidate path set that generates an end marker or whose token sequence length reaches the preset maximum generation length. If it exists, terminate the beam search and output all paths in the second candidate path set as multiple candidate answer sequences. If it does not exist, repeat the step of predicting the next token.

[0043] Specifically, during the answer generation task initiation phase, a preset bundle width parameter is read, and a preset number of candidate generation paths are initialized accordingly. Each candidate generation path contains an initial token sequence and an initial cumulative probability score, forming a first candidate path set. The initial token sequence is determined by the encoding sequence of the start marker and the domain structured prompt words, while the initial cumulative probability score represents the baseline score of the current path before expansion. The process then proceeds to the stepwise decoding phase, where each candidate generation path in the first candidate path set is used to predict the next token. The generated token sequence corresponding to the candidate generation path is forward-reasoned with the input prompt word in the large-scale model of the supervised domain to obtain the probability distribution of the next token in the vocabulary space. Each candidate generation path is then combined with all possible next tokens in the probability distribution to expand each original path into multiple branch paths, which are then aggregated into an expanded path set. To perform an effective global comparison of the extended path set, a cumulative probability score is calculated for each extended path. Based on the cumulative probability score, the extended path set is sorted and truncated. A preset number of extended paths with the highest cumulative probability scores are selected from the extended path set to update the first candidate path set and obtain the second candidate path set. This ensures that the second candidate path set retains several of the best generated trajectories at each step to steadily improve the overall quality of the final answer. A termination condition is determined for the second candidate path set, i.e., whether there is a path in the second candidate path set that generates an end marker or whose token sequence length reaches a preset maximum generation length. If there is at least one path in the second candidate path set that meets the termination condition, the bundle search is terminated and all paths in the second candidate path set are output as multiple candidate answer sequences. If there is no path in the second candidate path set that meets the termination condition, the second candidate path set is regarded as the new first candidate path set, and the process returns to the step of predicting the next token to continue iterating. Thus, under the combined effect of bundle width constraints, length penalty constraints, and repetition penalty constraints, the process gradually approaches the candidate answer sequence set with the optimal cumulative probability score and content that better conforms to the technical supervision context.

[0044] In one specific embodiment, the candidate answer text is subjected to quality assessments including citation integrity detection, logical consistency detection, and entity illusion detection to obtain a technically supervised standard answer that meets quality standards, including: Syntactic analysis is performed on the candidate answer text to identify factual statements. Each factual statement is checked to see if it has a corresponding citation tag. The ratio of the number of statements with citation tags to the total number of statements is calculated to obtain the citation integrity coverage score. The candidate answer text and the reference text with standard citation annotations are input into a natural language reasoning model to calculate the implication relationship. When the implication relationship judgment result is contradictory, the logical consistency score is set to zero. When the implication relationship judgment result is implication or neutral, the logical consistency score is set to one, thus obtaining a normalized logical consistency score. Entities in the candidate answer text are extracted and matched with a preset whitelist of entities in the technical supervision domain and the reference text with standard citation annotations. The number of unmatched entities is counted. The product of the number of unmatched entities and the preset illusion penalty coefficient is subtracted from one to obtain the entity illusion control score, thus obtaining the normalized entity illusion control score. The answer quality comprehensive score is obtained by multiplying the reference integrity coverage score by a first preset quality weight coefficient, the normalized logical consistency score by a second preset quality weight coefficient, and the normalized entity illusion control score by a third preset quality weight coefficient. The comprehensive answer quality score is then determined to be higher than a preset quality threshold. If it is higher, the candidate answer text is taken as the technical supervision standard answer that meets the quality requirements.

[0045] Specifically, the technical supervision standard documents stored in the local vector knowledge base were obtained through the following dataset construction method: PDF documents of national standards, industry standards, and enterprise standards were processed in parallel using PaddleOCR, Tesseract, and ABBYY recognition engines. Character sequences output by the three engines were extracted from the same position in the same document, and a character-by-character comparison was performed. If the recognition results of the three engines were completely consistent, the character sequence was directly adopted. If two engines showed consistent results but the third differed, the character sequence with the majority consistency was adopted. Not simultaneously marking the location as pending manual review and recording it in the review list, after performing character-level voting on the entire document, a preliminary identified text and a list of locations pending review are obtained. The text in the list of locations pending review is manually proofread and corrected to obtain the verified standard document text. A preset number of standard question-and-answer pairs are extracted from a preset technical supervision training question bank as a verification set. Each standard question-and-answer pair contains question text and standard answer text. The verified standard document text is semantically segmented according to chapter structure and converted into vectors, which are then stored in a temporary vector database. The question texts in the verification set are input one by one into the temporary vector database for vector retrieval. The returned answer text is compared with the corresponding standard answer text using edit distance similarity and BLEU score. When the similarity is lower than a preset verification threshold, the source document section involved in the question-and-answer pair is located, marked as an area with questionable accuracy, and pushed to the manual review queue for OCR recognition and proofreading again. After completing the retrieval and verification of all verification set question-and-answer pairs, the overall verification pass rate is calculated. When the verification pass rate is higher than the preset pass rate threshold, the dataset quality is confirmed to be qualified, and a set of standard document texts that have passed quality verification is obtained. For each document in the set of standard document texts that have passed quality verification, regular expression matching technology is used to extract other standard numbers referenced in the document. The regular expression matching pattern is GB slash T space or hyphen number dot number hyphen four-digit year standard number format. For each extracted pair of reference relationships, a main standard node, cited standard node and reference relationship edge are constructed. The standard number, standard name, publication time and scope of application of the main standard node are used as node attributes, and the reference type is marked as normative reference or reference reference as edge attribute. The reference relationship network of all standard documents is stored using a graph database. When a standard document is updated, all related documents that cite the standard are automatically identified through graph query and marked as needing to be updated synchronously, resulting in a technical supervision standard document dataset containing the reference relationship graph.

[0046] The process involves converting document types into document authority scores based on a preset document type authority mapping table, converting document update times into timeliness scores based on preset timeliness decay rules, calculating diversity penalty coefficients based on text block serial numbers, and weighted summing of the four-dimensional scores to obtain a comprehensive ranking score. This includes: querying the document type corresponding to the standard number from the preset document type authority mapping table (document types include national standards, industry standards, enterprise standards, and case study documents); assigning a first authority score to national standards, a second authority score to industry standards, and a third authority score to enterprise standards based on the preset document type authority mapping table. A fourth authority score is assigned to the case document type. The first authority score is greater than the second, the second is greater than the third, and the third is greater than the fourth. The corresponding authority score is obtained from the query results as the document's authority score. The document age is calculated by the time difference between the document's update time and the current system time. The document age is segmented and mapped according to a preset timeliness decay rule. When the document age is less than a first preset age threshold, the timeliness score is set to the first timeliness score. When the document age is between the first and second preset age thresholds, the timeliness score is set to the second timeliness score. When the document age is between the second and third preset age thresholds, the timeliness score is set as the third timeliness value. When the document age exceeds the third preset age threshold, the timeliness score is set as the fourth timeliness value. The first timeliness value is greater than the second timeliness value, the second timeliness value is greater than the third timeliness value, and the third timeliness value is greater than the fourth timeliness value, thus obtaining the timeliness score. The number of text blocks appearing for each standard number in the initial search result set is counted. For search records with a text block number greater than one, the text block number of that search record is subtracted by one to obtain the duplicate number value. The duplicate number value raised to the power of the preset diversity penalty base is used as the diversity penalty coefficient for that search record. The base of the gender penalty is less than one. For search records with a text block number equal to one, the diversity penalty coefficient is set to a unit value to obtain the diversity penalty coefficient for each search record. The cosine similarity score is multiplied by the first preset weight coefficient to obtain the weighted semantic relevance item. The document authority score is multiplied by the second preset weight coefficient to obtain the weighted document authority item. The timeliness score is multiplied by the third preset weight coefficient to obtain the weighted timeliness item. The diversity penalty coefficient is multiplied by the fourth preset weight coefficient to obtain the weighted diversity penalty item. The weighted semantic relevance item, weighted document authority item, weighted timeliness item, and weighted diversity penalty item are arithmetically summed to obtain the comprehensive ranking score.

[0047] The process of performing syntactic analysis on candidate answer texts to identify factual statements and calculate citation integrity coverage scores, performing implication relation calculations to obtain normalized logical consistency scores, performing entity matching to obtain normalized entity illusion control scores, and calculating a comprehensive answer quality score based on 3D detection results includes: using a pre-defined syntactic analysis model to perform dependency parsing on candidate answer texts, identifying statements containing complete subject-verb-object structures, determining whether each identified statement contains factual content such as specific numerical values, standard numbers, technical indicators, or equipment parameters, marking statements containing factual content as factual statements, counting the total number of factual statements, and using regular expression matching techniques to detect each factual statement. Whether a statement is immediately followed by a bracketed citation marker (the citation marker format is a standard number followed by a hyphenated chapter number), count the number of factual statements with citation markers, and calculate the quotient of the number of factual statements with citation markers divided by the total number of factual statements to obtain the citation integrity coverage score; segment the candidate answer text into several independent sentence fragments, pair each sentence fragment with reference text with standard citation markers, concatenate the paired sentence fragments and reference texts, and input them into a pre-set natural language inference model. The natural language inference model is trained based on an entailment relation classification task, and outputs the probability distribution of three types of relations: entailment, contradiction, or neutral for the input text pairs. When the probability of an entailment relation is... The system determines the logical consistency of a sentence fragment with the highest probability of being consistent with the reference material. It determines a logical contradiction when the probability of a contradictory relationship is highest. The system statistically analyzes the logical relationship determination results for all sentence fragments. When a logically contradictory sentence fragment exists, the implication relationship determination result is marked as contradictory; otherwise, it is marked as either implication or neutral. When the implication relationship determination result is contradictory, the normalized logical consistency score is set to zero; when it is either implication or neutral, the normalized logical consistency score is set to one, resulting in the normalized logical consistency score. A pre-defined named entity recognition model is used to extract supervised domain entities from candidate answer texts. Entity types include... The system includes entities with standard numbers, equipment names, technical indicators, and numerical values. For each extracted entity, the system checks whether it exists in the preset whitelist of entities in the technical supervision field. If it does not exist, the system uses string matching to check whether the entity appears in the reference text with standard references. If the entity is not found in either the entity whitelist or the reference text, the entity is marked as a suspected hallucination entity and the entity text and entity type are recorded. The number of suspected hallucination entities is counted, and the difference between one and the number of suspected hallucination entities multiplied by the preset hallucination penalty coefficient is calculated as the normalized entity hallucination control score. When the calculation result is less than zero, the normalized entity hallucination control score is truncated to zero to obtain the normalized entity hallucination control score.The weighted reference integrity score is obtained by multiplying the reference integrity coverage score by a first preset quality weighting coefficient. The weighted logical consistency score is obtained by multiplying the normalized logical consistency score by a second preset quality weighting coefficient. The weighted entity illusion control score is obtained by multiplying the normalized entity illusion control score by a third preset quality weighting coefficient. The weighted reference integrity score, weighted logical consistency score, and weighted entity illusion control score are then summed arithmetically to obtain the overall answer quality score.

[0048] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0049] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0050] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A plug-in functionality testing method for a pre-trained power system model based on NLP, characterized in that, include: Receive technical supervision query questions, and perform vector encoding and retrieval on the technical supervision query questions to obtain an initial retrieval result set; The initial search result set is sorted by weight to obtain reference text with standard citation annotations. The reference text with standard citation annotations is then concatenated with a multi-level prompt word template to obtain domain structured prompt words. The domain structured cue words are input into a technically supervised domain large model that has been fine-tuned with LoRA low-rank adaptation for beam search to generate candidate answer text; The candidate answer texts are subjected to quality assessments including citation integrity detection, logical consistency detection, and entity illusion detection to obtain a technically supervised standard answer that meets quality standards.

2. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 1, characterized in that, The system receives technical supervision query questions, performs vector encoding and retrieval on the technical supervision query questions, and obtains an initial retrieval result set, including: The system receives technical supervision query questions, filters the technical supervision query questions for interjections, and replaces colloquial technical terms with corresponding standardized technical terms based on a preset standardized terminology dictionary to obtain the first query text. Obtain the character length in the first query text. When the character length is lower than a preset length threshold, extract extended words from the technical supervision terminology dictionary and co-occurrence word statistics table and add them to the first query text to obtain the second query text. The second query text is input into the BGE pre-trained model for word segmentation and Transformer encoding. The feature vectors of the CLS marker positions are extracted and used as the query semantic vectors. Based on the query semantic vector, cosine similarity calculation and retrieval are performed in the local vector knowledge base to obtain an initial retrieval result set.

3. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 2, characterized in that, Based on the query semantic vector, cosine similarity calculation and retrieval are performed in the local vector knowledge base to obtain an initial retrieval result set, including: Based on the query semantic vector, a metric distance is calculated in the IVF inverted file index of the local vector knowledge base. A preset number of cluster centers with the closest metric distance are selected, and the knowledge base vector set corresponding to the cluster centers is obtained. For each knowledge base vector in the knowledge base vector set, perform a dot product operation and vector magnitude normalization on the query semantic vector to calculate the cosine similarity score. Filter out knowledge base vectors with cosine similarity scores higher than a preset similarity threshold, along with their associated text block content, standard number, chapter title, and document update time. Sort them by cosine similarity score from high to low and extract a preset number of search records to obtain the initial search result set.

4. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 1, characterized in that, The initial search result set is weighted and sorted to obtain reference text with standard citation annotations. This reference text with standard citation annotations is then concatenated with a multi-level prompt term template to obtain domain-structured prompt terms, including: Extract the cosine similarity score, document type corresponding to the standard number, document update time, and text block sequence number of the same standard document for each search record from the initial search result set; Based on a preset document type authority mapping table, the document type is converted into a document authority score; based on a preset timeliness decay rule, the document update time is converted into a timeliness score; and based on the text block sequence number, a diversity penalty coefficient is calculated. The cosine similarity score is multiplied by a first preset weight coefficient, the document authority score is multiplied by a second preset weight coefficient, the timeliness score is multiplied by a third preset weight coefficient, and the diversity penalty coefficient is multiplied by a fourth preset weight coefficient, and then weighted and summed to obtain the comprehensive ranking score of each search record. For each search record in the initial search result set, a citation mark is generated based on the comprehensive ranking score. The citation mark is then concatenated with the corresponding text block content to obtain reference text with standard citation marks. By concatenating the reference text with standard citation annotations with the multi-level prompt word template, a domain-structured prompt word is obtained.

5. The plug-in function testing method for the NLP-based power system pre-trained model according to claim 4, characterized in that, By concatenating the reference text with standard citations and the multi-level prompt word template, a domain-structured prompt word is obtained, including: Construct a multi-level prompt word template that includes a role setting level, a task constraint level, a few-sample example level, and a thought chain reasoning level; The reference text with standard citations is embedded into the reference level position of the multi-level prompt word template according to the preset reference format to obtain the prompt word structure for filling references; By embedding the technical supervision query question into the query hierarchy of the prompt word structure of the fill reference materials, a domain-structured prompt word is obtained.

6. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 5, characterized in that, The role setting level defines the identity description of the technical supervision expert, the task constraint level defines the output specifications and prohibitions, the few sample example level contains a preset number of standard question and answer examples, and the thought chain reasoning level contains a step-by-step reasoning instruction sequence.

7. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 6, characterized in that, By embedding the technical supervision query question into the query hierarchy of the prompt word structure of the filled reference materials, domain-structured prompt words are obtained, including: Locate the preset query level position identifier in the prompt word structure of the filled reference materials, replace the query level position identifier with the technical supervision query question, and obtain the fully filled prompt word structure; Following the preset splicing order of role setting level, task constraint level, reference material level, query level, few sample example level, and thought chain reasoning level, the text content of each level in the complete filled prompt word structure is spliced ​​in sequence, and preset separators are inserted between each level to obtain the domain structured prompt words.

8. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 1, characterized in that, The domain-structured cue words are input into a technically supervised domain-wide model fine-tuned with LoRA low-rank adaptation for beam search, generating candidate answer texts, including: The domain structured prompts are input into a large-scale technically supervised domain model fine-tuned by LoRA low-rank adaptation, and preset temperature coefficient, preset kernel sampling probability threshold, preset maximum generation length and preset repetition penalty coefficient are set as generation parameters to obtain the configured answer generation task. Based on the generation parameters, the configured answer generation task is executed with a bundle search algorithm until an end marker is generated or a preset maximum generation length is reached, resulting in multiple candidate answer sequences. The candidate answer sequence with the highest cumulative probability score is selected from the multiple candidate answer sequences as the candidate answer text.

9. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 8, characterized in that, Based on the generation parameters, a beam search algorithm is executed on the configured answer generation task until an end marker is generated or a preset maximum generation length is reached, resulting in multiple candidate answer sequences, including: The configured answer generation task is initialized with a preset number of candidate generation paths with a specified bundle width. Each candidate generation path contains an initial token sequence and an initial cumulative probability score, resulting in a first set of candidate paths. For each candidate generation path in the first candidate path set, predict the probability distribution of the next token in the current decoding step. Combine each path with all its possible next tokens to form an extended path set. Calculate the cumulative probability score of each extended path. Select the extended paths with the highest cumulative probability scores (a preset number of bundle widths) to update the first candidate path set, thus obtaining the second candidate path set. The function for calculating the cumulative probability score is: ； Represents the cumulative probability score. Indicates the current decoding step number. This represents the length penalty coefficient. This indicates that the i-th token is generated. This represents the sequence of the first i-1 tokens. Indicates input prompt words, Represents conditional probability. Represent the natural logarithm function; Determine whether there is a path in the second candidate path set that generates an end marker or whose token sequence length reaches the preset maximum generation length. If it exists, terminate the beam search and output all paths in the second candidate path set as multiple candidate answer sequences. If it does not exist, repeat the step of predicting the next token.

10. The plug-in functionality testing method for a pre-trained NLP-based power system model according to claim 1, characterized in that, The candidate answer texts are subjected to quality assessments including citation integrity detection, logical consistency detection, and entity illusion detection to obtain a qualified technical supervision standard answer, including: The candidate answer text is syntactically analyzed to identify factual statements. Each factual statement is checked to see if it has a corresponding citation tag. The ratio of the number of statements with citation tags to the total number of statements is calculated to obtain the citation integrity coverage score. The candidate answer text and the reference text with standard citation annotations are input into a natural language reasoning model to calculate the implication relationship. When the implication relationship judgment result is contradictory, the logical consistency score is set to zero. When the implication relationship judgment result is implication or neutral, the logical consistency score is set to one, thus obtaining a normalized logical consistency score. Entities in the candidate answer text are extracted and matched with a preset whitelist of entities in the technical supervision domain and the reference text with standard citation annotations. The number of unmatched entities is counted. The product of the number of unmatched entities and the preset illusion penalty coefficient is subtracted from one to obtain the entity illusion control score, thus obtaining the normalized entity illusion control score. The answer quality comprehensive score is obtained by multiplying the reference integrity coverage score by a first preset quality weight coefficient, the normalized logical consistency score by a second preset quality weight coefficient, and the normalized entity illusion control score by a third preset quality weight coefficient. The comprehensive answer quality score is then determined to be higher than a preset quality threshold. If it is higher, the candidate answer text is taken as the technical supervision standard answer that meets the quality requirements.

Images