Text data set quality evaluation method and device of electric power system and computer equipment

By performing text segmentation and feature analysis on power system text datasets, the homogeneity coverage index and information overlap index of non-isolated samples are selected. This replaces traditional manual or static cleaning methods, improving the automated quality assessment, convenience, and intelligence of text datasets, and ensuring that the assessment results truly reflect the model's generalization ability and safety in real power scenarios.

CN122064994AActive Publication Date: 2026-05-19CHINA SOUTHERN POWER GRID ARTIFICIAL INTELLIGENCE TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202610519854.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-20
Publication Date
2026-05-19
Estimated Expiration
2046-04-20

AI Technical Summary

Technical Problem

Existing methods for constructing text datasets for power systems result in model fit evaluations that fail to accurately reflect generalization capabilities and security in real-world power scenarios, leading to a severe imbalance in the difficulty distribution between the corpus and validation sets.

Method used

By acquiring a text dataset from the power system, the text is segmented and non-isolated samples are distinguished from isolated samples. The sample features of non-isolated samples are extracted for feature homogeneity analysis. The homogeneity coverage index and information overlap index are calculated. The validation set and corpus are selected, and the artificial intelligence model is fine-tuned and validated to obtain validation results that reflect the quality of the text dataset.

Benefits of technology

It has achieved automated quality assessment of text datasets, improving convenience and intelligence. The assessment results can truly reflect the model's generalization ability and security under power business logic and the latest regulations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122064994A_ABST
    Figure CN122064994A_ABST
Patent Text Reader

Abstract

The invention relates to a text data set quality evaluation method and device of a power system and computer equipment. The method comprises the following steps: acquiring a text data set of the power system, performing text segmentation on the text data set to obtain independent samples, and distinguishing non-isolated samples and isolated samples from a plurality of independent samples; extracting a sample feature of each non-isolated sample, and performing feature homogeneous analysis on the sample features to obtain a homogeneous coverage index; for each non-isolated sample, obtaining an information overlapping index based on the sample information of the current non-isolated sample and the sample information of each other independent sample; screening out a verification set based on the homogeneous coverage index and the information overlapping index of each non-isolated sample, and taking the isolated samples and the remaining non-isolated samples as a corpus set; and sequentially performing fine tuning processing and verification on the artificial intelligence model of the power system through the corpus set and the verification set to obtain a quality verification result for reflecting the text data set. By adopting the method, the text data set of the power system can be evaluated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, and computer equipment for quality assessment of text datasets in power systems. Background Technology

[0002] To reduce power grid accidents and large-scale power outages caused by incorrect dispatch instructions or misunderstandings of procedures, the power system needs to strictly manage relevant text datasets.

[0003] Currently, in the training and application of artificial intelligence models for power systems, existing dataset construction methods often employ random partitioning or simple static cleaning. This leads to a severe imbalance in the difficulty distribution of the corpus and validation sets when evaluating the model's adaptability to text datasets, resulting in evaluation results that fail to truly reflect the model's generalization ability and safety in real-world power scenarios. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, and computer equipment for evaluating the quality of text datasets in power systems, which can ensure that the model adaptability evaluation results of text datasets can truly reflect the generalization ability and safety of models in actual power scenarios.

[0005] Firstly, this application provides a method for quality assessment of text datasets in power systems, including:

[0006] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0007] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0008] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0009] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0010] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0011] In one embodiment, based on the semantic vectors of independent samples, non-isolated samples and isolated samples are distinguished among multiple independent samples, including:

[0012] For each independent sample, calculate the distance between the semantic vector of the independent sample and the semantic vectors of other independent samples;

[0013] The number of other independent samples whose distance falls within the preset neighborhood radius is counted, and this number is used as the neighborhood density of the independent samples.

[0014] Determine if the neighborhood density is less than a preset density threshold. If it is, treat the independent sample as a non-isolated sample; otherwise, treat the independent sample as an isolated sample.

[0015] In one embodiment, sample features of each non-isolated sample are extracted, and feature homogeneity analysis is performed on the sample features to obtain the homogeneity coverage index of each non-isolated sample, including:

[0016] For each non-isolated sample, extract the logical structure features, execution scenario features, and timing features of the non-isolated sample, perform inference tests on the non-isolated sample, and obtain the difficulty balance component of the non-isolated sample.

[0017] Based on the characteristics of the logical structure, the logical homogeneous components of non-isolated samples are determined through logical structure similarity analysis.

[0018] Based on the characteristics of the execution scenario, the scenario homogeneous components of non-isolated samples are determined, the cognitive difficulty of non-isolated samples is assessed, and the application homogeneous components of non-isolated samples are determined by combining the cognitive difficulty and the scenario homogeneous components.

[0019] Breakpoint detection is performed on temporal features to obtain temporal compliance components of non-isolated samples;

[0020] The homogeneous coverage index is obtained by weighting and summing the logical homogeneous components, application homogeneous components, difficulty-balanced components, and time-series compliance components.

[0021] In one embodiment, the logical structure features include logical propositions and logical connections; based on the logical structure features, logical homogeneous components of non-isolated samples are determined through logical structure similarity analysis, including:

[0022] Based on the logical propositions and logical connections of non-isolated samples, construct a logical structure graph of non-isolated samples and identify the logical type of the logical structure graph;

[0023] Based on the logic type of non-isolated samples, calculate the proportion of identical logic among non-isolated samples.

[0024] Extract the logical gene sequence from the logical structure map, and calculate the logical structure similarity of non-isolated samples based on the logical gene sequence;

[0025] By combining the proportion of identical logic and the similarity of logical structure, the logical homogeneous components of non-isolated samples are determined.

[0026] In one embodiment, the cognitive difficulty of non-isolated samples is assessed, and the application homogeneity component of the non-isolated samples is determined by combining the cognitive difficulty with the homogeneous component of the scenario, including:

[0027] For each non-isolated sample, the text of the non-isolated sample is parsed to obtain the parsing results. Based on the parsing results, the professional vocabulary density and power business logic complexity of the non-isolated sample are calculated.

[0028] We calculate the cognitive difficulty of non-isolated samples by weighting the logical complexity of power business and the density of professional vocabulary.

[0029] Calculate the average cognitive difficulty for each non-isolated sample;

[0030] For each non-isolated sample, calculate the deviation between the cognitive difficulty of the non-isolated sample and the mean, and use the deviation as the homogeneous component of the difficulty of the non-isolated sample.

[0031] By combining the homogeneous components of difficulty and scenario, the application homogeneous components of non-isolated samples are determined.

[0032] In one embodiment, for each non-isolated sample, an information overlap index for the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample, including:

[0033] For each non-isolated sample, calculate the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and regard other independent samples whose similarity coefficient exceeds the preset similarity coefficient threshold as highly overlapping samples.

[0034] Calculate the ratio of the number of highly overlapping samples to the number of other independent samples, and use this ratio as the proportion of high overlap.

[0035] Calculate the average similarity coefficient of each highly overlapping sample, and use the average value as the average similarity.

[0036] The information overlap index of the current non-isolated samples is obtained by weighting the high overlap ratio and the average similarity.

[0037] Secondly, this application also provides a device for evaluating the quality of text datasets in power systems, comprising:

[0038] The differentiation module is used to acquire the text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0039] The first analysis module is used to extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0040] The second analysis module is used to obtain the information overlap index of the current non-isolated sample based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample for each non-isolated sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0041] The filtering module is used to filter out the validation set of the power system artificial intelligence model from multiple non-isolated samples based on the homogeneity coverage index and information overlap index of each non-isolated sample, and use the remaining non-isolated samples from multiple isolated samples as the corpus of the power system artificial intelligence model.

[0042] The validation module is used to fine-tune the artificial intelligence model of the power system using a corpus, resulting in a fine-tuned artificial intelligence model of the power system. The fine-tuned artificial intelligence model of the power system is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0043] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0044] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0045] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0046] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0047] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0048] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0049] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0050] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0051] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0052] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0053] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0054] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0055] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0056] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0057] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0058] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0059] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0060] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0061] The aforementioned method, apparatus, and computer equipment for quality assessment of text datasets in power systems acquire the text dataset of the power system, segment the text dataset to obtain independent samples, and distinguish non-isolated samples from isolated samples based on the semantic vectors of the independent samples. Sample features of each non-isolated sample are extracted, and feature homogeneity analysis is performed on these features to obtain a homogeneity coverage index for each non-isolated sample. For each non-isolated sample, an information overlap index is obtained based on the information similarity between the current non-isolated sample information and the information of each other independent sample. Other independent samples include the isolated samples among the multiple independent samples and other non-isolated samples besides the current non-isolated sample. Through this process, the homogeneity coverage index and information overlap index of non-isolated samples can be accurately calculated, providing a foundation for subsequent validation set partitioning and ensuring that the subsequent evaluation results accurately reflect the model's generalization ability under power business logic and the latest regulations. Based on the homogeneity coverage index and information overlap index of each non-isolated sample, a validation set for the power system AI model is selected from multiple non-isolated samples. The remaining non-isolated samples from the multiple isolated samples and the remaining non-isolated samples are used as the corpus for the power system AI model. The power system AI model is then fine-tuned using this corpus to obtain a fine-tuned power system AI model. This fine-tuned model is then validated using the validation set to obtain validation results, which reflect the quality of the text dataset. This intelligent process of partitioning the validation set and corpus based on the homogeneity coverage index and information overlap index of each non-isolated sample replaces traditional manual or simple static cleaning methods, significantly improving the convenience and intelligence of automated quality assessment of the text dataset. It also ensures that the final model fit evaluation results of the text dataset truly reflect the model's generalization ability and safety in real-world power scenarios. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a diagram illustrating the application environment of a text dataset quality assessment method for a power system in one embodiment.

[0064] Figure 2 This is a flowchart illustrating a method for evaluating the quality of a text dataset in a power system, as shown in one embodiment.

[0065] Figure 3This is a flowchart illustrating a method for assessing the quality of text datasets in a power system, as described in another embodiment.

[0066] Figure 4 This is a block diagram of a text dataset quality assessment device for a power system in one embodiment;

[0067] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0070] The method for evaluating the quality of text datasets in power systems provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Specifically, terminal 102 or server 104 executes a method for quality assessment of a text dataset of a power system. This method includes: acquiring a text dataset of the power system; segmenting the text dataset to obtain independent samples; distinguishing between non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples; extracting sample features of each non-isolated sample; performing feature homogeneity analysis on the sample features to obtain a homogeneity coverage index for each non-isolated sample; and for each non-isolated sample, obtaining an information overlap index based on the information similarity between the current non-isolated sample information and the sample information of each other independent sample. Other independent samples are the isolated samples among multiple independent samples and other non-isolated samples other than the current non-isolated sample; based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples among multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model; the power system artificial intelligence model is fine-tuned through the corpus to obtain the fine-tuned power system artificial intelligence model, and the fine-tuned power system artificial intelligence model is validated through the validation set to obtain the validation results, which are used to reflect the quality of the text dataset.

[0071] Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0072] In one exemplary embodiment, such as Figure 2 As shown, a method for quality assessment of text datasets in power systems is provided, and this method is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps 202 to 210. Wherein:

[0073] Step 202: Obtain the text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples from multiple independent samples based on the semantic vectors of the independent samples.

[0074] The power system refers to the electricity production and consumption system composed of power generation, transmission, transformation, distribution, and consumption. The text dataset includes heterogeneous data from multiple sources, such as power industry standards and regulations, equipment maintenance records, dispatch operation tickets, fault analysis reports, safety regulations documents, QA questions and answers, and historical operating data. The text dataset reflects the effective support capability for training artificial intelligence models for the power system. Independent samples are the smallest data units with complete business meaning. Semantic vectors are used to digitally represent the deep semantic features of independent samples within the power business context.

[0075] For example, text segmentation algorithms in Natural Language Processing (NLP) technology can be used to cut the text dataset into multiple independent samples. For instance, substation maintenance procedures can be segmented into independent maintenance items according to clauses, or a fault record can be treated as an independent sample.

[0076] For example, a pre-trained power domain language encoding model is used to map independent samples into semantic vectors. This pre-trained power domain language encoding model can be a BERT architecture model or a RoBERTa architecture model that has undergone secondary training based on a massive corpus of power regulations, dispatch instructions, and maintenance reports. Independent samples are input into the pre-trained power domain language encoding model, and the model's pooling layer outputs dense vectors of fixed dimensions (e.g., 768 or 1024 dimensions).

[0077] For example, in the semantic vector space, based on the semantic vectors of multiple independent samples, the neighborhood density of each independent sample is calculated. For each independent sample, if the neighborhood density of the independent sample is greater than or equal to a preset density threshold, the independent sample is regarded as an isolated sample; if the neighborhood density of the independent sample is less than the preset density threshold, the independent sample is regarded as a non-isolated sample.

[0078] Step 204: Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0079] The sample features include the logical structure features, execution scenario features, and temporal features of non-isolated samples. The homogeneity coverage index is a numerical value used to characterize the representativeness of non-isolated samples; that is, a high homogeneity coverage index indicates high representativeness of non-isolated samples.

[0080] For example, based on sample characteristics, logical homogeneous components, application homogeneous components, and time-series compliance components are calculated. Difficulty balance components are obtained by performing difficulty analysis on non-isolated samples. The homogeneous components, application homogeneous components, difficulty balance components, and time-series compliance components are weighted and summed to obtain the homogeneous coverage index.

[0081] Step 206: For each non-isolated sample, based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample, obtain the information overlap index of the current non-isolated sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0082] The sample information includes the semantic feature vectors and key knowledge entities of the current non-isolated samples. The information overlap index is used to describe the degree of uniqueness loss of the current non-isolated samples in the text dataset. The higher the information overlap index, the less new information the non-isolated sample carries.

[0083] For example, semantic feature vectors and key knowledge entities of independent samples are extracted, where independent samples include non-isolated samples and isolated samples. The semantic feature vectors and key knowledge entities are used as sample information. By counting the number of information intersection elements and the number of union elements between the sample information of the current non-isolated sample and the sample information of each other independent sample, the similarity coefficient between the current non-isolated sample and each other independent sample is calculated. The information overlap index of the current non-isolated sample is determined by the similarity coefficient.

[0084] Step 208: Based on the homogeneity coverage index and information overlap index of each non-isolated sample, select the validation set of the power system artificial intelligence model from multiple non-isolated samples, and use the multiple isolated samples and the remaining non-isolated samples in the multiple non-isolated samples as the corpus of the power system artificial intelligence model.

[0085] Among them, the power system artificial intelligence model is a pre-trained model capable of performing tasks for specific business scenarios in the power industry; the corpus is used to fine-tune the power system artificial intelligence model to obtain a fine-tuned power system artificial intelligence model; and the validation set is a subset of data used to evaluate the gain of the corpus on the fine-tuned power system artificial intelligence model.

[0086] For example, a preset verification ratio is obtained, and the product of the number of independent samples and the preset verification ratio is calculated to obtain the number of verification sets. For example, if the number of verification sets is 10, then based on the homogeneity coverage index and information overlap index of each non-isolated sample, 10 non-isolated samples are selected from multiple non-isolated samples as the verification set of the power system artificial intelligence model.

[0087] For example, non-isolated samples with high homogeneity coverage index and low information overlap index are selected from multiple non-isolated samples as the validation set for the artificial intelligence model of the power system.

[0088] Step 210: Fine-tune the power system artificial intelligence model using the corpus to obtain the fine-tuned power system artificial intelligence model. Validate the fine-tuned power system artificial intelligence model using the validation set to obtain the validation results. The validation results are used to reflect the quality of the text dataset.

[0089] The validation results are used to characterize the gains that the corpus brings to the power system AI model. The fine-tuned power system AI model is then used to create the optimized power system AI model.

[0090] For example, "fine-tuning" refers to the process of further training and adjusting all or part of the parameters of a power system AI model that has already been pre-trained on a large-scale general dataset, using a corpus to adapt it to a new task. By inputting the corpus into the power system AI model and performing fine-tuning by calling a deep learning framework, a fine-tuned power system AI model can be obtained.

[0091] For example, a validation set is input into a fine-tuned power system AI model. The model performance (e.g., accuracy, recall, and F1 score) of the fine-tuned power system AI model on the validation set is calculated and compared with the performance of the power system AI model to generate validation results. A weighted average of the indicators in the validation results is then calculated to obtain the model fit score for the text dataset. Specifically, when the validation results show excellent signal-to-noise ratio, accuracy score > 4.5, conciseness reaching full marks (5.0), and extremely low redundancy, the model fit score for the text dataset is excellent, with a score of 90-100. When the validation results show effective domain knowledge injection, accuracy score > 4.0, and basic formatting, the model fit score for the text dataset is good, with a score of 80-89. When the validation results show that the data contains some knowledge but suffers from illusions or formatting redundancy, the model fit score for the text dataset is average, with a score of 70-79.

[0092] For example, the metadata configuration or requirements specification document of the text dataset is read, and the pre-defined scale assessment indicators of each subset of the text dataset are parsed to determine the scale integrity thresholds such as storage capacity and sample size. The data storage directory of the text dataset is traversed to count the physical storage size and number of files. Based on the physical storage size and number of files, the actual scale of the text dataset is calculated. The actual scale is compared item by item with the scale integrity thresholds to calculate the satisfaction rate and gap size of each scale integrity indicator, verifying whether the text dataset meets the predefined scale requirements. Based on the overall achievement rate and weight allocation of the scale integrity indicators, the trained integrity evaluation model is used to calculate and output the scale integrity score of the text dataset.

[0093] For example, based on the existence of target power application business context content in the text dataset, a scenario category dimension system is constructed; multiple statistical diversity indicators are obtained, and the breadth and uniformity of scenario distribution in the text dataset are analyzed based on these indicators to generate objective quantitative results. These objective quantitative results are then compared with the constructed category dimension system to perform scenario distribution analysis, and the content diversity score of the text dataset is determined based on the analysis results. The scenario category dimension system ensures coverage of all key scenario types in real-world applications; scenario distribution analysis is achieved through clustering and statistics using the scenario category dimension system.

[0094] For example, based on the annotation specification documents of typical power system scenarios such as intelligent diagnosis of equipment faults and Q&A on compliance with safety procedures in the text dataset, a rule for judging the correctness of annotation and an error classification framework are established; a representative sample subset of sufficient size and balanced distribution is selected from the text dataset through a stratified random sampling strategy; through an automated verification toolchain, the annotation content of the representative sample subset is compared with the preset standard item by item to calculate the core quality indicators such as annotation accuracy, omission rate and mislabeling rate; based on the core quality indicators, the accuracy score of the text dataset is obtained through linear or nonlinear calculation methods.

[0095] For example, a standard terminology library, business entity list, or domain classification model for a specific industry is loaded. Based on this library, a feature benchmark set for defining industry attributes is generated. The text dataset is traversed, and key entity and scenario information is extracted using text mining or image recognition techniques. The key entity and scenario information are semantically compared and attribute-mapped with the industry benchmarks. Based on the mapping results, the density of industry terms, coverage of specific business scenarios, and confidence of the classification model are calculated. The confidence score is used to quantify the strength of the association between the content and the industry domain. Based on the confidence score, a business relevance score for the text dataset is determined.

[0096] For example, weights are assigned to the scale completeness score, model fit score, content diversity score, content accuracy score, and business relevance score. For instance, weights could be assigned 10%, 40%, 10%, 25%, and 15%, respectively. A weighted sum is then calculated based on these weights to obtain the quality score of the text dataset. Industry relevance refers to the size of the text dataset meeting the requirements for developing and training AI models for the target application scenario. Model fit refers to the text dataset significantly improving the performance of AI models for the power system. Content diversity refers to the accuracy of the annotations in the text dataset reflecting the actual content and characteristics of the data, precisely labeling all the information required for developing and training AI models for the target application scenario. Annotation accuracy refers to the accuracy of the labeled data in the text dataset corresponding to the target. Business relevance indicators include industry conformity, information thematic accuracy, and domain relevance.

[0097] Optionally, based on the validation results, a quality assessment result for the text dataset can be generated through methods such as weighted summation. The quality assessment result includes a usability description text and a quality rating for the text dataset.

[0098] The aforementioned method for quality assessment of text datasets in power systems involves acquiring a text dataset of the power system, segmenting the text dataset to obtain independent samples, and distinguishing between non-isolated and isolated samples based on the semantic vectors of the independent samples. The sample features of each non-isolated sample are extracted, and feature homogeneity analysis is performed to obtain a homogeneity coverage index for each non-isolated sample. For each non-isolated sample, the information overlap index is obtained based on the information similarity between the current non-isolated sample information and the information of each other independent sample. Other independent samples include each independent sample among the multiple independent samples and other non-isolated samples besides the current non-isolated sample. Through this process, the homogeneity coverage index and information overlap index of non-isolated samples can be accurately calculated, providing a foundation for subsequent validation set partitioning and ensuring that the subsequent evaluation results accurately reflect the model's generalization ability under power business logic and the latest regulations. Based on the homogeneity coverage index and information overlap index of each non-isolated sample, a validation set for the power system AI model is selected from multiple non-isolated samples. The remaining non-isolated samples from the multiple isolated samples and the remaining non-isolated samples are used as the corpus for the power system AI model. The power system AI model is then fine-tuned using this corpus to obtain a fine-tuned power system AI model. This fine-tuned model is then validated using the validation set to obtain validation results, which reflect the quality of the text dataset. This intelligent process of partitioning the validation set and corpus based on the homogeneity coverage index and information overlap index of each non-isolated sample replaces traditional manual or simple static cleaning methods, significantly improving the convenience and intelligence of automated quality assessment of the text dataset. It also ensures that the final model fit evaluation results of the text dataset truly reflect the model's generalization ability and safety in real-world power scenarios.

[0099] In an exemplary embodiment, distinguishing between non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of independent samples includes: for each independent sample, calculating the distance between the semantic vector of the independent sample and the semantic vectors of other independent samples; counting the number of other independent samples whose distances fall within a preset neighborhood radius, and using the number as the neighborhood density of the independent sample; determining whether the neighborhood density is less than a preset density threshold, and if so, classifying the independent sample as a non-isolated sample, and if not, classifying the independent sample as an isolated sample.

[0100] The preset density threshold is a distance threshold set based on the distribution characteristics of text data in the power system. Domain density is used to characterize the degree of clustering of independent samples in the semantic vector space. Non-isolated samples are those with high domain density, which can represent mainstream power business scenarios; isolated samples are those with low domain density, which may be anomalous or isolated data.

[0101] For example, in the semantic vector space, the distance between the semantic vector of an independent sample and the semantic vector of other independent samples is calculated, where other independent samples refer to all independent samples other than the independent sample being sought.

[0102] In this embodiment, by utilizing semantic vectors generated from a power-specific language model, the distance between the semantic vectors of independent samples and those of other independent samples is calculated. This accurately captures the unique terminological relationships and logical implications within power texts, overcoming the limitations of traditional keyword matching in understanding deep semantics and ensuring the accuracy of distance calculations. Furthermore, by classifying independent samples into non-isolated and isolated samples based on neighborhood density, isolated noise samples that deviate from mainstream business logic, are unclear in expression, or have unknown origins can be adaptively identified and removed. This effectively prevents these "dirty data" samples from interfering with subsequent model training and evaluation, avoiding the model learning incorrect business rules. Since the above process is fully automated, eliminating the need for manual review, it significantly improves the efficiency and objectivity of dataset cleaning, ensuring that the data entering the subsequent evaluation stage possesses high semantic cohesion and business representativeness, laying a solid foundation for building a high-quality power artificial intelligence dataset.

[0103] In one embodiment, sample features of each non-isolated sample are extracted, and feature homogeneity analysis is performed on the sample features to obtain a homogeneity coverage index for each non-isolated sample. This includes: for each non-isolated sample, extracting the logical structure features, execution scenario features, and temporal features of the non-isolated sample; performing inference tests on the non-isolated sample to obtain the difficulty balance component of the non-isolated sample; determining the logical homogeneous component of the non-isolated sample through logical structure similarity analysis based on the logical structure features; determining the scenario homogeneous component of the non-isolated sample based on the execution scenario features; evaluating the cognitive difficulty of the non-isolated sample; and determining the application homogeneous component of the non-isolated sample by combining the cognitive difficulty and the scenario homogeneous component; performing breakpoint detection on the temporal features to obtain the temporal compliance component of the non-isolated sample; and weighting and summing the logical homogeneous component, application homogeneous component, difficulty balance component, and temporal compliance component to obtain the homogeneity coverage index.

[0104] The difficulty balance component is used to evaluate the cognitive complexity of non-isolated samples. The temporal compliance component is used to remove or downweight samples that have become invalid due to procedure updates. The scenario homogeneity component is used to characterize the degree to which non-isolated samples are close to the overall dataset in terms of operational details and professional knowledge. The logical homogeneity component is used to accurately measure the consistency and representativeness of the current sample with the overall text dataset at the level of power business logic.

[0105] For example, the logical structure features of each non-isolated sample are determined by semantic parsing of its logical structure; the execution scenario features of each non-isolated sample are determined by identifying its execution scenario; and the referenced procedure version and effective date are used as the temporal features of each non-isolated sample through semantic parsing. Semantic parsing refers to using dependency parsing and semantic role labeling techniques in natural language processing to identify conditions (such as "if," "when"), causality (such as "leads to," "cause"), and adversative relationships in the text. The execution scenario refers to the specific environmental context in which the power business described by the sample occurs.

[0106] For example, non-isolated samples are input into a benchmark model for inference testing. The confidence level, inference time, or error rate are calculated based on the output of the benchmark model. The confidence level, inference time, or error rate are weighted and summed, or the average of the confidence level, inference time, or error rate is calculated to obtain the difficulty balance component of the non-isolated samples. The benchmark model is a pre-trained, stable small-scale language model or rule engine in the power field.

[0107] For example, a logical structure map is constructed based on logical structure features, logical structure similarity analysis is performed based on the logical structure map, and the logical homogeneous components of non-isolated samples are determined based on the analysis results.

[0108] Optionally, determine whether the logical gene sequence of a non-isolated sample is completely identical in all execution scenarios. If not, collect the execution scenarios of all independent samples excluding the non-isolated sample and record them as other scenarios. Determine whether other scenarios completely cover the logical gene sequence of the non-isolated sample. If not, collect the execution scenario features of the non-isolated sample's execution scenarios. Collect other execution scenario features of other scenarios and obtain the common features between execution scenario features and other execution scenario features by comparing the execution scenario features with other execution scenario features. Determine whether there is a scenario covering all the common features in all execution scenarios for the non-isolated sample. If not, calculate the cross-scenario feature coverage of the non-isolated sample. Collect the first knowledge domain coverage of the non-isolated sample in all execution scenarios and the second knowledge domain coverage of other scenarios. Calculate the overlap between the first knowledge domain coverage and the second knowledge domain coverage. Combine the overlap and cross-scenario feature coverage to calculate the scenario homogeneous components. Complete coverage refers to whether there exists a combination of scenarios in other scenario sets that can reproduce all the key logical nodes of the non-isolated sample. If there is complete coverage, it means that the logic of all scenarios in the non-isolated sample can be known through training on other independent samples, and the corresponding application homogeneous component is the maximum value. Cross-scenario feature coverage is a comprehensive index obtained by weighting and fusing the number of sets, the proportion of identical features, and the average co-occurrence frequency.

[0109] Specifically, if the logical gene sequence of a non-isolated sample is exactly the same in all execution scenarios, it means that the logic of the non-isolated sample will not change in all scenarios, and the homogeneous component should be applied as the maximum value. For example, the logical action of "disconnecting the switch" in the "normal planned maintenance" scenario requires the preconditions of "receiving the dispatch instruction" and "verifying the dual names of the equipment" and the post-action result is "the equipment is put into cold standby mode"; while in the "line fault emergency trip" scenario, the preconditions of this action are "protection device action signal" and "fault current exceeding the limit", which can be executed without manual instruction, and the post-action result is "isolate the fault point".

[0110] Specifically, if there are instances in the execution scenarios of non-isolated samples that cover all the same features, it means that the business logic of the non-isolated samples has a complete "benchmark" object in the current dataset, and its scenario features are not fragmented but highly aggregated. In this case, the homogeneous component should be applied as the maximum value.

[0111] Specifically, the steps for calculating the cross-scene feature coverage of non-isolated scenarios include: generating sets of identical features between execution scenario features and other execution scenario features, and counting the number of sets of identical features; collecting other scenarios corresponding to the sets of identical features and recording them as set scenarios, and counting the ratio of identical features in the sets of identical features to the features in the set scenarios and recording it as the feature similarity ratio; counting the average co-occurrence frequency of identical features with other identical features in the same scenario, and combining the number of sets and the feature similarity ratio to obtain the cross-scene feature coverage. The average co-occurrence frequency refers to the average probability of any two features in the same set appearing simultaneously in the same other scenario, which can be obtained using a frequent itemset mining algorithm.

[0112] Optionally, the cognitive difficulty of non-isolated samples is assessed, and the homogeneous difficulty component of non-isolated samples is determined based on the cognitive difficulty. The statistical values ​​of the homogeneous difficulty component and the homogeneous scenario component are calculated, such as the mean, variance, or standard deviation. Alternatively, the homogeneous difficulty component and the homogeneous scenario component are weighted and summed to obtain the homogeneous application component of non-isolated samples.

[0113] For example, the temporal characteristics of non-isolated samples are compared with the timeline of procedure changes. If a non-isolated sample introduces an obsolete procedure or has a logical conflict with the latest effective procedure, the time point is determined as a temporal breakpoint. Based on the temporal breakpoint, a penalty coefficient or compliance score is generated, and the penalty coefficient or compliance score is used as the temporal compliance component of the non-isolated sample.

[0114] For example, the entropy weight method is used to determine the weight coefficients of the logical homogeneous component, the application homogeneous component, the difficulty-balanced component, and the timing compliance component. The homogeneous coverage index is obtained by linearly or nonlinearly combining the logical homogeneous component, the application homogeneous component, the difficulty-balanced component, and the timing compliance component. The linear calculation is a weighted summation calculation.

[0115] In this embodiment, by performing reasoning tests on non-isolated samples, a balanced difficulty component for these samples is obtained. This prevents an overconcentration of easy samples or a lack of difficult samples in the text dataset, ensuring a balanced difficulty distribution. The homogeneous coverage index obtained through the above process lays the foundation for subsequently selecting a high-quality validation set. It effectively avoids the evaluation distortion problem caused by high-quality data being misjudged as low-quality data due to questions exceeding the syllabus, ensuring that the validation set is highly consistent with the training data distribution and conforms to the latest standards in four dimensions: logical structure, application scenario, difficulty gradient, and procedural timeliness.

[0116] In one embodiment, the logical structure features include logical propositions and logical connections. Based on the logical structure features, logical homogeneous components of non-isolated samples are determined through logical structure similarity analysis, including: constructing a logical structure map of non-isolated samples based on their logical propositions and logical connections, and identifying the logical types of the logical structure map; calculating the proportion of identical logic in non-isolated samples based on their logical types; extracting logical gene sequences from the logical structure map, and calculating the logical structure similarity of non-isolated samples based on these sequences; and combining the proportion of identical logic and the logical structure similarity to determine the logical homogeneous components of non-isolated samples.

[0117] Logical propositions are formalized expressions of "premise-conclusion" structures, abstracting text from non-isolated samples. A logical structure graph is a graph data structure built with propositions as nodes and logical connections as directed edges, used to represent the inherent deductive paths of power fault handling or operational procedures. Logical types refer to the category labels obtained after topological classification of the logical structure graph, such as "single-cause-single-effect," "multi-cause-parallel," "time-series-chain," or "cyclic feedback." Logical gene sequences are used to represent the logical relationships in power business operations.

[0118] For example, for each non-isolated sample, it is retrieved whether other non-isolated samples have the same logical type. The number of other non-isolated samples with the same logical type as the non-isolated sample is counted, and the ratio of the number of other non-isolated samples to the total number of non-isolated samples is calculated to obtain the proportion of samples with the same logical type. If no other samples with the same logical type exist, the proportion of samples with the same logical type is 0.

[0119] For example, for each non-isolated sample, the logical gene sequence of the non-isolated sample is used as the target sequence, and the logical gene sequences of other non-isolated samples of the same logical type are used as reference sequences. The Levenstein distance algorithm is used to calculate the sequence edit distance between the target sequence and multiple reference sequences, resulting in multiple sequence matching degrees. These multiple sequence matching degrees are then weighted and averaged to obtain the logical structural similarity of the non-isolated sample. Here, the target sequence refers to the logical gene feature string formed after encoding the current non-isolated sample; the reference sequences are the logical gene sequences of other non-isolated samples with the same logical type label as the current non-isolated sample; the sequence edit distance refers to the minimum number of character insertion, deletion, or replacement operations required to convert the target sequence into a reference sequence; and the sequence matching degree is a value obtained based on edit distance normalization, used to quantify the micro-similarity between the non-isolated sample and other non-isolated samples in logical deduction details (such as the order of judgment conditions and the tightness of entity association).

[0120] For example, based on the same logical proportion and logical structure similarity, the logical homogeneous components of non-isolated samples are calculated by weighted summation or geometric average.

[0121] In this embodiment, a logical structure graph of non-isolated samples is constructed based on their logical propositions and logical connections, and the logical types of the logical structure graph are identified. Based on the logical types of non-isolated samples, the proportion of identical logic in the non-isolated samples is calculated. The logical gene sequences of the logical structure graph are extracted, and the logical structure similarity of the non-isolated samples is calculated based on the logical gene sequences. Combining the proportion of identical logic and the logical structure similarity, the logical homogeneous components of the non-isolated samples are determined, which lays the foundation for the determination of the subsequent validation set. Specifically, non-isolated samples with higher logical homogeneous components are more suitable as validation sets, and their inherent logical structure graphs are highly consistent with the logical gene sequences of a large number of mainstream samples in the dataset, representing the most typical and standardized power business processing logic. Including such samples in the validation set allows for the construction of a highly representative and stable logical benchmark. This ensures that the evaluation process of using power system AI models for text datasets focuses on verifying their mastery of core business rules, rather than being interfered with by sparse special operating conditions or noisy logic. Furthermore, since non-isolated samples with higher levels of logical homogeneity exhibit significant group universality in their logical types, using them as a validation benchmark effectively avoids evaluation distortion caused by logical bias in the validation set. This accurately reflects the generalization ability and logical consistency of power system AI models in general power scenarios, thereby precisely selecting high-quality power system AI models that both conform to regulations and have broad applicability, thus ensuring the safety and reliability of intelligent power system applications.

[0122] In one embodiment, assessing the cognitive difficulty of non-isolated samples and determining the application homogeneous component of non-isolated samples by combining the cognitive difficulty and the scenario homogeneous component includes: for each non-isolated sample, parsing the text of the non-isolated sample to obtain the parsing result, calculating the professional vocabulary density and power business logic complexity of the non-isolated sample based on the parsing result; performing a weighted calculation on the power business logic complexity and professional vocabulary density to obtain the cognitive difficulty of the non-isolated sample; calculating the average cognitive difficulty of each non-isolated sample; for each non-isolated sample, calculating the deviation between the cognitive difficulty of the non-isolated sample and the average value, and using the deviation as the difficulty homogeneous component of the non-isolated sample; combining the difficulty homogeneous component and the scenario homogeneous component to determine the application homogeneous component of the non-isolated sample.

[0123] The analysis results include the power operation procedure reference level, the number of fault handling logic branches, and the length of the dispatch instruction dependency chain. The power operation procedure reference level refers to the nesting depth of explicit or implicit references to standard procedures in the sample text. The number of fault handling logic branches is determined by constructing a decision tree model to count the total number of "if-then" judgment nodes and anomaly handling paths in the sample. The dispatch instruction dependency chain length refers to the sequence length of the preceding and subsequent feedback instructions required to complete the business described in the sample. Power business logic complexity is used to quantify the logical complexity of non-isolated samples from a process structure perspective. Vocabulary density refers to the percentage of words within a unit of text length that hit the power industry standard terminology database (including equipment names, operating verbs, or protection settings). Cognitive difficulty characterizes the comprehensive requirements of non-isolated samples on operators' professional knowledge and logical reasoning abilities.

[0124] For example, based on the parsing results, the complexity of the power business logic for non-isolated samples is calculated using dependency parsing and rule matching methods in natural language processing.

[0125] For example, preset weights are obtained for the complexity of power business logic and the density of professional vocabulary. For instance, the complexity of power business logic is 0.6 and the density of professional vocabulary is 0.4. The weighted sum of the complexity of power business logic and the density of professional vocabulary is calculated to obtain the cognitive difficulty of non-isolated samples.

[0126] For example, for each non-isolated sample, the standardized distance between the cognitive difficulty of the non-isolated sample and the mean, such as the Z-score or the absolute value of the relative error, is calculated. This standardized distance is used as the deviation, and the deviation is used as the homogeneous component of the difficulty of the non-isolated sample.

[0127] Optionally, the weights of the difficulty homogeneous component and the scene homogeneous component can be determined by the entropy weight method. Based on the weights, the difficulty homogeneous component and the scene homogeneous component are weighted and summed to obtain the application homogeneous component of the non-isolated sample.

[0128] In this embodiment, by quantifying the cognitive difficulty and calculating the homogeneous component of difficulty, the model evaluation bias caused by uneven distribution of sample difficulty can be effectively eliminated. When the homogeneous component of difficulty is high, it indicates that the difficulty level of non-isolated samples is comparable to the overall level of the corpus. At this time, the features learned by the power system artificial intelligence model using the corpus can fully cover the needs of the validation set, and the validation results can truly reflect the normal business capabilities of the power system artificial intelligence model. Conversely, if the homogeneous component of difficulty is too low, the difficulty of the validation set samples is far greater than that of the corpus. The power system artificial intelligence model will perform poorly because it does not encounter complex logic, and will be misjudged as having low training set quality or poor model generalization ability. In fact, it is caused by the difficulty gradient discontinuity. If the difficulty of the validation set samples is far lower than that of the corpus, it will lead to an inflated evaluation result, masking the defects of the power system artificial intelligence model in handling complex faults. Therefore, by calculating the homogeneous difficulty component through the above process, we can lay the foundation for the subsequent selection of the validation set to ensure that the validation set and the corpus are aligned on the difficulty scale. This allows the quality assessment conclusion of the text dataset to focus on the depth of understanding of business logic by the power system artificial intelligence model rather than simply the difficulty adaptability issue, thereby accurately guiding the optimization direction of the corpus.

[0129] In one embodiment, for each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. This includes: for each non-isolated sample, calculating the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and classifying other independent samples whose similarity coefficients exceed a preset similarity coefficient threshold as highly overlapping samples; calculating the ratio of the number of highly overlapping samples to the number of samples of other independent samples, and using the ratio as the high overlap ratio; calculating the average of the similarity coefficients of each highly overlapping sample, and using the average as the average similarity; and performing a weighted product of the high overlap ratio and the average similarity to obtain the information overlap index of the current non-isolated sample.

[0130] The sample information includes semantic feature vectors and key knowledge entities. Average similarity is used to assess the closeness between highly overlapping samples.

[0131] Optionally, the semantic feature vectors of non-isolated samples can be obtained by mapping the non-isolated sample text into high-dimensional numerical vectors using a pre-trained language model (such as BERT or a power industry-specific model).

[0132] Optionally, electrical proper nouns (such as “220kV busbar”, “differential protection”, and “closing command”) extracted from the samples can be used as key knowledge entities through Named Entity Recognition (NER) technology.

[0133] Optionally, the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample can be calculated. This can be done by calculating the number of common features or attributes shared by the sample information of the current non-isolated sample and the sample information of each other independent sample (i.e., the intersection), and the total number of all features or attributes shared by the sample information of the current non-isolated sample and the sample information of each other independent sample (i.e., the union). The ratio between the absolute value of this intersection and the absolute value of this union is then used as the similarity coefficient.

[0134] For example, the preset similarity coefficient threshold is a similarity threshold set according to the overall distribution characteristics of the text dataset, used to filter out neighboring samples that are highly similar to the information.

[0135] In one embodiment, by calculating the information overlap index, the implicit redundancy risk in the dataset can be accurately identified and warned against, preventing misjudgments of dataset quality due to "false prosperity." A higher information overlap index indicates that the non-isolated sample is highly similar to many other samples in the training set at the semantic and entity levels. During training, the model is prone to overfitting to these high-frequency repetitive patterns, exhibiting extremely high accuracy on the validation set. However, this is not because the model truly grasps the generalized power business logic, but because it has "memorized" the features of the recurring samples. Without this index for correction, the dataset might be mistakenly considered comprehensive and of extremely high quality, masking the fundamental flaw of insufficient data diversity and causing the model to fail when faced with subtle differences unseen in real-world scenarios. Therefore, using the information overlap index to reduce sample weights or remove highly overlapping samples forces the model to focus on more representative differentiated features, ensuring that the evaluated dataset quality truly reflects its generalization potential rather than its memorization ability.

[0136] Next reference Figure 3 The present application will use a specific embodiment to illustrate a method for evaluating the quality of text datasets in a power system.

[0137] Step 1: Obtain the text dataset of the power system and divide the text data into non-isolated samples and isolated samples.

[0138] Specifically, a text dataset of the power system is obtained, and the text dataset is segmented to obtain multiple independent samples. For each independent sample, the distance between the semantic vector of the independent sample and the semantic vector of other independent samples is calculated. The number of other independent samples whose distance falls within the preset neighborhood radius is counted, and the number is used as the neighborhood density of the independent sample. It is determined whether the neighborhood density is less than the preset density threshold. If it is, the independent sample is regarded as a non-isolated sample; otherwise, the independent sample is regarded as an isolated sample.

[0139] Step 2: Calculate the homogeneity coverage index and information overlap index for each non-isolated sample.

[0140] Specifically, for each non-isolated sample, its logical structure features, execution scenario features, and temporal features are extracted. Reasoning tests are then performed on the non-isolated samples to obtain their difficulty balance components. Logical structure features include logical propositions and logical connections. Based on the logical propositions and logical connections of the non-isolated samples, a logical structure graph of the non-isolated samples is constructed, and the logical types of the logical structure graph are identified. Based on the logical types of the non-isolated samples, the proportion of identical logical operations is calculated. Logical gene sequences are extracted from the logical structure graph, and based on these sequences, the logical structure similarity of the non-isolated samples is calculated. Combining the proportion of identical logical operations and the logical structure similarity, the logical homogeneous components of the non-isolated samples are determined. Based on the characteristics of the execution scenario, the scenario homogeneous components of non-isolated samples are determined. For each non-isolated sample, the text of the non-isolated sample is parsed to obtain the parsing results. Based on the parsing results, the professional vocabulary density and power business logic complexity of the non-isolated sample are calculated. The power business logic complexity and professional vocabulary density are weighted to obtain the cognitive difficulty of the non-isolated sample. The average cognitive difficulty of each non-isolated sample is calculated. For each non-isolated sample, the deviation between the cognitive difficulty of the non-isolated sample and the average value is calculated, and the deviation is used as the difficulty homogeneous component of the non-isolated sample. Combining the difficulty homogeneous component and the scenario homogeneous component, the application homogeneous component of the non-isolated sample is determined. Breakpoint detection is performed on the temporal features to obtain the temporal compliance component of the non-isolated sample. The logic homogeneous component, application homogeneous component, difficulty balance component, and temporal compliance component are weighted and summed to obtain the homogeneous coverage index.

[0141] Specifically, for each non-isolated sample, the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample is calculated. Other independent samples whose similarity coefficient exceeds a preset similarity coefficient threshold are regarded as highly overlapping samples. The ratio of the number of highly overlapping samples to the number of other independent samples is calculated, and the ratio is used as the high overlap ratio. The average similarity coefficient of each highly overlapping sample is calculated, and the average is used as the average similarity. The high overlap ratio and the average similarity are weighted and multiplied to obtain the information overlap index of the current non-isolated sample.

[0142] Step 3: Based on the homogeneity coverage index and information overlap index of each non-isolated sample, determine the validation set in the non-isolated samples, and use the remaining non-isolated samples and isolated samples as the corpus.

[0143] To obtain the validation set size, calculate the product of the number of independent samples and the validation set size. For example, if the validation set size is 10, then based on the homogeneity coverage index and information overlap index of each non-isolated sample, select 10 non-isolated samples from multiple non-isolated samples as the validation set for the power system artificial intelligence model. Alternatively, select non-isolated samples with high homogeneity coverage index and low information overlap index from multiple non-isolated samples as the validation set for the power system artificial intelligence model.

[0144] Step 4: Fine-tune the artificial intelligence model of the power system using the corpus to obtain the fine-tuned artificial intelligence model of the power system. Then, validate the fine-tuned artificial intelligence model of the power system using the validation set to obtain validation results that can reflect the quality of the power system text dataset.

[0145] Specifically, based on the validation results, an evaluation is conducted to obtain a model fit score for the power system text dataset. The scale completeness score, content diversity score, content accuracy score, and business relevance score of the text dataset are calculated separately. Weights are assigned to each of these scores, and a weighted sum is calculated based on these weights to obtain the quality score of the text dataset.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0147] Based on the same inventive concept, this application also provides a power system text dataset quality assessment device for implementing the above-described power system text dataset quality assessment method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more power system text dataset quality assessment device embodiments provided below can be found in the limitations of the power system text dataset quality assessment method described above, and will not be repeated here.

[0148] In one exemplary embodiment, such as Figure 4 As shown, a text dataset quality assessment device 400 for a power system is provided, comprising: a differentiation module 402, a first analysis module 404, a second analysis module 406, a filtering module 408, and a verification module 410, wherein:

[0149] The differentiation module 402 is used to acquire the text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vector of the independent samples.

[0150] The first analysis module 404 is used to extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0151] The second analysis module 406 is used to obtain the information overlap index of the current non-isolated sample based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample for each non-isolated sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0152] The filtering module 408 is used to filter out the validation set of the power system artificial intelligence model from multiple non-isolated samples based on the homogeneity coverage index and information overlap index of each non-isolated sample, and use the multiple isolated samples and the remaining non-isolated samples in the multiple non-isolated samples as the corpus of the power system artificial intelligence model.

[0153] The verification module 410 is used to fine-tune the artificial intelligence model of the power system using the corpus to obtain the fine-tuned artificial intelligence model of the power system, and to verify the fine-tuned artificial intelligence model of the power system using the verification set to obtain the verification result. The verification result is used to reflect the quality of the text dataset.

[0154] In one embodiment, the differentiation module 402 is further configured to, for each independent sample, calculate the distance between the semantic vector of the independent sample and the semantic vector of other independent samples; count the number of other independent samples whose distance falls within a preset neighborhood radius, and use the number as the neighborhood density of the independent sample; determine whether the neighborhood density is less than a preset density threshold, and if so, treat the independent sample as a non-isolated sample, otherwise treat the independent sample as an isolated sample.

[0155] In one embodiment, the first analysis module 404 is further configured to, for each non-isolated sample, extract the logical structure features, execution scenario features, and temporal features of the non-isolated sample; perform inference tests on the non-isolated sample to obtain the difficulty balance component of the non-isolated sample; determine the logical homogeneous component of the non-isolated sample through logical structure similarity analysis based on the logical structure features; determine the scenario homogeneous component of the non-isolated sample based on the execution scenario features; evaluate the cognitive difficulty of the non-isolated sample; combine the cognitive difficulty and the scenario homogeneous component to determine the application homogeneous component of the non-isolated sample; perform breakpoint detection on the temporal features to obtain the temporal compliance component of the non-isolated sample; and perform a weighted summation of the logical homogeneous component, application homogeneous component, difficulty balance component, and temporal compliance component to obtain the homogeneity coverage index.

[0156] In one embodiment, the first analysis module 404 is further configured to construct a logical structure map of non-isolated samples based on the logical propositions and logical connections of the non-isolated samples, identify the logical types of the logical structure map; calculate the proportion of identical logic in the non-isolated samples based on the logical types of the non-isolated samples; extract the logical gene sequence of the logical structure map, calculate the logical structure similarity of the non-isolated samples based on the logical gene sequence; and determine the logical homogeneous components of the non-isolated samples by combining the proportion of identical logic and the logical structure similarity.

[0157] In one embodiment, the first analysis module 404 is further configured to, for each non-isolated sample, parse the text of the non-isolated sample to obtain the parsing result, calculate the professional vocabulary density and power business logic complexity of the non-isolated sample based on the parsing result; perform weighted calculation on the power business logic complexity and professional vocabulary density to obtain the cognitive difficulty of the non-isolated sample; calculate the average cognitive difficulty of each non-isolated sample; for each non-isolated sample, calculate the deviation between the cognitive difficulty of the non-isolated sample and the average value, and use the deviation as the homogeneous component of the difficulty of the non-isolated sample; combine the homogeneous component of difficulty and the homogeneous component of scenario to determine the homogeneous component of application of the non-isolated sample.

[0158] In one embodiment, the second analysis module 406 is further configured to, for each non-isolated sample, calculate the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and designate other independent samples whose similarity coefficient exceeds a preset similarity coefficient threshold as highly overlapping samples; calculate the ratio of the number of highly overlapping samples to the number of other independent samples, and use the ratio as the high overlap ratio; calculate the average of the similarity coefficients of each highly overlapping sample, and use the average as the average similarity; and perform a weighted product of the high overlap ratio and the average similarity to obtain the information overlap index of the current non-isolated sample.

[0159] Each module in the aforementioned power system text dataset quality assessment device 400 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0160] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores text data related to the power system. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for quality assessment of text datasets in a power system.

[0161] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0162] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0163] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0164] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0165] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0166] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0167] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0168] In one embodiment, when the processor executes the computer program, it further implements the following steps: for each independent sample, calculate the distance between the semantic vector of the independent sample and the semantic vector of other independent samples; count the number of other independent samples whose distance falls within a preset neighborhood radius, and use the number as the neighborhood density of the independent sample; determine whether the neighborhood density is less than a preset density threshold, and if so, treat the independent sample as a non-isolated sample, otherwise treat the independent sample as an isolated sample.

[0169] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each non-isolated sample, extracting the logical structure features, execution scenario features, and timing features of the non-isolated sample; performing inference tests on the non-isolated sample to obtain the difficulty balance component of the non-isolated sample; determining the logical homogeneous component of the non-isolated sample through logical structure similarity analysis based on the logical structure features; determining the scenario homogeneous component of the non-isolated sample based on the execution scenario features; evaluating the cognitive difficulty of the non-isolated sample; and determining the application homogeneous component of the non-isolated sample by combining the cognitive difficulty and the scenario homogeneous component; performing breakpoint detection on the timing features to obtain the timing compliance component of the non-isolated sample; and weighting and summing the logical homogeneous component, application homogeneous component, difficulty balance component, and timing compliance component to obtain the homogeneity coverage index.

[0170] In one embodiment, when the processor executes the computer program, it further performs the following steps: constructing a logical structure graph of non-isolated samples based on the logical propositions and logical connections of the non-isolated samples, and identifying the logical types of the logical structure graph; calculating the proportion of identical logic in the non-isolated samples based on the logical types of the non-isolated samples; extracting the logical gene sequence of the logical structure graph, and calculating the logical structure similarity of the non-isolated samples based on the logical gene sequence; and determining the logical homogeneous components of the non-isolated samples by combining the proportion of identical logic and the logical structure similarity.

[0171] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each non-isolated sample, parses the text of the non-isolated sample to obtain the parsing result, and calculates the professional vocabulary density and power business logic complexity of the non-isolated sample based on the parsing result; performs a weighted calculation on the power business logic complexity and professional vocabulary density to obtain the cognitive difficulty of the non-isolated sample; calculates the average cognitive difficulty of each non-isolated sample; for each non-isolated sample, calculates the deviation between the cognitive difficulty of the non-isolated sample and the average value, and uses the deviation as the homogeneous component of the difficulty of the non-isolated sample; combines the homogeneous component of difficulty and the homogeneous component of scenario to determine the homogeneous component of application of the non-isolated sample.

[0172] In one embodiment, when the processor executes the computer program, it further implements the following steps: for each non-isolated sample, calculates the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and identifies other independent samples whose similarity coefficients exceed a preset similarity coefficient threshold as highly overlapping samples; calculates the ratio of the number of highly overlapping samples to the number of other independent samples, and uses the ratio as the high overlap ratio; calculates the average similarity coefficient of each highly overlapping sample, and uses the average as the average similarity; and performs a weighted product of the high overlap ratio and the average similarity to obtain the information overlap index of the current non-isolated sample.

[0173] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0174] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0175] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0176] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0177] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0178] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0179] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: for each independent sample, calculate the distance between the semantic vector of the independent sample and the semantic vector of other independent samples; count the number of other independent samples whose distance falls within a preset neighborhood radius, and use the number as the neighborhood density of the independent sample; determine whether the neighborhood density is less than a preset density threshold, and if so, treat the independent sample as a non-isolated sample, otherwise treat the independent sample as an isolated sample.

[0180] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each non-isolated sample, extracting the logical structure features, execution scenario features, and temporal features of the non-isolated sample; performing inference tests on the non-isolated sample to obtain the difficulty balance component of the non-isolated sample; determining the logical homogeneous component of the non-isolated sample through logical structure similarity analysis based on the logical structure features; determining the scenario homogeneous component of the non-isolated sample based on the execution scenario features; evaluating the cognitive difficulty of the non-isolated sample; and determining the application homogeneous component of the non-isolated sample by combining the cognitive difficulty and the scenario homogeneous component; performing breakpoint detection on the temporal features to obtain the temporal compliance component of the non-isolated sample; and weighting and summing the logical homogeneous component, application homogeneous component, difficulty balance component, and temporal compliance component to obtain the homogeneity coverage index.

[0181] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing a logical structure graph of non-isolated samples based on the logical propositions and logical connections of the non-isolated samples, and identifying the logical types of the logical structure graph; calculating the proportion of identical logic in the non-isolated samples based on the logical types of the non-isolated samples; extracting the logical gene sequence of the logical structure graph, and calculating the logical structure similarity of the non-isolated samples based on the logical gene sequence; and determining the logical homogeneous components of the non-isolated samples by combining the proportion of identical logic and the logical structure similarity.

[0182] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each non-isolated sample, parses the text of the non-isolated sample to obtain the parsing result, and calculates the professional vocabulary density and power business logic complexity of the non-isolated sample based on the parsing result; performs a weighted calculation on the power business logic complexity and professional vocabulary density to obtain the cognitive difficulty of the non-isolated sample; calculates the average cognitive difficulty of each non-isolated sample; for each non-isolated sample, calculates the deviation between the cognitive difficulty of the non-isolated sample and the average value, and uses the deviation as the homogeneous component of the difficulty of the non-isolated sample; combines the homogeneous component of difficulty and the homogeneous component of scenario to determine the homogeneous component of application of the non-isolated sample.

[0183] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each non-isolated sample, calculate the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and designate other independent samples whose similarity coefficient exceeds a preset similarity coefficient threshold as highly overlapping samples; calculate the ratio of the number of samples of highly overlapping samples to the number of samples of other independent samples, and use the ratio as the high overlap ratio; calculate the average similarity coefficient of each highly overlapping sample, and use the average as the average similarity; and perform a weighted product of the high overlap ratio and the average similarity to obtain the information overlap index of the current non-isolated sample.

[0184] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0185] Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples.

[0186] Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample.

[0187] For each non-isolated sample, the information overlap index of the current non-isolated sample is obtained based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample.

[0188] Based on the homogeneity coverage index and information overlap index of each non-isolated sample, the validation set of the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples from multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model.

[0189] The power system artificial intelligence model is fine-tuned using a corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using a validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

[0190] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: for each independent sample, calculate the distance between the semantic vector of the independent sample and the semantic vector of other independent samples; count the number of other independent samples whose distance falls within a preset neighborhood radius, and use the number as the neighborhood density of the independent sample; determine whether the neighborhood density is less than a preset density threshold, and if so, treat the independent sample as a non-isolated sample, otherwise treat the independent sample as an isolated sample.

[0191] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each non-isolated sample, extracting the logical structure features, execution scenario features, and temporal features of the non-isolated sample; performing inference tests on the non-isolated sample to obtain the difficulty balance component of the non-isolated sample; determining the logical homogeneous component of the non-isolated sample through logical structure similarity analysis based on the logical structure features; determining the scenario homogeneous component of the non-isolated sample based on the execution scenario features; evaluating the cognitive difficulty of the non-isolated sample; and determining the application homogeneous component of the non-isolated sample by combining the cognitive difficulty and the scenario homogeneous component; performing breakpoint detection on the temporal features to obtain the temporal compliance component of the non-isolated sample; and weighting and summing the logical homogeneous component, application homogeneous component, difficulty balance component, and temporal compliance component to obtain the homogeneity coverage index.

[0192] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing a logical structure graph of non-isolated samples based on the logical propositions and logical connections of the non-isolated samples, and identifying the logical types of the logical structure graph; calculating the proportion of identical logic in the non-isolated samples based on the logical types of the non-isolated samples; extracting the logical gene sequence of the logical structure graph, and calculating the logical structure similarity of the non-isolated samples based on the logical gene sequence; and determining the logical homogeneous components of the non-isolated samples by combining the proportion of identical logic and the logical structure similarity.

[0193] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each non-isolated sample, parses the text of the non-isolated sample to obtain the parsing result, and calculates the professional vocabulary density and power business logic complexity of the non-isolated sample based on the parsing result; performs a weighted calculation on the power business logic complexity and professional vocabulary density to obtain the cognitive difficulty of the non-isolated sample; calculates the average cognitive difficulty of each non-isolated sample; for each non-isolated sample, calculates the deviation between the cognitive difficulty of the non-isolated sample and the average value, and uses the deviation as the homogeneous component of the difficulty of the non-isolated sample; combines the homogeneous component of difficulty and the homogeneous component of scenario to determine the homogeneous component of application of the non-isolated sample.

[0194] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each non-isolated sample, calculate the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and designate other independent samples whose similarity coefficient exceeds a preset similarity coefficient threshold as highly overlapping samples; calculate the ratio of the number of samples of highly overlapping samples to the number of samples of other independent samples, and use the ratio as the high overlap ratio; calculate the average similarity coefficient of each highly overlapping sample, and use the average as the average similarity; and perform a weighted product of the high overlap ratio and the average similarity to obtain the information overlap index of the current non-isolated sample.

[0195] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0196] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0197] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0198] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for quality assessment of text datasets in power systems, characterized in that, The method includes: Obtain a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vectors of the independent samples. Extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample; For each non-isolated sample, based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample, the information overlap index of the current non-isolated sample is obtained, wherein the other independent samples are each isolated sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample. Based on the homogeneity coverage index and information overlap index of each non-isolated sample, a validation set for the power system artificial intelligence model is selected from multiple non-isolated samples, and the remaining non-isolated samples among the multiple isolated samples and multiple non-isolated samples are used as the corpus of the power system artificial intelligence model. The power system artificial intelligence model is fine-tuned using the corpus to obtain a fine-tuned power system artificial intelligence model. The fine-tuned power system artificial intelligence model is then validated using the validation set to obtain validation results. The validation results are used to reflect the quality of the text dataset.

2. The method according to claim 1, characterized in that, The step of distinguishing non-isolated samples and isolated samples among the plurality of independent samples based on the semantic vectors of the independent samples includes: For each independent sample, calculate the distance between the semantic vector of the independent sample and the semantic vectors of other independent samples; The number of other independent samples whose distance falls within the preset radius is counted, and this number is taken as the neighborhood density of the independent samples; Determine whether the density of the domain is less than a preset density threshold. If it is, the independent sample is treated as a non-isolated sample; otherwise, the independent sample is treated as an isolated sample.

3. The method according to claim 1, characterized in that, The process of extracting sample features from each non-isolated sample and performing feature homogeneity analysis on these features to obtain a homogeneity coverage index for each non-isolated sample includes: For each non-isolated sample, extract the logical structure features, execution scenario features, and timing features of the non-isolated sample, perform inference tests on the non-isolated sample, and obtain the difficulty balance component of the non-isolated sample. Based on the aforementioned logical structure characteristics, logical homogeneous components of the non-isolated samples are determined through logical structure similarity analysis. Based on the execution scenario characteristics, the scenario homogeneous components of the non-isolated samples are determined, the cognitive difficulty of the non-isolated samples is evaluated, and the application homogeneous components of the non-isolated samples are determined by combining the cognitive difficulty and the scenario homogeneous components. Breakpoint detection is performed on the temporal features to obtain the temporal compliance components of the non-isolated samples; The homogeneous coverage index is obtained by weighting and summing the logical homogeneous component, the application homogeneous component, the difficulty balance component, and the timing compliance component.

4. The method according to claim 3, characterized in that, The logical structure features include logical propositions and logical connections; the step of determining the logically homogeneous components of the non-isolated samples through logical structure similarity analysis based on the logical structure features includes: Based on the logical propositions and logical connections of the non-isolated samples, construct a logical structure graph of the non-isolated samples and identify the logical type of the logical structure graph; Based on the logic type of the non-isolated samples, calculate the proportion of identical logic among the non-isolated samples; Extract the logical gene sequence of the logical structure map, and calculate the logical structure similarity of the non-isolated samples based on the logical gene sequence; By combining the proportion of identical logic and the similarity of logical structures, the logical homogeneous components of the non-isolated samples are determined.

5. The method according to claim 3, characterized in that, The assessment of the cognitive difficulty of the non-isolated samples, combined with the cognitive difficulty and the scene homogeneity components, determines the application homogeneity components of the non-isolated samples, including: For each non-isolated sample, the text of the non-isolated sample is parsed to obtain the parsing result. Based on the parsing result, the professional vocabulary density and power business logic complexity of the non-isolated sample are calculated. The cognitive difficulty of the non-isolated sample is obtained by weighting the complexity of the power business logic and the density of the professional vocabulary. Calculate the average cognitive difficulty for each non-isolated sample; For each non-isolated sample, the deviation between the cognitive difficulty of the non-isolated sample and the average value is calculated, and the deviation is used as the homogeneous component of the difficulty of the non-isolated sample. By combining the homogeneous components of difficulty and scenario, the application homogeneous component of the non-isolated sample is determined.

6. The method according to claim 1, characterized in that, For each non-isolated sample, based on the information similarity between the current non-isolated sample information and the sample information of each other independent sample, the information overlap index of the current non-isolated sample is obtained, including: For each non-isolated sample, calculate the similarity coefficient between the sample information of the current non-isolated sample and the sample information of each other independent sample, and regard other independent samples whose similarity coefficient exceeds a preset similarity coefficient threshold as highly overlapping samples; Calculate the ratio of the number of highly overlapping samples to the number of other independent samples, and use the ratio as the high overlap percentage; Calculate the average similarity coefficient of each highly overlapping sample, and use the average value as the average similarity. The information overlap index of the current non-isolated samples is obtained by weighting the high overlap ratio and the average similarity.

7. A device for quality assessment of text datasets in a power system, characterized in that, The device includes: The differentiation module is used to acquire a text dataset of the power system, perform text segmentation on the text dataset to obtain independent samples, and distinguish non-isolated samples and isolated samples among multiple independent samples based on the semantic vector of the independent samples. The first analysis module is used to extract the sample features of each non-isolated sample, perform feature homogeneity analysis on the sample features, and obtain the homogeneity coverage index of each non-isolated sample. The second analysis module is used to obtain the information overlap index of the current non-isolated sample for each non-isolated sample based on the information similarity between the sample information of the current non-isolated sample and the sample information of each other independent sample. The other independent samples are each independent sample among multiple independent samples and other non-isolated samples other than the current non-isolated sample. The filtering module is used to filter out the validation set of the power system artificial intelligence model from multiple non-isolated samples based on the homogeneity coverage index and information overlap index of each non-isolated sample, and use the multiple isolated samples and the remaining non-isolated samples in the multiple non-isolated samples as the corpus of the power system artificial intelligence model. The verification module is used to fine-tune the power system artificial intelligence model using the corpus to obtain a fine-tuned power system artificial intelligence model, and to verify the fine-tuned power system artificial intelligence model using the verification set to obtain a verification result. The verification result is used to reflect the quality of the text dataset.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.