Chinese power text error correction method and system based on SLM-ELCTRA model

Abstract

The invention discloses a Chinese power text error correction method and system based on an SLM-ELCTRA model. The method comprises the following steps: collecting various types of power text data; establishing a statistical language model based on the space lag model; according to the power text data, outputting the probability that each word string in the power text data accords with the lexical method; when the probability that the word string accords with the lexical method is smaller than a set threshold value, identifying error positions in the power text data based on different granularities, and generating a candidate word set; establishing an error correction model based on a pre-training language model, and inputting the error position, the candidate word set and the original power text into the pre-training language model; and inputting the to-be-corrected power text into the error correction model, correcting by taking the candidate word with the highest context matching degree and the maximum probability of conforming to the lexical method as the optimal replacement word, and outputting the corrected power text. The method is suitable for various practical application scenes such as technical report writing, standard document compiling and operation manual making in the power industry, and the accuracy and efficiency of text processing in the power industry are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and more specifically, relates to a Chinese electric power text error correction method and system based on the SLM-ELECTRA model. Background Art

[0002] At present, text input errors are mainly caused by voice or similar characters, of which nearly 83% are related to pinyin similarity (phonetic similarity errors), 48% are related to glyph similarity (phonetic similarity errors), and 31% are both. However, the accuracy of input data is the basic premise of common tasks in natural language processing, and it is also the key to improving the performance of upper-level applications such as keyword extraction and text automatic summarization. In order to solve the problem that the explosive growth of text data is often accompanied by a rapid decline in text quality, the research on Chinese text error correction based on natural language processing technology has emerged, which can effectively improve the quality of Chinese input text. In the power industry, text information processing is a vital task. Whether it is daily official document release, news writing, or editorial response to text related to the power system, a large amount of text information needs to be accurately processed. With the continuous advancement of technology and the expansion of its application scope, text error correction technology will bring more innovation and development opportunities to the power industry, which is of great significance to promoting the sustainable development of the power industry.

[0003] Traditional research on grammatical error detection has mainly focused on English. Compared with English, Chinese does not have significant word boundaries, nor does it have identification elements such as tense, singular and plural. It has high grammatical complexity and contains rich semantic information. At present, many researchers refer to the English grammatical error correction method and use generative methods to directly correct errors, skipping the error detection link. Only a small number of studies use sequence labeling methods for Chinese grammatical error detection. However, generative methods often start from global semantics and ignore the help of Chinese text word string information for grammatical error detection. There are also problems such as large amount of data required, difficulty in training and poor reliability, which make it not well suitable for Chinese grammatical error detection tasks. Therefore, how to make full use of text word string information to improve the effect of Chinese grammatical error detection under limited data is a difficulty in this study. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention provides a Chinese electric power text error correction method based on the SLM-ELECTRA model.

[0005] The present invention adopts the following technical solution.

[0006] The first aspect of the present invention provides a Chinese electric power text error correction method based on the SLM-ELECTRA model, comprising the following steps:

[0007] Collect various types of power texts, including: operation data, fault reports, equipment maintenance records and customer feedback; split power texts into word strings of different lengths;

[0008] A statistical language model is established based on the spatial lag model architecture; the statistical language model outputs the probability that word strings of different lengths in the power text conform to the lexical structure; the word strings whose probability of conforming to the lexical structure is less than the set probability threshold are marked as error positions, and candidate words are generated for the error positions to update the power text; the statistical language model outputs the probability that each candidate word in the updated power text conforms to the lexical structure, and each candidate word is arranged in descending order according to the probability of conforming to the lexical structure to obtain a candidate word set;

[0009] An error correction model is established based on the pre-trained language model architecture. The error position and candidate word set are input into the error correction model. The error correction model generates candidate word features according to the context of the error position, and determines the matching degree between each candidate word in the candidate word set and the candidate word features. The candidate word with the highest probability of conforming to the lexical structure and the highest matching degree with the candidate word features is used as the replacement word for the error position to correct errors in the power text.

[0010] Preferably, the collection of various types of power texts includes:

[0011] Utilize application programming interfaces and sensor data interfaces to collect power text from multiple sources, including equipment monitoring systems, operation reports, and power load data;

[0012] Add error labels, power equipment category labels and data source labels to each type of power text, where error labels include terminology errors, spelling errors and grammatical errors;

[0013] The power text is organized into structured power text data according to error labels, power equipment category labels and data source labels.

[0014] Preferably, the statistical language model outputs the probability that word strings of different lengths in the power text conform to the lexical structure, including:

[0015] Based on the N-gram model, the structured power text data is segmented at different granularities to generate word strings of different lengths; the granularities include single-word, double-word, and triple-word.

[0016] Preferably, the statistical language model outputs the probability of each candidate word in the updated power text being lexical, and arranges each candidate word in descending order of the probability of being lexical to obtain a candidate word set; including:

[0017] Using the statistical language model, the probability of word strings of different lengths in the power text being lexical is output, and a probability threshold is set at the same time, and word strings with a probability of being lexical less than the set probability threshold are marked as error positions;

[0018] Generate candidate words for the word string in the wrong position according to the following rules:

[0019] 1) Obtain the pinyin of the wrong position and generate candidate words with similar pinyin according to the pinyin similarity principle;

[0020] 2) Obtain the glyphs at the wrong position and generate candidate words with similar glyphs according to the glyph similarity principle;

[0021] 3) Obtain the context of the error location and generate semantically related candidate words according to the semantic relevance principle.

[0022] Preferably, the probability of a word string being lexical is calculated as follows:

[0023]

[0024] In the formula, P(X piece ) represents the probability of matching the lexical structure, X piece is a word string, P(ε b ) represents the probability of the electric power text string, P(ε b+1 |ε b ) indicates that ε is known b In the case of b+1 Conditional probability; P(ε b+2 |ε b ε b+1 ) indicates that ε is known b and ε b+1 In the case of b+2 Conditional probability;

[0025] It means that given the b to ε e-1 When all elements of e Conditional probability of occurrence;

[0026] ε b Indicates the starting element of the power text string; ε e Indicates the end element of the power text string; ε b+1 ,ε b+2 ,…,ε e-1 Represents the element in the middle of an electric text fragment.

[0027] Preferably, the error correction model is established based on the pre-trained language model architecture, the error position and the candidate word set are input into the error correction model, the error correction model generates candidate word features according to the context of the error position, and determines the matching degree between each candidate word in the candidate word set and the candidate word features; the candidate word with the highest probability of conforming to the lexical structure and the highest matching degree with the candidate word features is used as the replacement word for the error position, and the power text error correction is performed, including:

[0028] The error position, the candidate word set and the context of the error position are input into the error correction model, and the error correction model generates the context feature of the error position and the candidate word feature through the context feature generator;

[0029] The error correction model uses the generator part of the pre-trained language model and combines the context features and candidate word features into the generator to generate multiple candidate replacement words. The discriminator part of the pre-trained language model is used to calculate the matching degree between each candidate replacement word and the error position context according to the context features and candidate word features.

[0030] The probability of lexical matching calculated by the statistical language model and the matching degree calculated by the pre-trained language model are weighted and fused according to the set fusion weight coefficient to obtain the comprehensive matching degree of the candidate replacement word;

[0031] The candidate replacement words are sorted according to the comprehensive matching degree, and the candidate replacement word with the highest comprehensive matching degree is selected as the replacement word for the error position.

[0032] Preferably, the error correction model generates context features and candidate word features of the error position through a context feature generator, including:

[0033] Mark the error location with a special marker [MASK];

[0034] The Transformer encoder of the pre-trained language model is used to encode the contextual information of the error position and generate contextual features, including the word string and part of speech in the context;

[0035] Based on the context features, the features of the candidate words are extracted, including phonetic, glyphic and semantic features.

[0036] Preferably, the comprehensive matching degree of the candidate replacement words is calculated as follows:

[0037] P 综合 =α·P(X piece )+(1-α)·P match

[0038] Where P 综合 represents the comprehensive matching degree of candidate replacement words; P(X piece ) represents the probability of conforming to the lexical structure; Pmatch It represents the matching degree between the candidate replacement word and the context of the error position; α represents the set fusion weight coefficient.

[0039] The second aspect of the present invention provides a Chinese electric power text error correction system based on the SLM-ELECTRA model, comprising: a data acquisition and processing module, a statistical language model module and a candidate word screening and correction module; characterized in that:

[0040] The data collection and processing module is used to collect various types of power text data, including operation data, fault reports, equipment maintenance records and customer feedback, and to segment the power text into word strings of different lengths;

[0041] The statistical language model module is used to establish a statistical language model, output the probability that word strings of different lengths in the power text conform to the lexical structure, mark the error positions, and generate candidate words to update the power text;

[0042] The candidate word screening and correction module is used to build an error correction model, input the error position and candidate word set into the error correction model, use the context of the error position to generate candidate word features, calculate the matching degree between each candidate word and the candidate word features, and select the optimal replacement word to correct text errors.

[0043] Preferably, the statistical language model module includes:

[0044] Using the statistical language model, the probability of word strings of different lengths in the power text being lexical is output, and a probability threshold is set at the same time, and word strings with a probability of being lexical less than the set probability threshold are marked as error positions;

[0045] When generating candidate words for a word string at an incorrect position, the pinyin of the incorrect position is obtained and candidate words with similar pinyins are generated according to the pinyin similarity principle; candidate words with similar glyphs are generated according to the glyph similarity principle; and semantically related candidate words are generated according to the context of the incorrect position and the semantic relevance principle.

[0046] A third aspect of the present invention provides a terminal, including a processor and a storage medium;

[0047] The storage medium is used to store instructions;

[0048] The processor is used to operate according to the instructions to execute the steps of the Chinese electric power text error correction method based on the SLM ELECTRA model.

[0049] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of a Chinese electric power text error correction method based on the SLM ELECTRA model are performed.

[0050] Compared with the prior art, the beneficial effects of the present invention include at least:

[0051] The present invention combines the SLM statistical language model and the ELECTRA pre-trained language model to achieve deep fusion between the probability of conforming to the lexical structure and the contextual semantics, ensuring that the candidate words are not only grammatically legal, but also consistent with the contextual semantics, thereby improving the accuracy and professionalism of error correction. Secondly, by dynamically adjusting the fusion weights of SLM and ELECTRA, the present invention can adaptively select the best model according to the complexity of the context, thereby showing higher flexibility and adaptability in different types of texts, overcoming the limitations of fixed model weights in the prior art. In addition, the present invention adopts a multi-granularity N-gram model, which improves the recognition ability of diversified errors in power texts through granularity adaptive adjustment, especially when dealing with complex syntax, the effect is more accurate. Finally, based on the fine-tuning mechanism, by combining supervised learning to optimize the joint training of the generator and the discriminator, the present invention significantly improves the accuracy and stability of the error correction model, especially when dealing with complex contexts, with better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is an overall logic flow chart provided according to an embodiment of the present invention;

[0053] Figure 2 It is a schematic diagram of calculating the probability of a bidirectional multi-granularity N-gram word string conforming to the lexical structure provided in accordance with an embodiment of the present invention;

[0054] Figure 3 is a flowchart of error detection and correction of the ELECTRA pre-trained language model provided in accordance with an embodiment of the present invention;

[0055] Figure 4 It is a diagram of experimental results provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0057] like Figure 1 As shown, Example 1 of the present invention provides a Chinese electric power text error correction method based on the SLM-ELECTRA model, comprising the following steps:

[0058] Step 1: Collect various types of power texts, including: operation data, fault reports, equipment maintenance records and customer feedback; divide the power texts into word strings of different lengths;

[0059] Preferably, step 1 comprises:

[0060] Collect power text data from multiple sources including equipment monitoring systems, operation reports, and power load data, using API interfaces and sensor data interfaces;

[0061] Specifically, data collection and preprocessing: collect a large amount of text data from the power company's internal documents, professional websites, forums, equipment descriptions, technical parameters, operating procedures, etc.; clean the data to remove noise and irrelevant information to ensure data quality;

[0062] Process the collected power texts through natural language processing, including removing advertisements and blank symbols, and segmenting the texts into separate word strings and marking the parts of speech;

[0063] Fill missing fields in the collected power text using mean, mode, and context-based filling;

[0064] Classify power text data into types such as operation data, fault reports, equipment maintenance records, and customer feedback according to their content, and add error labels to each type of power text data, including terminology errors, spelling errors, and grammatical errors, as well as power equipment types and problem categories;

[0065] The processed power text data is organized into a structured dataset according to categories, labels and sources, which is used as the power text dataset.

[0066] Step 2, establish a statistical language model based on the spatial lag model architecture; the statistical language model outputs the probability that word strings of different lengths in the power text conform to the lexical structure; the word strings whose probability of conforming to the lexical structure is less than the set probability threshold are marked as error positions, and candidate words are generated for the error positions to update the power text; the statistical language model outputs the probability that each candidate word in the updated power text conforms to the lexical structure, and the candidate words are arranged in descending order according to the probability of conforming to the lexical structure to obtain a candidate word set;

[0067] Preferably, step 2 comprises:

[0068] Use the power text dataset and train the dataset based on N-gram in the statistical language model SLM;

[0069] Based on the N-gram statistical language model, bidirectional multi-granularity analysis is used to identify word string error locations in power text data. Multi-granularity refers to multiple context segmentation methods based on single words, two-word combinations, and three-word combinations. This includes traversing the context window word by word and calculating the probability that the word string is lexical. When the probability of lexical compliance is lower than the set threshold of 0.5, potential error locations are marked. The granularity is dynamically adjusted to capture context information, and errors with similar pinyin and glyphs are located.

[0070] Based on the potential error positions identified by multi-granularity N-grams, the recognition results at different granularities are weighted and fused by the probability of conforming to the lexical structure, and the comprehensive credibility score of each error position is output. The optimized error position list is generated by sorting by credibility score.

[0071] Based on the finalized list of error positions, a candidate word set is generated in combination with a statistical language model, including generating phonetically similar candidate words through the edit distance method based on the principle of phonetic similarity, generating structurally similar candidate words through the glyph feature vector based on the glyph similarity principle, and screening and sorting the candidate words using the context association probability of the statistical language model in combination with context semantic information, to generate an optimized candidate word set dominated by the probability of conforming to the lexical structure;

[0072] The error position, context word string and corresponding candidate word set are output in the form of a data table, where each error position contains the probability of lexical compliance, context granularity information and candidate word ranking results. The ranking of candidate words is based on their context matching degree and lexical compliance probability, forming a standardized output data format.

[0073] Preferably, the training of the data set based on N-gram in the statistical language model SLM includes calculating the probability of the word string being lexical in accordance with the following formula for analyzing the word string context matching degree:

[0074]

[0075] Where:

[0076] ε b Indicates the starting element of the power text string; ε e Indicates the end element of the power text string; ε b+1 ,ε b+2 ,…,ε e-1 Represents the element in the middle of the power text fragment;

[0077] P(X piece ) represents the probability of conforming to the lexical structure; P(ε b ) represents the probability of the electric power text string; P(ε b+1 |ε b ) and P(εb+2 |ε b ε b+1 ) represents the conditional probability of the current element occurring given the context; It means that given the b to ε e-1 When all elements of e The conditional probability of occurrence.

[0078] Preferably, the potential error positions identified according to the multi-granularity N-grams are weightedly integrated with the recognition results at different granularities by the probability of conforming to the lexical structure; including:

[0079] For each error position, the probability of word strings conforming to morphology at single-word, double-word, and triple-word granularity is calculated using a statistical language model, and the coverage and frequency statistics of the context window corresponding to the position at each granularity are recorded;

[0080] According to the numerical range of the probability of conforming to the lexical structure and the degree of context coverage, a weight factor is assigned. The single-word granularity is used to capture the word string features, and the weight is calculated as follows:

[0081] w1=P(X piece )×R

[0082] Where:

[0083] P(X piece ) represents the probability of conforming to the lexical structure; R represents the contextual association strength; w1 represents the single-word granularity weight;

[0084] The two-word and three-word granularity is used for context association, and the weight is calculated as follows:

[0085] w2=P(X piece )×(1+R)

[0086] Where:

[0087] P(X piece ) represents the probability of conforming to the lexical structure; R represents the contextual association strength; w2 represents the weight of the two-word and three-word granularity;

[0088] By weighted fusion of the recognition results of each granularity, the comprehensive credibility score of each error location is finally output, and the optimized error location list is generated by sorting by credibility score.

[0089] like Figure 2As shown in the figure, the specific process of using the bidirectional multi-granularity N-gram model to calculate the probability that a word string is lexical: typos are detected according to the statistical language model N-gram. Then the probability that the word string is lexical is calculated. If the probability of lexical compliance is lower than the threshold, the error is detected. N-gram is an algorithm based on the statistical language model. Its basic idea is to perform a sliding window operation of size N on the content of the text according to bytes to form a byte fragment sequence (gram) of length N. Assuming that the appearance of the Nth word is only related to the previous N-1 words and not to any other words, the probability of the whole sentence is the product of the probability of occurrence of each word. Filter according to the pre-set threshold to form a gram list, that is, the vector feature space of the text. Each gram in the list is a feature vector dimension. Based on the N-gram method, it can be used to evaluate the rationality of a sentence or the degree of difference between two strings. Assume a sentence X = [ε1,ε2...ε n ], N-gram language model word string word string X piece =[ε b ,ε b+1 ...ε e The probability of ] being lexical is: According to this formula, a bidirectional multi-granularity N-gram model is used to optimize the error detection effect, and then the probability of the word string being lexical is calculated.

[0090] Step 3,

[0091] An error correction model is established based on the pre-trained language model (Efficiently Learning an Encoder that Classifies Token Replacements Accurately, ELECTRA) architecture. The error position and the candidate word set are input into the error correction model. The error correction model generates candidate word features according to the context of the error position, and determines the matching degree between each candidate word in the candidate word set and the candidate word features. The candidate word with the highest probability of conforming to the lexical structure and the highest matching degree with the candidate word features is used as the replacement word for the error position, and error correction of power text is performed.

[0092] Preferably, step 3 comprises:

[0093] Extract the context word string of the current error position from the generated error position list and the corresponding candidate word set, and vectorize the context word string of the current error position as the input of the ELECTRA pre-trained language model. At the same time, embed the probability of the word string calculated by the SLM statistical language model to the input feature as the preliminary confidence score of the error position in the context.

[0094] In the generator part of the ELECTRA pre-trained language model, the error position is replaced with a special marker [MASK], the context is encoded through the Transformer structure, a high-dimensional vector of each word string is generated, and the output probability distribution of [MASK] is adjusted by combining the probability that the word string conforms to the lexical structure of the SLM statistical language model;

[0095] When the ELECTRA pre-trained language model generates candidate words, the probability of the SLM statistical language model conforming to the lexical structure is used to filter and sort the candidate words;

[0096] According to the length of the current error position context, the fusion weight α of the SLM statistical language model and the ELECTRA pre-trained language model is dynamically adjusted. When the context fragment is a phrase, the SLM statistical language model is prioritized to verify whether the vocabulary combination is reasonable; when the context is a long sentence, the ELECTRA pre-trained language model is prioritized to analyze the semantic relationship of the context word string;

[0097] The candidate words output by the generator are re-embedded into the context and input into the discriminator part of the ELECTRA pre-trained language model. The classifier is used to determine whether each candidate word conforms to the overall semantics of the context. The probability of the context conforming to the lexical structure provided by the SLM statistical language model is used to correct the judgment result.

[0098] After completing the above steps, the ELECTRA pre-trained language model is fine-tuned through supervised learning using power text data containing correct and incorrect annotations. The generator is optimized by minimizing the cross entropy loss between the [MASK] replacement word and the true label, while the discriminator optimizes the error recognition ability by maximizing the classification accuracy. The SLM statistical language model is combined with the probability of the lexical compliance at each error position to participate in the weight adjustment of the loss function to optimize the combination of the generator and the discriminator.

[0099] After integrating the SLM statistical language model with the ELECTRA pre-trained language model, an optimized power text error correction model using statistical information and contextual semantics is generated.

[0100] Preferably, in the process of combining the generated candidate words with the lexical probability of the SLM statistical language model in the ELECTRA pre-trained language model, the candidate words are filtered and sorted using the lexical probability of the SLM statistical language model; including:

[0101] The combination of the probability of lexical matches generated by the SLM statistical language model and the probability of candidate words generated by the ELECTRA pre-trained language model is calculated as follows:

[0102] P1=α·P SLM +(1-α)·PELECTRA

[0103] Where:

[0104] P1 represents the final probability that the candidate word is selected as the replacement word for the wrong position under given context conditions; P SLM Indicates the probability that a candidate word is lexical based on the SLM statistical language model in a given context;

[0105] P ELECTRA It represents the generation probability of a candidate word in a given context, which is calculated based on the generator part of the ELECTRA pre-trained language model; α represents the fusion weight coefficient.

[0106] like Figure 3 As shown in the figure, the detailed steps of error detection and correction based on the ELECTRA model are:

[0107] In the process of checking for typos, the power text containing the erroneous information is first input into the ELECTRA discriminator, and the vectorized representation of the context information is obtained through the transformer encoder, and then enters the hidden layer. The classification discriminator determines whether the input information is the same as the correct power text. If it is the same, True is output; if it is not the same, False is output. After determining the position of the misspelled word, the first step is to correct the misspelled word based on the statistical language model, construct a confusion set consisting of misspelled words, phonetically similar words, and polysyllabic words in the text, and then select words from it to replace the misspelled words, and finally use the probability of the word string at the replacement position being lexical as the basis for sorting. For the above error message, the output is False, the typo position is replaced with [MASK], and then it is input into the ELECTRA generator to predict [MASK]. The vectorized representation of the context information is obtained by the transformer encoder, and then through the hidden layer and the embedding layer, the vector representation of the context information is finally mapped to [MASK]. Through the FNN network and soft-max transformation, the probability distribution in the vocabulary is obtained, and the candidate set and its sorting are obtained.

[0108] When fine-tuning the ELECTRA discriminator model, the direct idea is to replace the words at specific positions in the text with random new words with a certain probability, so as to obtain a set of texts containing typos and their corresponding True / False labels. In practice, it is found that using the typo detection function and candidate set sorting function of the N-gram model can help to assist the construction of the corpus required for the ELECTRA discriminator fine-tune, and even the fine-tune process. Take any sentence of text, first use the typo detection function of the N-gram model to obtain the distribution of the probability that the word string at each position conforms to the lexical structure. Map the error detection result to 0 / 1, which is used as the attention mask when calculating the loss in the ELECTRA discriminator fine-tune training. According to the error detection result of the N-gram model, randomly select a word in the position that does not contain typos, replace it with the new word with the highest probability of conforming to the lexical structure of the N-gram candidate sorting, and generate a label for loss calculation. Finally, when calculating the discriminator loss, only the position where attention mask = 1 is calculated for updating the discriminator parameters.

[0109] Fine-tune the ELECTRA generator, replace the words at specific positions in the text with [MASK] token with a certain probability, and then predict the [MASK] position based on the context information. The replaced words are used as labels to participate in the generator loss calculation. At this time, if the quality of the power text used for training is unknown, the words replaced by [MASK] token have a certain probability of being typos, which will obviously interfere with the training. It can be found that the typo detection function of the N-gram model can effectively avoid this problem. You only need to use the N-gram model to calculate the probability that the word string at each position conforms to the lexical structure for any sentence of text, and replace the positions with the probability of conforming to the lexical structure higher than the threshold with [MASK] token.

[0110] Use the validation set and test set to fully evaluate the model, including indicators such as accuracy, recall, and F1 score.

[0111] This embodiment evaluates the effectiveness of the correction model through four indicators: accuracy, precision, recall rate and comprehensive index F1. The model combines the SLM model and the ELECTRA pre-training model. The "Warmup and Decay" learning rate scheduling strategy is adopted to dynamically adjust the learning rate (Learning Rate) parameters to speed up the model training efficiency. Secondly, balance the computational efficiency and memory consumption, and adjust the training batch size (Batch Size). According to the language characteristics of the power text and the size of the data set, the embedding layer dimension is adjusted, and the size ratio of the generator and the discriminator is determined to ensure the performance and computational complexity of the model.

[0112] The key hyperparameters and their settings are as follows:

[0113] Learning rate: initially set to 3e-5, with a warm-up phase in the first 10% of the steps, followed by linear decay; Batch size: 32, chosen to balance memory constraints and training efficiency; Number of epochs: 10, to ensure sufficient training without overfitting; Dropout Rate: 0.1, to prevent overfitting by randomly omitting units during training; Maximum sequence length: 128 tokens, covering most of the sentences in our dataset.

[0114] Preferably, according to the context matching degree and the probability of conforming to the lexical structure of the candidate replacement words, the candidate word with the highest probability of conforming to the lexical structure and the highest context matching degree is selected as the optimal replacement word for correction, and the corrected power text is output; including:

[0115] Input the power text to be corrected, the error location is marked as [MASK], and extract the context information of the error location;

[0116] The SLM statistical language model calculates the probability of each candidate word being lexical based on the context and screens the candidate words based on the preliminary confidence score. The ELECTRA pre-trained language model evaluates the semantic matching of the candidate words through the generator and discriminator and calculates the context matching of each candidate word.

[0117] The probability of lexical matching generated by the SLM statistical language model is weighted and fused with the context matching calculated by the ELECTRA pre-trained language model to obtain the final score of each candidate word;

[0118] The candidate words are sorted according to the final score, and the candidate word with the highest score is selected as the optimal replacement word; the selected optimal replacement word replaces the wrong position [MASK] in the original power text to generate a corrected power text.

[0119] Preferably, the weighted fusion of the lexical probability generated by the SLM statistical language model and the contextual matching degree calculated by the ELECTRA pre-trained language model to obtain the final score of each candidate word includes:

[0120] P2=α′·PSLM+(1-α′)·P ELECTRA

[0121] Where:

[0122] α′ represents the weight coefficient combining the preliminary confidence score and the context matching degree; P2 represents the final score of the candidate word.

[0123] Example 2 of the present invention provides a Chinese electric power text error correction system based on the SLM-ELECTRA model, including: a data acquisition and processing module, an error location identification module, a candidate word generation and model fusion module, and a candidate word screening and correction module;

[0124] The data collection and processing module is used to collect original power texts from multiple sources, generate power text data sets through data cleaning, denoising and classification, and provide basic data for subsequent error recognition, candidate word generation and model training;

[0125] The error location identification module is used to analyze the word strings in the power text through the SLM statistical language model, identify potential error locations, and use the bidirectional multi-granularity N-gram model to accurately identify the error locations at different granularities. The granularity is dynamically adjusted based on the context information, and a candidate word set is generated on this basis. At the same time, the candidate words are weighted and optimized by the probability of conforming to the lexical structure.

[0126] The candidate word generation and model fusion module is used to input the error position and candidate word set into the ELECTRA pre-trained language model, embed the probability of the word string calculated by the SLM statistical language model as a priori features, and generate candidate words through the generator of the ELECTRA model, identify and correct the candidate words through the discriminator, and dynamically adjust the fusion weight of the SLM and ELECTRA models according to the context information and the probability of lexical compliance, and finally generate an optimized power text error correction model;

[0127] The candidate word screening and correction module is used to calculate the probability of each candidate word string conforming to the lexical structure through the SLM statistical language model to screen out a set of legal candidate words, use the ELECTRA pre-trained language model to sort the candidate words by context matching, select the candidate word with the highest context matching and the highest probability of conforming to the lexical structure as the optimal replacement word for correction to generate grammatically correct and semantically consistent power text, and output the final correction result.

[0128] like Figure 4 As shown, by comparing the error correction performance of different models, the effect of the present invention is demonstrated. In this embodiment, three groups of control experiments are set up under the same environment to verify the effectiveness of the present method:

[0129] (1) Only use the SLM (N-gram) statistical language model;

[0130] (2) Only use the ELECTRA pre-trained model;

[0131] (3) SLM-ELECTRA: SLM and ELECTRA are integrated to form the Chinese grammatical error detection method proposed in the present invention;

[0132] The training set is verified, and the experimental results are as follows Figure 4 As shown in the figure. This method not only has the highest accuracy of 0.6876, but also the error correction F1 of the power text finally reaches 0.6354, which is significantly better than the N-gram method and the Electra method, which are 0.5871 and 0.4738 respectively. Finally, repeat the above operation to predict the test set, and find that the combined model is still significantly better than another single model.

[0133] The present invention integrates SLM with ELECTRA, and uses the ELECTRA pre-trained language model in the input layer to avoid the problem of data mismatch between the pre-training stage and the fine-tuning stage; in order to improve efficiency and save costs, the present invention optimizes the shortcomings of the small number of samples in the power text error correction task by comparing the error correction effects of three different models. The present invention has strong versatility, does not require manual extraction of lexical and syntactic features, saves manpower and time costs, has good effects, and meets the needs of power text error detection and correction.

[0134] The above is only a preferred implementation of a Chinese power text error correction method based on the SLM-ELECTRA model. The protection scope of a Chinese power text error correction method based on the SLM-ELECTRA model is not limited to the above embodiments. All technical solutions under this idea belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, several improvements and changes without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0135] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A Chinese power text error correction method based on the SLM-ELECTRA model, characterized in that: The following steps are involved: Collect various types of power texts, including: operation data, fault reports, equipment maintenance records and customer feedback; split power texts into word strings of different lengths; A statistical language model is established based on the spatial lag model architecture; the statistical language model outputs the probability that word strings of different lengths in the power text conform to the lexical structure; the word strings whose probability of conforming to the lexical structure is less than the set probability threshold are marked as error positions, and candidate words are generated for the error positions to update the power text; the statistical language model outputs the probability that each candidate word in the updated power text conforms to the lexical structure, and each candidate word is arranged in descending order according to the probability of conforming to the lexical structure to obtain a candidate word set; An error correction model is established based on the pre-trained language model architecture. The error position and candidate word set are input into the error correction model. The error correction model generates candidate word features according to the context of the error position, and determines the matching degree between each candidate word in the candidate word set and the candidate word features. The candidate word with the highest probability of conforming to the lexical structure and the highest matching degree with the candidate word features is used as the replacement word for the error position to correct errors in the power text.

2. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 1 is characterized by: The collection of various types of power texts includes: Utilize application programming interfaces and sensor data interfaces to collect power text from multiple sources, including equipment monitoring systems, operation reports, and power load data; Add error labels, power equipment category labels and data source labels to each type of power text, where error labels include terminology errors, spelling errors and grammatical errors; The power text is organized into structured power text data according to error labels, power equipment category labels and data source labels.

3. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 1 is characterized by: The statistical language model outputs the probability that word strings of different lengths in the power text conform to the lexical structure, including: Based on the N-gram model, the structured power text data is segmented at different granularities to generate word strings of different lengths; the granularities include single-word, double-word, and triple-word.

4. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 1 is characterized by: The statistical language model outputs the probability of each candidate word in the updated power text being lexical, and arranges each candidate word in descending order according to the probability of being lexical, so as to obtain a candidate word set; include: Using the statistical language model, the probability of word strings of different lengths in the power text being lexical is output, and a probability threshold is set at the same time, and word strings with a probability of being lexical less than the set probability threshold are marked as error positions; Generate candidate words for the word string in the wrong position according to the following rules: 1) Obtain the pinyin of the wrong position and generate candidate words with similar pinyin according to the pinyin similarity principle; 2) Obtain the glyphs at the wrong position and generate candidate words with similar glyphs according to the glyph similarity principle; 3) Obtain the context of the error location and generate semantically related candidate words according to the semantic relevance principle.

5. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 4 is characterized by: The probability of a word string being lexical is calculated as follows: In the formula, P(X piece ) represents the probability of matching the lexical structure, X piece is a word string, P(ε b ) represents the probability of the electric power text string, P(ε b+1 |ε b ) indicates that ε is known b In the case of b+1 Conditional probability; P(ε b+2 |ε b ε b+1 ) indicates that ε is known b and ε b+1 In the case of b+2 Conditional probability; It means that given the b to ε e-1 When all elements of e Conditional probability of occurrence; ε b Indicates the starting element of the electric power text string; ε e Indicates the end element of the power text string; b+1 ,ε b+2 ,…,ε e-1 Represents the element in the middle of an electric text fragment.

6. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 1 is characterized by: The error correction model is established based on the pre-trained language model architecture, the error position and the candidate word set are input into the error correction model, the error correction model generates candidate word features according to the context of the error position, and determines the matching degree between each candidate word in the candidate word set and the candidate word features; The candidate word with the highest probability of conforming to the lexical structure and the highest matching degree with the candidate word features is used as the replacement word for the error position to correct the power text errors, including: The error position, the candidate word set and the context of the error position are input into the error correction model, and the error correction model generates the context feature of the error position and the candidate word feature through the context feature generator; The error correction model uses the generator part of the pre-trained language model and combines the context features and candidate word features into the generator to generate multiple candidate replacement words. The discriminator part of the pre-trained language model is used to calculate the matching degree between each candidate replacement word and the error position context according to the context features and candidate word features. The probability of lexical matching calculated by the statistical language model and the matching degree calculated by the pre-trained language model are weighted and fused according to the set fusion weight coefficient to obtain the comprehensive matching degree of the candidate replacement word; The candidate replacement words are sorted according to the comprehensive matching degree, and the candidate replacement word with the highest comprehensive matching degree is selected as the replacement word for the error position.

7. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 6 is characterized by: The error correction model generates context features and candidate word features of the error position through a context feature generator, including: Mark the error location with a special marker [MASK]; The Transformer encoder of the pre-trained language model is used to encode the contextual information of the error position and generate contextual features, including the word string and part of speech in the context; Based on the context features, the features of the candidate words are extracted, including phonetic, glyphic and semantic features.

8. The Chinese power text error correction method based on the SLM-ELECTRA model according to claim 6 is characterized by: The comprehensive matching degree of the candidate replacement words is calculated as follows: P.S 综合 Nα·P(X piece )+(1-α)·P match Where P 综合 represents the comprehensive matching degree of candidate replacement words; P(X piece ) represents the probability of conforming to the lexical structure; P match It represents the matching degree between the candidate replacement word and the context of the error position; α represents the set fusion weight coefficient.

9. A Chinese power text error correction system based on the SLM-ELECTRA model, comprising: Data collection and processing module, statistical language model module and candidate word screening and correction module; characterized by: The data collection and processing module is used to collect various types of power text data, including operation data, fault reports, equipment maintenance records and customer feedback, and to segment the power text into word strings of different lengths; The statistical language model module is used to establish a statistical language model, output the probability that word strings of different lengths in the power text conform to the lexical structure, mark the error positions, and generate candidate words to update the power text; The candidate word screening and correction module is used to build an error correction model, input the error position and candidate word set into the error correction model, use the context of the error position to generate candidate word features, calculate the matching degree between each candidate word and the candidate word features, and select the optimal replacement word to correct text errors.

10. The Chinese power text error correction system based on the SLM-ELECTRA model according to claim 9 is characterized by: The statistical language model module includes: Using the statistical language model, the probability of word strings of different lengths in the power text being lexical is output, and a probability threshold is set at the same time, and word strings with a probability of being lexical less than the set probability threshold are marked as error positions; When generating candidate words for a word string at an incorrect position, the pinyin of the incorrect position is obtained and candidate words with similar pinyins are generated according to the pinyin similarity principle; candidate words with similar glyphs are generated according to the glyph similarity principle; and semantically related candidate words are generated according to the context of the incorrect position and the semantic relevance principle.

11. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the Chinese electric power text error correction method based on the SLM ELECTRA model according to any one of claims 1-8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of a Chinese electric power text error correction method based on the SLM ELECTRA model described in any one of claims 1 to 8 are implemented.

Images