Electric power communication scheduling data intention identification method based on confusion degree screening

Abstract

The invention relates to the field of electric power communication scheduling data, and provides an electric power communication scheduling data intention identification method based on confusion degree screening, which comprises the following steps of: collecting related consultation data of an electric power communication scheduling system in the whole year, the data covering but not limited to typical electric power communication scheduling data such as a transmission service, a PCM service, a VPN service and the like; an n-gram language model is constructed by using a KenLM algorithm to calculate a sample perplexiity (PPL), a language probability value is efficiently retrieved through a TRIE structure, and medium and high-quality corpora are screened to improve semantic consistency and expression normalization; and through combination of semantic rule filtering and content deduplication, high-quality data with a more standard structure and no information redundancy are extracted. Then, a Transform model is constructed based on a multi-head self-attention mechanism, context semantic features are extracted, and a final feature matrix is obtained through a multi-layer encoder; finally, intention prediction probability distribution is output through a full connection layer and a Softmax classification layer, and a final scheduling intention label is determined according to a threshold value.

Description

Technical Field

[0001] The present invention relates to the field of electric power communication dispatching data, and in particular to a method for identifying intentions of electric power communication dispatching data based on perplexity screening. Background Art

[0002] With the continued development of new power systems, power communication and dispatching are gradually evolving from traditional manual operations to intelligent and automated ones. As power systems expand in size and the proportion of renewable energy access continues to rise, dispatching tasks are becoming more frequent, diverse, and time-sensitive, placing higher demands on the dispatching system's intelligent identification and response capabilities.

[0003] In recent years, intent recognition methods based on deep learning have become an important research direction of intelligent dispatching systems. Among them, natural language processing models represented by Transformer have been gradually introduced into the field of power communication dispatching due to their advantages in long text modeling and semantic feature extraction.

[0004] However, despite the powerful modeling capabilities of the Transformer, current Transformer-based methods for identifying intent in power communication dispatching still suffer from insufficient accuracy in practical applications. On the one hand, the dispatching corpus is generally characterized by uneven language quality and non-standard semantic expression, which directly affects the effectiveness of model training. On the other hand, the lack of an effective data screening mechanism makes the model susceptible to interference from low-quality samples, limiting the Transformer model's performance in semantic modeling and classification accuracy. Introducing a data quality evaluation mechanism for power communication corpus within the existing Transformer modeling framework to improve the model's understanding of dispatching instructions, professional terminology, and multi-round conversation intent, thereby significantly enhancing the accuracy of intent recognition, has become one of the key technical challenges in promoting the intelligent upgrade of the dispatching system and ensuring the safe and stable operation of power grid dispatching. Summary of the Invention

[0005] (1) Technical problems solved

[0006] The present invention relates to the field of electric power communication dispatching, and in particular to a method for identifying intent in electric power communication dispatching data combined with language model perplexity screening. The method aims to improve the accuracy and intelligence of intent recognition in existing dispatching systems and address key technical issues such as the sensitivity of current Transformer model-based intent recognition methods to low-quality corpus and limited training accuracy.

[0007] (2) Technical solution

[0008] The current power communication dispatching system urgently needs a semantic modeling method with high recognition accuracy and strong generalization ability when processing complex instructions, alarm statements and multi-round dialogue scenarios.

[0009] This paper innovatively introduces a language model perplexity (PPL) screening mechanism, combined with a multi-head self-attention Transformer architecture, to perform a two-layer optimization of the power dispatch corpus: perplexity screening is used in the early stage to improve the corpus quality, and deep modeling is used in the later stage to extract high-order semantic features, forming a synergistically enhanced intent recognition process.

[0010] A method for identifying intent in power communication dispatching data based on perplexity screening, the method comprising the following steps:

[0011] S1. Collect the data related to the dispatch consultation of power communication business for the whole year. The data covers but is not limited to typical power communication dispatch data such as transmission business, PCM business, VPN business, etc., including the query dialogue containing n rounds of questions. , knowledge related to the n-round problem and responses to the questions ;

[0012] S2. Preprocess the power communication dispatching business data to retain only English and simplified Chinese data; then, standardize the character encoding of the corpus (e.g., unify it to UTF-8 encoding) and remove garbled and illegal characters to obtain the preprocessed communication business dispatching data;

[0013] S3. Perplexity (PPL) is calculated for the preprocessed data using an n-gram language model built using the KenLM algorithm. The TRIE structure within this language model efficiently queries the language probability of each corpus sample, assessing its fluency and rationality. All samples are sorted in ascending order by perplexity. Data ranked in the top 30% by perplexity are considered high-quality, while data ranked between 30% and 60% by perplexity are considered medium-quality. Only these two categories of data are retained as input for subsequent model training.

[0014] S4. Further data cleaning is performed on the automatically scored data. Specifically, this includes: verifying the semantic consistency and formatting of the corpus content based on preset rules, removing samples containing special symbols, semantic conflicts, or formatting anomalies; and performing content deduplication to eliminate duplicate, near-duplicate, or template-based conversation content, generating a high-quality data sequence with a clear structure and no duplication.

[0015] S5. Construct a Transformer model based on a multi-head self-attention mechanism. This model embeds sequence order information through positional encoding and uses a self-attention module to assign context-dependent dynamic weights to the fused feature vectors, achieving deep modeling of the semantic representation of power communication dispatch data. The constructed Transformer model's attention module is used to pre-train the pre-processed power communication dispatch data to obtain the final output feature matrix.

[0016] S6: The resulting output feature matrix is ​​fed into a fully connected layer for linear transformation. It then undergoes nonlinear processing using the ReLU (Rectified Linear Unit) activation function before being fed into a softmax layer to generate a probability distribution P for intent prediction. Based on the probability distribution P and the set threshold, the final dispatch intent label is determined, and the intent recognition result is output.

[0017] The Transformer encoder introduces a self-attention mechanism that simultaneously focuses on all features in the input sequence, modeling inter-feature dependencies. This mechanism dynamically assigns weights based on the importance of each feature, highlighting key semantic features and suppressing irrelevant information. Furthermore, the multi-head attention mechanism, by computing multiple independent attention sub-layers in parallel, enables the model to simultaneously capture the multidimensional semantic features of the input sequence across different representation subspaces, enhancing its ability to recognize complex intent patterns.

[0018] Furthermore, the categories of the power communication dispatching system related data in S1 include but are not limited to power load dispatching, equipment status monitoring, fault handling, communication link maintenance and energy optimization management.

[0019] Furthermore, the perplexity scoring mechanism in the S3 is based on the n-gram language model constructed using the Kneser-Ney smoothing strategy, and uses the TRIE structure to perform efficient probability queries on samples and calculate the perplexity (PPL). After sorting in ascending order, the top 30% and 30%–60% samples are selected as training data.

[0020] Furthermore, the Transformer model in S5 includes an encoder structure, which is composed of a stack of multiple Transformer encoding layers, each encoding layer including a multi-head self-attention mechanism and a feedforward neural network module.

[0021] Furthermore, the output result of the output feature matrix in S6 is the probability distribution corresponding to the intent category, which is calculated through the Softmax layer and the final intent label is determined based on the set decision threshold.

[0022] (3) Beneficial effects

[0023] In summary, the present invention introduces a corpus screening mechanism based on perplexity scoring, forms a collaborative optimization at three levels: corpus quality evaluation mechanism, data screening strategy and Transformer modeling structure, significantly improves the language rationality and semantic consistency of training samples, and effectively solves the problems of inaccurate recognition and insufficient semantic understanding ability in the application of existing models in the field of power dispatching; it breaks through the key bottleneck problem in the current semantic recognition of power dispatching, and provides strong technical support for the construction of a high-performance intelligent dispatching system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0025] Figure 1 A flowchart of a method for identifying intent in power communication dispatching data based on perplexity screening provided by an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the implementation architecture of a power communication dispatching data intention recognition system of the present invention;

[0027] Figure 3 Schematic diagram of the structure of the Transformer encoder of the present invention;

[0028] Figure 4 This is a data diagram of the electric power communication dispatching business of the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be noted that the terms "power communication scheduling data," "intent recognition," "Transformer model," "perplexity calculation," "self-attention mechanism," "Token sequence," etc. are defined based on the method framework proposed in the present invention and are only used to clearly describe the technical solution, rather than to limit the actual power communication system or algorithm implementation. Unless otherwise explicitly stated, the above terms should be understood in conjunction with the context and the common sense of those skilled in the art.

[0031] In addition, the terms "first", "second", "third", etc. are only used to distinguish different modules or steps (such as "first encoding layer", "second decoding layer"), and do not imply the order of execution or difference in importance.

[0032] In the present invention, "intent recognition" should be understood in a broad sense, including but not limited to scenarios such as fault alarm identification, resource scheduling request classification, and business status query matching; "Transformer model" covers its variants (such as BERT, GPT) and improved structures (such as the introduction of domain adaptive pre-training or numerical feature enhancement modules). The connection, processing, and input methods of "power communication scheduling data" can be real-time streaming or offline batch processing; the original log text can be directly input, or it can be transformed through word segmentation, normalization, or structured conversion. Those skilled in the art can adjust the implementation details according to the specific application scenario.

[0033] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] In order to better understand the purpose of the present invention, the present invention is described in further detail below. Figures 1 to 4 .

[0035] A method for identifying intentions in power communication dispatching data based on perplexity screening.

[0036] S1 includes: collecting power communication business dispatch consultation data covering the entire year, including but not limited to typical power communication dispatch data such as transmission business, PCM business, VPN business, etc., including n rounds of question inquiry dialogues , knowledge related to the n-round problem and responses to the questions ;

[0037] Step S2 includes: performing multilingual encoding parsing and cleaning on the collected power communication dispatch data corpus, using Unicode standardization technology (such as NFKC) to normalize the text and unify the internal character representation; at the same time, using a language recognition tool (such as Lang Detect) to detect the language of the corpus, and only retaining text content in English or Simplified Chinese; then, converting all text into the UTF-8 encoding format to ensure encoding consistency and avoid garbled characters. , and convert all Traditional Chinese text to Simplified Chinese text Next, some meaningless symbols are deleted, such as invisible control characters, special symbols, emoticons, etc., but the key symbols for power dispatching (such as 、 );

[0038] The step S3 includes: performing a quality score on the pre-processed corpus data obtained in step S2, and using a language model to calculate the perplexity (PPL) as a data quality evaluation indicator. To improve the efficiency and accuracy of the model scoring, this embodiment uses the KenLM algorithm tool to construct an n-gram language model based on Kneser-Ney smoothing, and uses its internal TRIE data structure for efficient querying. First, determine the given power communication scheduling data set , where each piece of data is a word sequence The KenLM algorithm uses Kneser-Ney smoothing to calculate N-gram probabilities. , the formula is as follows:

[0039] ,in: is the count of N-grams in the training data. is the discount factor (usually 0.75). is the normalization factor, This is the smoothed recursive probability estimate to ensure that the sum of the probabilities is 1. In the specific implementation, the KenLM algorithm uses TRIE (inverted prefix tree) to sort and store n-gram records in suffix order. Each layer of n-gram table is an ordered array structure. When querying, in order to improve the search speed, the interpolation search algorithm is introduced, which estimates the possible position of the target word in the array. as follows:

[0040] in, Represents an array of ordered word IDs, and are the starting and ending indexes of the current search interval, is the hash value or vocabulary code of the target word. This search method can shorten the search time from the traditional binary search to Reduce to , significantly improving the query efficiency of the model on large-scale corpus. Then calculate the perplexity of the data. The perplexity measures the degree of fit of the language model to the data. The lower the PPL, the higher the data quality. , its PPL is calculated as follows:

[0041] Next, the PPL scores are ranked from low to high and graded. The top 30% of PPL data are considered high-quality data. , data with a PPL ranking between 30% and 60% are considered to be of medium quality Only high-quality and medium-quality data are retained;

[0042] Step S4 includes: rule-based content filtering: multiple filtering rules are designed at the paragraph and sentence levels to ensure the effectiveness of the data filtering process. At the paragraph and sentence levels, the focus of the rules shifts to abnormality. In general, this function is to filter out bad information based on the two types of paragraphs and sentences respectively; content deduplication: duplicate text in the pre-trained data may affect the subsequent transformer architecture's intent recognition performance for power communication scheduling data. Therefore, this paper plans to use BloomFilter and Simhash to deduplicate text. On this basis, caching and paging technologies will also be used to optimize this process. The specific deduplication steps are as follows: First, use BloomFilter to deduplicate text based on URL (Uniform Resource Locator), which will greatly reduce the amount of computation required for subsequent content deduplication. Second, use BloomFilter to perform precise deduplication on the imported data. Finally, use SimHash to perform fuzzy deduplication on the text content. Although this step may exclude some high-quality data, the evaluation of the sampled deduplication data shows that this loss is acceptable in order to pursue higher training efficiency.

[0043] The step S5 includes: performing tokenization and embedding processing on the data sequence generated in step S4, and then inputting it into the intent recognition model based on the Transformer architecture for training and reasoning.

[0044] During the model training process, the size of the weight matrix corresponding to different cross-features is automatically learned to indicate the importance of the features, which helps the model focus on more important features. For the processed power communication dispatch data, it will be input into multiple Transformer encoder layers for processing. Each encoder layer includes a self-attention mechanism and a feedforward neural network. The self-attention mechanism allows the model to focus on different positions in the input sequence to better understand the relationship between words. The Transformer encoder contains a self-attention mechanism that can simultaneously consider all input features in the sequence and calculate the impact of each feature on other features. The model can also assign different weights according to the importance of each feature, so as to better identify key features related to lane change intentions. And its multi-head attention mechanism enables the model to focus on different parts of the input sequence in different subspaces at the same time by calculating multiple attention sub-layers in parallel, thereby capturing richer feature representations. The formula is as follows: First, the input sequence is subjected to three independent linear transformations to obtain the query matrix Q, key matrix K, and value matrix V:

[0045]

[0046] In the formula is a trainable weight matrix;

[0047] A set of ( ) is called an attention head. Apply the above attention mechanism to h heads, each head corresponds to a different linear transformation of the query, key, and value;

[0048]

[0049]

[0050] Where;

[0051] Where i=1, 2, ..., n; n is the number of attention heads, is a learnable weight parameter matrix; the output multi-head attention result is fused with the feature vector in the original input sequence through tensor expansion to form a weighted feature expression;

[0052] Concatenate the outputs of all heads and perform a linear transformation to obtain the final multi-head attention output;

[0053]

[0054] in, is a trainable weight matrix, is the dimension of the value.

[0055] The multi-head attention mechanism computes multiple independent attention heads in parallel, enabling the model to simultaneously focus on different parts of the input sequence in different representation subspaces, thereby capturing richer and more diverse features. This mechanism greatly enhances the performance of the Transformer encoder in processing complex sequence data tasks, such as question-answering intent recognition in power communication dispatch data.

[0056] Through position encoding, Embedding calculation: It is converted into Q, K, and V through the weight matrix. Then the attention weight is calculated to evaluate the correlation between features. The obtained attention weight A is used to perform weighted summation on the value V to obtain the output of self-attention, input the multi-head attention mechanism and obtain the output Z. It is then transformed nonlinearly through a feedforward neural network consisting of two fully connected layers, and finally the residual connection and normalization are performed to output the final feature matrix.

[0057] ;

[0058] The step S6 includes: performing a linear transformation of the finally generated vector B through a fully connected layer: ,in and is a trainable parameter. Then use the ReLU (Rectified Linear Unit) activation function for nonlinear calculation: , then, the nonlinear output Input to the Softmax layer to generate the probability distribution P of intent prediction: ,in and For the weights and bias terms of the final classification layer, set a decision threshold based on the output probability distribution P , if the probability of a certain category , it is determined to be the final predicted intention category, and the category is output as the recognition result label of the power communication scheduling data.

[0059] Intent recognition in power communication dispatch data is a multimodal time series classification task. The Transformer architecture, due to its powerful cross-modal feature extraction capabilities, is an ideal choice. This task requires processing both time series data on equipment status and textual data on dispatch instructions. Traditional unimodal models struggle to effectively capture the complex relationships between the two. The Transformer model utilizes a self-attention mechanism to achieve three core functions: time series feature extraction, text semantic understanding, and cross-modal feature fusion. A dedicated cross-attention head establishes a mapping between numerical indicators and textual instructions, automatically associating, for example, "bandwidth utilization > 90%" with the instruction "activate backup channel."

[0060] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for identifying intentions in power communication dispatching data based on perplexity screening, characterized in that: The following steps are involved: S1. Collect data related to power communication business dispatch consultation throughout the year, including but not limited to typical power communication dispatch data such as transmission business, PCM business, and VPN business; S2. Parsing and preprocessing the data to construct a structured data sequence; S3. Using the KenLM algorithm based on the perplexity scoring mechanism of the language model to evaluate the quality of the structured data sequence, and then ranking and grading the perplexity scores from low to high; The top 30% of PPL data are considered high-quality data , data with a PPL ranking between 30% and 60% are considered to be of medium quality ; And retain high-quality and medium-quality data samples according to the PPL sorting results as input for subsequent model training; Perform word embedding processing on the filtered data sequence and add position encoding information; Form the input vector representation of the Transformer model; S4. Further data cleaning is performed on the automatically scored data. Specifically, this includes: verifying the semantic consistency and formatting of the corpus content based on preset rules, removing samples containing special symbols, semantic conflicts, or formatting anomalies; and performing content deduplication to eliminate duplicate, near-duplicate, or template-based conversation content, generating a high-quality data sequence with a clear structure and no duplication. S5. Build an intent recognition network based on the Transformer model, including multiple encoder layers. Each encoder layer uses a multi-head self-attention mechanism to extract semantic features from the input data. S6. Based on the above output feature matrix, a linear transformation is performed through the fully connected layer, and a nonlinear mapping is performed in combination with the ReLU activation function. Finally, the probability distribution of the intent prediction is output through the Softmax layer. According to the set decision threshold, the probability distribution is judged and the intent label is output; Generate corresponding scheduling tasks based on intent label classification, and send the scheduling tasks to the corresponding execution module according to preset strategies and priorities.

2. The method for identifying intentions in power communication dispatching data based on perplexity screening according to claim 1 is characterized in that: The categories of data related to the power communication dispatching system in S1 include but are not limited to power load dispatching, equipment status monitoring, fault handling, communication link maintenance and energy optimization management.

3. The method for identifying intentions in power communication dispatching data based on perplexity screening according to claim 1 is characterized by: The perplexity scoring mechanism in S3 is based on the n-gram language model constructed using the Kneser-Ney smoothing strategy. It uses the TRIE structure to perform efficient probability queries on samples and calculate the perplexity (PPL). After sorting in ascending order, the top 30% and 30%–60% samples are selected as training data.

4. The method for identifying intentions in power communication dispatching data based on perplexity screening according to claim 1 is characterized in that: The Transformer model in S5 includes an encoder structure, which is composed of a stack of multiple Transformer encoding layers, each of which includes a multi-head self-attention mechanism and a feedforward neural network module.

5. The method for identifying intentions in power communication dispatching data based on perplexity screening according to claim 1, characterized in that: The output result of the output feature matrix in S6 is the probability distribution corresponding to the intent category, which is calculated through the Softmax layer and the final intent label is determined based on the set decision threshold.

6. The method for identifying intentions in power communication dispatching data based on perplexity screening according to claim 1, characterized in that: In S3, the KenLM algorithm uses the TRIE inverted prefix tree to sort and store n-gram records in suffix order. Each layer of n-gram table is an ordered array structure. When querying, in order to improve the search speed, the interpolation search algorithm is introduced to estimate the possible position of the target word in the array. as follows: in, Represents an array of ordered word IDs, and are the starting and ending indexes of the current search interval, Hash or lexical encoding of the target word; This search method can shorten the search time from traditional binary search to Reduce to , improve the query efficiency of the model on large-scale corpus; then calculate the perplexity of the data, PPL, which measures the degree of fit of the language model to the data. The lower the PPL, the higher the data quality. , its PPL is calculated as follows: .

7. The method for identifying intentions in power communication dispatching data based on perplexity screening according to claim 1, characterized in that: In S5, the encoder calculates the weight score of each value vector through the query vector (Q), key vector (K) and value vector (V), and combines these weights with the value vector to obtain a weighted sum; the attention calculation formula is as follows: Where Q, K, and V are query, key, and value vector matrices, respectively, and their dimensions are , and the dimensions of the three are equal; The multi-head attention mechanism is to splice together the projections of Q, K, and V obtained by h different linear transformations. Its calculation formula is as shown below: Among them, i=1, 2, ..., n, n is the number of attention heads, is a learnable weight parameter matrix; the output multi-head attention result is fused with the feature vector in the original input sequence through tensor expansion to form a weighted feature expression.

Images