A user question extension method, system and electronic device

Abstract

The application discloses a user question expansion method and system and electronic equipment, and relates to the technical field of artificial intelligence. The user question expansion method provided by the application comprises the following steps: after pre-processing obtained user question data, inputting the pre-processed user question data into a trained power dispatch pre-training language model to obtain a word vector; then, inputting the word vector into a trained conditional random field model to recognize and obtain a question entity; finally, determining the similarity between the question entity and each entity in a power dispatch knowledge graph, and completing the expansion of the user question data based on the similarity, so that the returned information is more accurate and comprehensive while improving the accuracy and recall rate of the query.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a user question expansion method and system applied to power dispatching process and electronic equipment. BACKGROUND

[0002] With the development and application of artificial intelligence technology, power dispatching is the key to maintaining the safe and healthy operation of the power grid system. In the process of power grid dispatching and operation, the disposal measures of the fault are obtained by searching the dispatching knowledge base for the dispatching fault, and the accuracy and comprehensiveness of the returned results are crucial to power dispatching operation. The existing user query has the following problems:

[0003] 1) There are synonyms in Chinese and colloquial phenomena in user query sentences, which result in multiple query sentences corresponding to one query semantics, and users are difficult to choose appropriate words and sentences, which reduces the accuracy of the query results;

[0004] 2) Due to the flexible and variable expression of natural language, the concept of the query is ambiguous, and the business level of the query users is uneven, so it is difficult for the natural language query question to clearly express the user's query intention, and the returned results are difficult to meet the needs of the users;

[0005] 3) The query question text is usually short, and the number of query keywords is small, which is difficult to express the retrieval requirements, resulting in low retrieval accuracy and recall rate, and even no retrieval results. SUMMARY

[0006] To solve the above problems existing in the prior art, the present application provides a user question expansion method, system and electronic equipment.

[0007] To achieve the above purpose, the present application provides the following solutions:

[0008] A user question expansion method, comprising:

[0009] Obtaining user question data and preprocessing the user question data;

[0010] Inputting the preprocessed user question data into a trained power dispatching pre-training language model to obtain a word vector; the power dispatching pre-training language model is a trained PowerBERT model; the PowerBERT model is a BERT model conforming to power dispatching;

[0011] Inputting the word vector into a trained conditional random field model to identify a question entity; the question entity is used to represent the text content associated with power dispatching in the user question data;

[0012] Determining the similarity between the question entity and each entity in the power dispatching knowledge graph;

[0013] based on the similarity.

[0014] Optionally, the training process of the PowerBERT model comprises:

[0015] obtaining a data set; the data set comprises power regulations, power plant duty systems and power dispatch plans;

[0016] preprocessing data in the data set;

[0017] performing word segmentation processing on the preprocessed data in the data set, and constructing an index dictionary of a vocabulary table;

[0018] generating an input sample based on the index dictionary of the vocabulary table;

[0019] generating an input tensor based on the input sample;

[0020] training a BERT model by using the input tensor through back propagation and an optimization algorithm to obtain a trained PowerBERT model; a loss function of the PowerBERT model adopts a cross-entropy loss function; an optimizer adopted in the optimization algorithm is an Adam optimizer.

[0021] Optionally, the preprocessing of the data in the data set specifically comprises:

[0022] adopting a regular expression and a string processing method to remove noise in the data in the data set; the noise includes special characters, HTML tags and non-ASCII characters;

[0023] converting Chinese numerals of equipment names contained in the data in the data set after removing the noise into Arabic numerals.

[0024] Optionally, the preprocessing of the data in the data set and the construction of the index dictionary of the vocabulary table specifically comprise:

[0025] performing word segmentation on the preprocessed data in the data set by using a jieba word segmenter;

[0026] traversing all segmented words to record the frequency of occurrence of each word;

[0027] sorting the words according to the frequency of occurrence of each word to obtain a vocabulary table, adding marks in the vocabulary table, and corresponding each word to a unique index value to generate an index dictionary of the vocabulary table; the added marks include a padding symbol, an unknown word symbol, a start symbol and an end symbol.

[0028] Optionally, the generation of the input sample based on the index dictionary of the vocabulary table specifically comprises:

[0029] According to the input requirement of the PowerBERT model, the length of the word segmentation sequence in the vocabulary table is limited;

[0030] Each word is converted into a tokenization ID corresponding to itself through an index dictionary of the vocabulary table to generate a tokenization ID sequence;

[0031] Position encoding is added to each tokenization ID; the position encoding is used to represent the position information of the word in the word segmentation sequence;

[0032] According to the input requirement of the PowerBERT model, the input sample is constructed; the input sample includes the tokenization ID sequence and the position encoding.

[0033] Optionally, the input tensor is formed by combining the tokenization ID sequence, the segment ID sequence and the position vector sequence.

[0034] Optionally, the annotation of the data in the training data set and the test data set used in the training process of the power dispatch pre-training language model and the conditional random field model adopts the BIO annotation method.

[0035] Optionally, the expansion of the user question data is based on the similarity, and specifically includes:

[0036] The entity with the maximum similarity in the power dispatch knowledge graph is selected, and based on the selected entity, a triple related to the entity is queried from the power dispatch knowledge graph using the Cypher language;

[0037] The triple is added to the user question data to complete the expansion of the user question data.

[0038] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0039] The user question expansion method provided by the present application inputs the preprocessed user question data into the trained power dispatch pre-training language model to obtain a word vector, then inputs the word vector and the label sequence into the trained conditional random field model to recognize a question entity, finally determines the similarity between the question entity and each entity in the power dispatch knowledge graph, and expands the user question data based on the similarity, which can improve the accuracy and recall rate of the query, and make the returned information more accurate and comprehensive.

[0040] In addition, the present application also provides the following implementation structure:

[0041] A user question expansion system applied to the user question expansion method provided above; the system includes:

[0042] a data acquisition module configured to acquire user query data and pre-process the user query data;

[0043] a character vector extraction module configured to input the pre-processed user query data into a trained power dispatch pre-training language model to obtain character vectors; the power dispatch pre-training language model is a trained PowerBERT model; the PowerBERT model is a BERT model conforming to power dispatch;

[0044] an entity extraction module configured to input the character vectors into a trained conditional random field model to identify query entities; the query entities are used to represent text content associated with power dispatch in the user query data;

[0045] a similarity determination module configured to determine the similarity between the query entities and each entity in a power dispatch knowledge graph;

[0046] a data expansion module configured to expand the user query data based on the similarity.

[0047] An electronic device comprises:

[0048] a memory configured to store a computer program;

[0049] a processor connected to the memory and configured to call and execute the computer program to implement the user query expansion method provided above.

[0050] The technical effects achieved by the above two implementation structures provided by the present application are the same as the technical effects achieved by the user query expansion method provided by the present application, and therefore will not be described again here. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0052] Figure 1 a flowchart of the user query expansion method provided by the present application;

[0053] Figure 2 an implementation framework diagram of the user query expansion method provided by the embodiment of the present application. DETAILED DESCRIPTION

[0054] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0055] The present application aims to provide a user question expansion method, system and electronic device, which can improve the accuracy and recall rate of the query, and the returned information is more accurate and comprehensive.

[0056] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0057] As shown in Figure 1 and Figure 2 The user question expansion method provided by the present application comprises:

[0058] Step 100: obtaining user question data and pre-processing the user question data.

[0059] Step 101: inputting the pre-processed user question data into a trained power dispatch pre-training language model to obtain word vectors. The power dispatch pre-training language model is a trained PowerBERT model. The PowerBERT model is a BERT model that meets the power dispatch.

[0060] In this step, the trained power dispatch pre-training language model is used to make the power dispatch pre-training language model more accurate in extracting word vectors. During the training process of the power dispatch pre-training language model, the user question data used can be the question data generated by a dispatcher of a certain power grid company in a power dispatch query system, and the number of collected data can be 14239. This user question data used in the training process is also used for training the conditional random field model in the following step.

[0061] Further, the user question data can be labeled using the BIO labeling method, where B represents the start position of the named entity word, I represents the non-starting part of the named entity word, and O represents the character part that does not need to be recognized. The above-mentioned collected user question data for training the power dispatch pre-training language model and the conditional random field model training and testing are divided into a training set and a test set according to a ratio of 8:2, and the characters of the training set and the test set need to be marked with corresponding labels. In the present application, the entity types can include "fault equipment", "equipment state", "position", and "equipment model", and an example of the labeling of the user question data is shown in Table 1 below.

[0062] Table 1 User question annotation example table

[0063]

[0064] Step 102: input the word vector into the trained conditional random field model to identify the question entity. The question entity is used to represent the text content associated with power dispatch in the user question data.

[0065] Based on the above description, after completing the user question data labeling, the question entity in the user question data is extracted based on the PowerBERT+CRF model, and the specific implementation steps can be:

[0066] First, input the annotated user question data X = (x1, x2, …, xn) into the power dispatch pre-training language model to obtain the word vector X' = (x'1, x'2, …, x'n), where n represents the maximum number of characters in a sentence, xi is the i-th character, x'i is the m-dimensional embedding representation vector of the i-th character, and m can be 768. n n i i

[0067] Second, use the conditional random field (CRF) model to predict the entity labeling of each position. The CRF model considers the relationship between entities and the global consistency of label sequences. The global optimal labeling result is obtained by using the conditional random field to constrain the connected relationship of entity character labels in the question. Here, the word vector is used as the input of CRF.

[0068] Input the word vector X' = (x'1, x'2, …, x'n) into the CRF to obtain the label sequence Y = (y1, y2, …, yn), where yi represents the corresponding entity label. n n i

[0069] The CRF model defines a conditional probability distribution P(Y|X), which represents the probability of the label sequence Y given the input sequence X. The CRF model adopts the form of linear chain conditional random field, and its probability calculation formula is:

[0070]

[0071] where X' is the input word vector sequence, Y represents the label sequence, exp represents the exponential function with natural constant e as the base, n represents the maximum number of characters in a sentence. Z(X') is the normalization factor, which is used to ensure that the sum of the probability distribution is 1. T k (y j ,y j-1 ,X') is the transition feature function, which is used to represent the transition probability of adjacent labels in the label sequence. S​​​​​​​i (y i X') is a state feature function, which is used to represent the label probability of each position.

[0072] The user question is based on the previously described PowerBERT+CRF model for entity recognition, and the identified entity set is E ner , where each entity e i includes entity text and corresponding entity type. For example, for the power dispatch domain question "How to handle the tripping of two 1 million unit generators in Pingwei Phase 3?", the following entities and corresponding entity types can be identified: E ner ={e1="Pingwei Phase 3\tGenerator"}, where "\t" is a delimiter, the text before the delimiter is an entity, and the text after the delimiter is an entity type.

[0073] Specifically, taking the question "How to handle the tripping of two 1 million unit generators in Pingwei Phase 3?" as an example, the entity recognition process using the PowerBERT+CRF model is as follows:

[0074] (1) "How to handle the tripping of two 1 million unit generators in Pingwei Phase 3?" is input into the trained PowerBERT model to obtain word vectors x'1-x' 20 ;

[0075] (2) The word vectors x'1-x' 20 are input into the trained CRF model to predict the entity label for each position, and the identified entity set E ner ={e1="Pingwei Phase 3\tGenerator"} is obtained, thereby achieving the processing of the question and obtaining the question entity.

[0076] Step 103: Determine the similarity between the question entity and each entity in the power dispatch knowledge graph. In this step, the question entity obtained in step 102 and the existing power dispatch knowledge graph are used for similarity calculation in user query expansion, in order to improve the accuracy and recall rate of existing query statements in the existing power query system. The power dispatch knowledge graph includes entity types, relationships, and triples formed based on entity types and relationships. Entity types include "line name", "fault equipment", "equipment state", "part", "equipment model", and "state". Relationships include "contains" and "corresponds to". Triples include (line name, contains, fault equipment) and (equipment state, corresponds to, state).

[0077] The specific implementation process of step 103 can be:

[0078] 1. Input entity vectorization:

[0079] Input 1: All entities in the power dispatch knowledge graph are gathered into a dictionary, denoted as Dict. Each entity is vectorized in the trained PowerBERT model, and a set of knowledge graph entity vector representations is obtained, denoted as V dict .

[0080] Specifically, first, all entities in the power dispatch knowledge graph are queried using the Cypher language "MATCH(c) RETURN c", where c represents an entity;

[0081] Secondly, all entities in the power dispatch knowledge graph are saved by line to construct a knowledge graph named entity dictionary, such as "transformer\n main transformer\n potential\n direct current resistance\n continuous spark discharge\n", etc., where "\n" represents a line feed character;

[0082] Thirdly, all entities are input into the trained PowerBERT model in turn to obtain entity vectors X' i =(x'1,x'2,…,x' n ), where n represents the maximum number of characters of an entity, x j ' is a 768-dimensional vector of the jth character, j=1,2,3...n;

[0083] Finally, the entity vector X' i is saved as a set to obtain the entity vector representation set V dict ={X'1,X'2,…,X' m}, where m represents the number of knowledge graph entities.

[0084] Input 2: The entity identified in the question analysis (i.e., the question entity), denoted as Entities. In the specific application process, each question entity can also be vectorized in the PowerBERT model to obtain the input entity word vector representation set V entity .

[0085] Further, taking the question "How to deal with the continuous spark discharge of the two 1 million unit generators in Pingwei Phase 3?" as an example:

[0086] First, after the user question data is processed by step 102, the question entity can be obtained as Entities={“Pingwei Phase 3”, “continuous spark discharge”};

[0087] Secondly, the question entity Entities={“Pingwei Phase 3”, “continuous spark discharge”} is input into the trained PowerBERT model to obtain entity word vectors, denoted as X'1={x' 11 -x' 14} and X'2={x'21 x' 26}。

[0088] Finally, the entity word vector X'1 and the entity word vector X'2 are put into the set V entity ={X'1,X'2}.

[0089] 2. Similarity calculation:

[0090] The similarity between all entities in the power dispatch knowledge graph and the query entity is calculated, and the cosine similarity is used to calculate the similarity between entities, that is:

[0091]

[0092] In the formula, · represents the dot product between vectors, and ||·|| represents the norm of the vector.

[0093] Wherein, the similarity value is [-1, 1], for example, there is a query entity "Pingwei Third Phase", and the knowledge graph entities "Pingwei Second Phase" and "Anxiang Third Phase". The calculation of the similarity can obtain that the similarity between "Pingwei Third Phase" and "Pingwei Second Phase" is 0.9, and the similarity between "Pingwei Third Phase" and "Anxiang Third Phase" is 0.5.

[0094] Step 104: Expansion of user query data based on similarity.

[0095] In actual application process, according to the calculated similarity, the entity with the maximum similarity can be selected, that is, the entity in the power dispatch knowledge graph is most similar to the query entity. Based on the knowledge graph entity, the Cypher language "MATCH (c:`z`)-[r]-(m) RETURN r,m LIMIT 1" is used to query the triples related to the entity from the power dispatch knowledge graph, wherein c is a placeholder, z and m are entities in the power dispatch knowledge graph, and r is a relationship in the power dispatch knowledge graph. The extracted triples (z, r, m) set is recorded as Triples. Wherein, the relationship is a way to describe the connection, contact or association between entities. The relationship represents the semantic association between entities and provides information about the relationship or connection between entities.

[0096] The most relevant triples in the set Triples are added one by one to the original question to complete the expansion of the question and increase the semantic information of the question. When the cosine similarity is used to calculate the similarity between the query entity and the knowledge graph entity, the greater the value of the similarity represents the greater the relevance.

[0097] Based on the above description, the specific expansion process of the user query data is as follows:

[0098] Assume the question entity in the user's question data is Q, and the expanded entity is Q. extended The expanded entity Q extended It can be obtained through the following formula:

[0099] Q exten6ed ={Q}+(argmax) (s,p,o)∈Trples similarity(V entry V o )).

[0100] Where (s,p,o) represents entity 1, relation, and entity 2 in the triple, and V entry V represents the word vector representation of the entity entry in the user's query. o This represents the word vector representation of entity o in a knowledge graph.

[0101] For example, taking the question "How should continuous spark discharge of the two 1 million kW generating units in Pingwei Phase III be handled?" as an example, the following is an expanded explanation:

[0102] (1) Input “How to handle the continuous spark discharge of the two 1 million kilowatt units in Pingwei Phase III” into the PowerBERT+CRF model to obtain the entity set Entities = {“Pingwei Phase III”, “Continuous spark discharge”}.

[0103] (2) Input the entity set Entities = {"Pingwei Phase III", "Continuous Spark Discharge"} into the trained PowerBERT model to obtain the vector representation V of the entity set. entity = {X'1, X'2}, with a vector dimension of 768.

[0104] (3) Calculate all entities V in the power dispatch knowledge graph. dict With the questioning entity V entity By comparing the similarity between entities, the entity with the highest similarity is selected. The results show that the entities “faulty equipment” and “Pingwei Phase III” in the power dispatch knowledge graph are most similar, as are the entities “continuous discharge” and “continuous spark discharge” in the power dispatch knowledge graph.

[0105] (4) Using the Cypher language, “MATCH(c:`faulty equipment`)-[r]-(m)RETURN r,m LIMIT 1” and “MATCH(c:`continuous discharge`)-[r]-(m)RETURN r,mLIMIT 1” respectively, query the power dispatch knowledge graph for the associated triples to obtain the triple set:

[0106] Triples = {(Pingwei Phase III, including transformers)}

[0107] (continuous discharge, corresponding, abnormal).

[0108] (5) Add the most relevant triples in the triples set to the user query data one by one, and expand the query, for example, "How to deal with the continuous spark discharge of two 100 million unit groups in Pingwei Third Phase 2?" can be expanded to "{Pingwei Third Phase} (Pingwei Third Phase, contains, transformer) 2 sets of 100 million unit groups {continuous spark discharge} (continuous discharge, corresponding, abnormal) How to deal with?". Through the above method, the user query is expanded, the semantic representation ability of the user query is enhanced, and the query rate is improved.

[0109] Further, in order to improve the accuracy of word vector extraction, in the present application, the training process of the PowerBERT model includes:

[0110] Step 1, obtaining a data set. Wherein, before obtaining the data set, the data set needs to be collected and prepared. The present application collects and collates a series of power regulations, power plant duty system and power dispatching plan and other related texts as the data set.

[0111] Step 2, pre-processing the data in the data set. Specifically, after data cleaning and preprocessing, including removing noise and converting Chinese numerals contained in the device name into Arabic numerals, the text data is obtained.

[0112] Among them, using regular expressions and string processing methods, special characters, HTML tags and non-ASCII characters and other noise in the data set are removed.

[0113] The Chinese numerals in the device name are converted into Arabic numerals, for example, if the device name in the text data contains Chinese numerals (such as one, two, three, etc.), the dictionary is used for numeral conversion, and the corresponding Arabic numerals (1, 2, 3, etc.) are converted.

[0114] Step 3, pre-processing the data in the data set, and constructing an index dictionary of the vocabulary table, specifically:

[0115] Use the jieba word segmentation tool to segment all the data in the data set to divide the text into word form.

[0116] Traverse all the segmented words and record the frequency of each word. Here, the dictionary is used for word frequency statistics.

[0117] According to the word frequency, the words are sorted, and the words with higher frequency are placed in front.

[0118] According to the task requirements, some special marks are added in the vocabulary table, for example, the filler ( <pad>), unknown word symbol ( <unk>), end of string (EOS) <start>), end of file ( <end>), etc.

[0119] Each word is mapped to a unique index value, and a dictionary data structure is used to create an index dictionary for the vocabulary.

[0120] Step 4, generate input samples based on the index dictionary of the vocabulary, specifically:

[0121] Add special markers at the beginning and end of the tokenized sequence, where [CLS] represents the beginning of the sequence and [SEP] represents the end of the sequence.

[0122] According to the input requirements of the PowerBERT model, limit the length of the sequence, for example, choose a fixed length of 768.

[0123] Convert each word to its corresponding tokenized ID through the index dictionary to generate a tokenized ID sequence.

[0124] Add position encoding to each tokenized ID to represent the position information of the word in the sequence. Here we use the sine function, for the pos-th word, with position index i and hidden unit dimension d, the position vector can be calculated as

[0125] According to the input requirements of the PowerBERT model, construct the input sample. The constructed input sample usually includes tokenized ID sequence, position encoding, etc.

[0126] Step 5, generate input tensor based on input sample, specifically:

[0127] Combine the tokenized ID sequence, segment ID sequence and position vector sequence of each input sample into an input tensor. These input tensors will be used as input in the training process.

[0128] Where the segment ID sequence is represented by a fixed value of all 0 or all 1. Convert the segment ID sequence of each sample into a one-dimensional tensor.

[0129] Convert the position vector sequence of each sample into a two-dimensional tensor. Each row represents the position vector of a sample.

[0130] Horizontally concatenate (axis = 1) the tokenized ID sequence, segment ID sequence and position vector sequence in the last dimension to form the final input tensor.

[0131] Step 6, train the BERT model using the input tensor through backpropagation and optimization algorithm to get the trained PowerBERT model. The loss function of PowerBERT model adopts cross-entropy loss function. The optimizer used in the optimization algorithm is Adam optimizer.

[0132] Before step 6, the loss function and the optimizer are also defined. For example, cross-entropy is selected as the loss function to measure the difference between the predicted results of the model and the actual labels, and the Adam optimizer is selected to update the weight parameters of the model. The Adam optimizer is an adaptive learning rate optimization algorithm that can automatically adjust the learning rate according to the gradient of the parameters.

[0133] wherein, for a sample, the calculation formula of the cross-entropy loss function L is:

[0134] L = -sum(y true *log(y pred )).

[0135] wherein, y true represents the one-hot vector of the real label, y pred represents the probability distribution vector output by the model, and log represents the natural logarithm function. Sum represents the summation operation on all categories.

[0136] The smaller the cross-entropy loss function, the smaller the difference between the predicted results of the model and the actual labels, and the better the performance of the model.

[0137] In the actual training process, the loss value is calculated according to the predicted results of the model and the actual labels. According to the loss value, the gradient is calculated by the back propagation algorithm. The back propagation algorithm starts from the loss function and calculates the gradient of each parameter to the loss function according to the chain rule. In this way, the gradient of each parameter with respect to the loss, i.e. the degree of influence of the change of the parameter on the loss, can be obtained. Finally, the optimizer updates the weight parameters of the model according to the calculated gradient. The optimizer will update the weights according to the specified learning rate and other parameters to minimize the value of the loss function.

[0138] As can be seen, the training process of the PowerBERT model mainly uses the prepared input tensor and the defined loss function to complete through the back propagation and optimization algorithm.

[0139] Based on the above description, in order to obtain the key information in the question, the user's question needs to be analyzed, and the named entity in the question often contains the key semantic information of the question. Therefore, the PowerBERT+CRF model is used as the network architecture to perform named entity recognition on the user's question to obtain the named entity in the question and obtain the key information in the question.

[0140] Compared with the prior art, the present application also has the following advantages:

[0141] 1) The PowerBERT model is obtained by fine-tuning the BERT model with the collected text corpus of the power system, which can improve the accuracy of the query result;

[0142] 2) The application performs entity recognition on the user query sentence. The PowerBERT model is used to encode the entities of the power dispatch knowledge graph and the user query entities to obtain word vector embedding representation, which improves the accuracy of the entity semantic representation to eliminate the ambiguity of the question sentence;

[0143] 3) The application selects the knowledge graph entities with high similarity by calculating the similarity between the user query entities and the knowledge graph entities, queries the knowledge graph to obtain triples, and expands the user query entities using the triples, thereby eliminating the ambiguity of the sentence, expanding the scope of the query, and improving the recall rate of the query.

[0144] Further, the application also provides the following implementation structure:

[0145] A user question expansion system applied to the user question expansion method provided above. The system comprises:

[0146] A data acquisition module for acquiring user question data and preprocessing the user question data.

[0147] A word vector extraction module for inputting the preprocessed user question data into the trained power dispatch pre-training language model to obtain word vectors. The power dispatch pre-training language model is a trained PowerBERT model. The PowerBERT model is a BERT model that meets the power dispatch requirements.

[0148] An entity extraction module for inputting the word vectors and label sequences into a trained conditional random field model to identify the question entities. The question entities are used to represent the text content associated with the power dispatch in the user question data.

[0149] A similarity determination module for determining the similarity between the question entities and each entity in the power dispatch knowledge graph.

[0150] A data expansion module for expanding the user question data based on the similarity.

[0151] An electronic device comprising:

[0152] A memory for storing a computer program.

[0153] A processor connected to the memory for retrieving and executing the computer program to implement the user question expansion method provided above.

[0154] In addition, the computer program in the memory described above is realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various program code storage media.

[0155] The present application adopts the PowerBERT architecture-based calculation of word embedding representation, performs word embedding vector representation on the entities in the knowledge graph entity and the user query statement, and obtains the relationship entity in the selected triple by calculating the similarity between the query statement entity vector and the entity word vector of the knowledge graph. The query statement of the user is expanded by using the triple, submitted to the power dispatch query system for retrieval to obtain the treatment measures, and the accuracy and recall rate of the query are improved, and the returned information is more accurate and comprehensive.

[0156] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0157] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above embodiment description is only used to help understand the method of the present application and its core idea. For those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.< / end> < / start> < / unk> < / pad>

Claims

1. A method for expanding user-generated questions, characterized in that, include: Acquire user question data and preprocess it; The preprocessed user query data is input into a pre-trained power dispatching language model to obtain word vectors; the pre-trained power dispatching language model is a pre-trained PowerBERT model; the PowerBERT model is a BERT model that conforms to power dispatching. The word vectors are input into a trained conditional random field model to identify the question entity; the question entity is used to represent the text content in the user question data that is related to power dispatch. Determine the similarity between the question entity and each entity in the power dispatch knowledge graph; Expand user question data based on the aforementioned similarity; The training process of the PowerBERT model includes: Obtain the dataset; the dataset includes power regulations, power plant duty rosters, and power dispatch plans. The data in the dataset is preprocessed, including: segmenting the preprocessed data in the dataset using the jieba word segmenter; traversing all segmented words and recording the frequency of each word; sorting the words according to their frequency to obtain a vocabulary list, adding tags to the vocabulary list, and associating each word with a unique index value to generate an index dictionary for the vocabulary list; the added tags include: padding characters, unknown word characters, start characters, and end characters; The preprocessed data in the dataset is segmented into words, and an index dictionary of the vocabulary is constructed. Based on the input requirements of the PowerBERT model, the length of the word segmentation sequence in the vocabulary is limited; Each word is converted into a tokenized ID corresponding to itself using the index dictionary of the vocabulary, thereby generating a tokenized ID sequence; A positional encoding is added to each tokenized ID; the positional encoding is used to represent the positional information of the word in the word segmentation sequence; wherein, using a sine function, for the pos-th word, with position index i and hidden unit dimension d, the position vector is calculated as: Based on the input requirements of the PowerBERT model, input samples are constructed; the input samples include tokenized ID sequences and positional codes; An input tensor is generated based on the input samples; the input tensor is formed by combining the tokenized ID sequence, segment ID sequence, and position vector sequence; wherein, the segment ID sequence is represented by fixed values ​​of all 0s or all 1s; the segment ID sequence of each sample is converted into a one-dimensional tensor; the position vector sequence of each sample is converted into a two-dimensional tensor; each row represents the position vector of a sample; the tokenized ID sequence, segment ID sequence, and position vector sequence are horizontally concatenated in the last dimension to form the final input tensor; The BERT model is trained using the input tensor through backpropagation and optimization algorithms to obtain the trained PowerBERT model.

2. The user question expansion method according to claim 1, characterized in that, The PowerBERT model uses the cross-entropy loss function; the optimization algorithm uses the Adam optimizer.

3. The user question expansion method according to claim 1, characterized in that, The preprocessing of the data in the dataset specifically includes: Noise in the dataset is removed using regular expressions and string processing methods; the noise includes special characters, HTML tags, and non-ASCII characters. Convert the Chinese numerals of the device names in the noise-removed data of the dataset into Arabic numerals.

4. The user question expansion method according to claim 1, characterized in that, The data in the training dataset and test dataset used during the training of the power dispatch pre-trained language model and the conditional random field model are labeled using the BIO annotation method.

5. The user question expansion method according to claim 1, characterized in that, Expanding user question data based on the aforementioned similarity specifically includes: Select the entity with the highest similarity in the power dispatch knowledge graph, and based on this selected entity, use the Cypher language to query the triples related to this entity from the power dispatch knowledge graph; The triples are added to the user question data to complete the expansion of the user question data.

6. A user-generated question extension system, characterized in that, The system is applied to the user question expansion method as described in any one of claims 1-5; the system includes: The data acquisition module is used to acquire user question data and preprocess the user question data; The word vector extraction module is used to input the preprocessed user query data into the trained power dispatch pre-trained language model to obtain word vectors; the power dispatch pre-trained language model is a trained PowerBERT model; the PowerBERT model is a BERT model that conforms to power dispatch. The entity extraction module is used to input the word vector into a trained conditional random field model to identify the question entity; the question entity is used to represent the text content in the user question data that is related to power dispatch. A similarity determination module is used to determine the similarity between the question entity and each entity in the power dispatch knowledge graph; The data expansion module is used to expand the user question data based on the similarity.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, connected to the memory, is configured to retrieve and execute the computer program to implement the user question extension method as described in any one of claims 1-5.

Images