Semantic matching method and system for knowledge retrieval and question answering of power transformers

Abstract

The present invention discloses a semantic matching method and system for knowledge retrieval and question-answering of power transformers, and belongs to the technical field of power transformer equipment. It is difficult for existing semantic matching methods to accurately identify the retrieval / question-answering semantics of grassroots operation and maintenance personnel, which affects the operation and maintenance efficiency of grassroots operation and maintenance personnel. The semantic matching method for knowledge retrieval and question-answering of power transformers of the present invention realizes the enhancement of transformer corpus text and data enhancement by constructing a corpus enhancement model based on a small sample transformer question template, and combining the transformer knowledge graph and synonym library; then, the transformer corpus data and artificial neural network technology are used to train and form an integrated recognition model; then, the integrated recognition model is used to realize the joint recognition of user query intention and slot on the basis of the corpus, thereby accurately identifying the user's retrieval / question-answering semantics and effectively improving the user's operation and maintenance efficiency.

Description

Technical Field

[0001] The invention relates to a semantic matching method and system for power transformer knowledge retrieval and question answering, and belongs to the technical field of power transformer equipment. Background Art

[0002] There is a large amount of knowledge in equipment operation and maintenance, such as equipment maintenance standards, guidelines, equipment failure cases, etc. This knowledge mainly serves the actual business of the grassroots power industry. From the perspective of basic operations such as grassroots inspections, troubleshooting, testing, and maintenance, this power knowledge is frequently encountered and of key concern to grassroots operation and maintenance personnel in business execution. At present, the application of this common knowledge is mainly based on keyword retrieval, and knowledge applications such as knowledge-based search and question-and-answer are still in the exploratory stage.

[0003] Furthermore, power companies have built a vertical knowledge graph around transformer equipment. The graph contains various knowledge points related to the transformer, including transformer inventory, transformer faults, detection standards, etc. For example, #1 main transformer is an entity knowledge. The entity has various attribute values, such as voltage level, and also has some relationships, such as the power station to which it belongs, and the relationship points to a substation.

[0004] When carrying out daily equipment operation and maintenance inspections, the grassroots operation and maintenance personnel of power companies need to flexibly search for the above knowledge points. The search forms include search engines and question-answering robots. However, they face the following challenges in the search / question-answering process, including:

[0005] 1) The query format is not standardized, with colloquialisms, abbreviations, and abbreviations.

[0006] When front-line employees use search and question-answering robots, the input format is characterized by colloquialism, a large number of abbreviations, and similar names. For example, when searching for transformer No. 1 of a substation, the input format includes transformer No. 1 of a substation, main transformer No. 1 of a station, transformer No. 1 of a power station, transformer No. 1 of a substation, and abbreviations such as "Tumen 220kV substation" and "Tumen Station".

[0007] 2) There are a large number of approximate questions

[0008] For example, there are many similar ways of asking questions about the manufacturer of a certain transformer, such as "the manufacturer of transformer No. 1 of a certain substation", "where is transformer No. 1 of a certain substation produced?", "which factory produces transformer No. 1 of a certain substation?", "which factory manufactured transformer No. 1 of a certain substation", etc.

[0009] 3) Transformer question-answering corpus is relatively small

[0010] The question and answer on power transformers belongs to a very niche vertical field. There is relatively little public corpus data in this field, and it belongs to a small sample data space.

[0011] Due to the above-mentioned problems, the existing semantic matching methods for power transformer knowledge retrieval and question-answering are difficult to accurately identify the retrieval / question-answering semantics of grassroots operation and maintenance personnel, which affects the operation and maintenance efficiency of grassroots operation and maintenance personnel. Summary of the invention

[0012] In view of the defects of the prior art, the purpose of the present invention is to provide a corpus enhancement model based on a small sample transformer question template, and combine the transformer knowledge graph and the synonym library to achieve transformer corpus text enhancement and data enhancement; use transformer corpus data and artificial neural network technology to train and form an integrated recognition model; then use the integrated recognition model to realize the joint recognition of user query intention and slot on the basis of the corpus, and then accurately identify the user's retrieval / question and answer semantics, effectively improve the user's operation and inspection efficiency, and provide a semantic matching method and system for power transformer knowledge retrieval and question and answer.

[0013] To achieve the above object, the technical solution of the present invention is:

[0014] A semantic matching method for knowledge retrieval and question answering of power transformers.

[0015] The first step is to build a transformer-based knowledge graph KG;

[0016] In the second step, based on the knowledge graph KG and synonym library in the first step, and using the small sample transformer question template, a corpus enhancement model is established to quickly construct transformer corpus data;

[0017] The third step is to use artificial neural network technology to train the transformer corpus data in the second step to form an integrated recognition model;

[0018] The integrated recognition model uses the bidirectional long short-term memory network BILSTM and the attention mechanism Attention to judge the transformer question-answering intention and effectively identify the slot of the input sentence according to the question-answering intention;

[0019] The identified slot information is linked with the transformer of the knowledge graph KG in the first step to form an effective recognition system;

[0020] The fourth step is to use the integrated recognition model in the third step to effectively match the semantics of the search and / or question-answer statements input by the user.

[0021] After continuous exploration and experimentation, the present invention constructs a corpus enhancement model based on a small sample transformer question template, and combines the transformer knowledge graph and the synonym library to achieve transformer corpus text enhancement and data enhancement; then uses the transformer corpus data and artificial neural network technology to train and form an integrated recognition model; then the integrated recognition model is used to realize the joint recognition of user query intention and slot on the basis of the corpus, thereby accurately identifying the user's retrieval / question and answer semantics, effectively improving the user's operation and inspection efficiency, and is particularly suitable for application in power transformer knowledge retrieval and question and answer.

[0022] As the preferred technical measures:

[0023] The knowledge graph in the first step is a graph-based data structure, which uses triple RDF to describe the attribute values ​​of transformers and the relationship between transformers, and includes several nodes (Point) and edges (Edge);

[0024] The node is used to represent various attribute values ​​of a transformer;

[0025] The edge is used to represent the relationship between transformers;

[0026] The attribute values ​​include transformer inventory, transformer faults, and detection standards.

[0027] As the preferred technical measures:

[0028] The small sample transformer question template in the second step is a slot-based question template, which is used to express the semantics of user input, and includes the sentence input by the user and the semantic annotation of the sentence;

[0029] The method of constructing the corpus enhancement model is as follows:

[0030] We use deep learning methods to train question-and-answer related data, perform small sample data enhancement, and increase the amount of corpus data in small samples by using a large number of professional vocabularies that enhance the universality of the data set.

[0031] As the preferred technical measures:

[0032] The attention mechanism in the third step quickly scans the semantics of the search and / or question-answer sentences input by the user to obtain the target area that needs to be focused on, i.e., the focus of attention.

[0033] Then, more attention resources are invested in the focus of attention to obtain more detailed information of the target that needs attention and suppress other useless information;

[0034] Based on the attention mechanism, the performance of the Encoder+Decoder model based on RNN (Long Short-Term Neural Network LSTM or GRU) can be improved.

[0035] An integrated recognition model, including a question input layer, a semantic analysis and matching layer, a query retrieval layer, and an answer display layer;

[0036] The output of the previous layer is used as the input of the next layer;

[0037] The question input layer is used to receive user input and segment the user input statement to extract statement vocabulary;

[0038] For the problems of ambiguity, colloquialism, abbreviation, various synonymous expressions for the same question, and multiple interactive inputs in user input; use the Chinese word segmenter jieba to segment the input statement, and the word segmentation strategy is the longest vocabulary strategy, and stop words and meaningless words are removed;

[0039] There are problems such as ambiguity, colloquialism, abbreviation, various synonymous expressions for the same question, and multiple interactive inputs in user input; for example, when the user wants to query "Where is the manufacturer of the No. 1 main transformer of Changchun Station?", this question may also be input as "Which manufacturer produces the No. 1 main transformer of the 220KV substation in Changchun", "Which factory manufactures the No. 1 transformer of Changchun Station", etc. synonymous questions;

[0040] The present invention segments the input statement based on the Chinese word segmenter jieba, and the word segmentation strategy is the longest vocabulary strategy, and stop words and meaningless words are removed. For example, in the sentence "Which factory manufactures the No. 1 transformer of Changchun Station", meaningless words such as "of" need to be removed;

[0041] The semantic analysis and matching layer is used to perform semantic analysis on the statement transmitted by the question input layer;

[0042] Using a bidirectional long short-term memory network BILSTM and an attention mechanism Attention to analyze and match the semantics of the input statement; it specifically includes the following steps:

[0043] S1, represent the vocabulary formed by the word segmenter with vectors, and calculate the word vectors of each vocabulary based on the pre-trained model BERT;

[0044] S2, encode the features of the word vectors in S1 through a bidirectional long short-term memory network BILSTM and an attention mechanism Attention;

[0045] S3, input the encoded word vectors in S2 into a semantic intent classifier and a semantic slot classifier, and through a transformer link operation, output the standard transformer name or / and attribute name or / and relationship name information of the query;

[0046] The query retrieval layer is used to build a query subgraph based on the concepts, transformers, and attributes passed by the intent recognition and slot extraction layer; and convert the query into a search engine (Elasticsearch, ES) query statement, returning potential triples or answers Q / A;

[0047] Taking the above example of "Which factory manufactured the No. 1 main transformer of Changchun Station" as an example, after identification by the intent recognition model and the slot recognition model; based on "Changchun 220KV substation-No. 1 main transformer-manufacturer", first follow the following ES query statement based on "Changchun 220kV substation-No. 1 main transformer" to obtain detailed information of No. 1 main transformer;

[0048] The answer display layer is used to display different answers. The answer types are divided into three categories: collection type query, transformer type query, and transformer attribute query.

[0049] As the preferred technical measures:

[0050] The semantic analysis and matching layer includes a word vector layer, a bidirectional long short-term neural network BILSTM encoding layer, an attention layer, and an intention and slot joint recognition layer;

[0051] The word vector layer is a vector representing vocabulary based on the pre-trained model BERT, which is used to convert text into a computable word vector model;

[0052] Through the word vector model, each word is converted from a one-dimensional space to a multi-dimensional space with a continuous vector of embedded numbers;

[0053] Preferably, the pre-trained model BERT is used to calculate the vocabulary input by the user to calculate the vector representation of the vocabulary, and the dimension of the calculated word vector is 204;

[0054] The bidirectional long short-term memory network BILSTM encoding layer is composed of two layers of long short-term neural network LSTM with different directions;

[0055] Each layer of the long short-term neural network LSTM includes a group of artificial neural network units with a chain structure;

[0056] The attention layer is used to determine the key features of the user input, that is, to calculate the local feature vector;

[0057] The intent and slot joint recognition layer is used to decode and classify the word vector features calculated based on attention. It includes the intent classification model and the slot classification model.

[0058] The number of LSTM network units used in each layer is 300.

[0059] As the preferred technical measures:

[0060] The long short-term neural network LSTM includes at least one cell state component, a forget gate, a memory gate, and an output gate. Through the interaction of the cell state component, the forget gate, the memory gate, and the output gate, long-distance information processing of sequence data is realized;

[0061] The cell state component is a data link, which is used to carry information and enable the information to be transmitted unchanged;

[0062] The forget gate is passed through the forgetting factor f t Decide what information to forget or discard from the cell state;

[0063] Forgetting factor f t The calculation formula is as follows:

[0064] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0065] In the above formula, W is the weight matrix, which is calculated through subsequent model training. σ is an activation function, namely the ogistic function, which is used for the output of hidden layer neurons and has a value range of (0,1). It maps a real number to the interval of (0,1) for binary classification.

[0066] The forgetting factor is h t-1 and x t As input, it is calculated through the σ neural network and outputs a number between 0 and 1, where 1 means "completely retain the information" and 0 means "completely discard the information";

[0067] Cell state C t-1 Each element in C will be judged by the forget gate to determine whether to keep or forget C t-1 Which part of the unit module C? For example, in a natural language model, if you want to predict the next word from a word, t-1 If a certain attribute contained in is not needed in the next module unit, this attribute data can be discarded through the forget gate;

[0068] The memory gate is used to decide what new information will be stored in the cell state;

[0069] The memory gate calculation method is as follows:

[0070] First, a σ neural network layer is used to decide which information needs to be updated, using i t To express;

[0071] it The calculation formula is as follows:

[0072] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0073] Among them, W i It is the weight matrix, which can be calculated through subsequent data training;

[0074] Secondly, a new candidate value is created at the tanh layer, where tanh is one of the hyperbolic functions;

[0075] tanh() is the hyperbolic tangent. The function passes through the origin and the range of the generated values ​​is (-1,1).

[0076] The value produced by the tanh layer is added to the cell state, and the new value is expressed as To represent, the input is h t-1 and x t , the calculation formula is as follows:

[0077]

[0078] Among them, W c is the weight matrix, which can be obtained through training, b c is the linear offset.

[0079] Finally, in the old cell state C t-1 Based on this, update its value to get the new C t The value of

[0080] C t The calculation formula is as follows:

[0081]

[0082] The output gate is used to add some filtering based on the current cell state and output the desired information;

[0083] The calculation formula of the output gate is as follows:

[0084] First, we establish a gate of the σ neural network layer to determine which parts of the output cell will be output. The output information is represented by o t Indicates that t The calculation formula is as follows:

[0085] o t =σ(W o ·[h t-1 ,x t ]+bo )

[0086] Among them, W o is the weight matrix, calculated through subsequent training, b o is the linear offset.

[0087] Then the cell state is activated by the function tanh and then t Multiply, and then output the part you want to output h t ,h t The calculation formula is as follows:

[0088] h t =o t ×tanh(C t )

[0089] After being encoded by the bidirectional long short-term memory network BILSTM encoding layer, the full features of the input sentence are encoded and the global feature vector is output;

[0090] The global features include the text value, part of speech of the current word, the text value, part of speech, position of the first 1-50 words, and the text value, part of speech, and position features of the next 50 words;

[0091] Then, after the global feature vector is input, the subsequent attention layer extracts local key features.

[0092] As the preferred technical measures:

[0093] The local feature vector of the attention layer is C t , which is calculated as follows:

[0094]

[0095]

[0096] e ij =v a T tanh(w a C t-1 +u a h j );

[0097] Where T is the number of vector sequences output by the bidirectional long short-term memory network BILSTM layer, a tj is the weight given by the attention layer to all feature vectors, e ij is the associated energy of input and output, v a is the global eigenvector, c t-1 is the state vector of the attention model at the previous moment, h jis the feature vector sequence output by the bidirectional long short-term memory network BILSTM layer, u a is the weight of the eigenvector at the previous moment, w a is the weight of the attention mechanism at the previous moment.

[0098] For example, the query sentences "Where is the manufacturer of Changchun Station No. 1 main transformer?", "Which manufacturer is Changchun 220KV substation No. 1 main transformer?", "Which factory manufactured Changchun Station No. 1 transformer", although the vocabulary after word segmentation is different, the key points of the three sentences are "manufacturer" and "transformer". This key point is the so-called "attention". The role of the attention layer is to form a local feature vector.

[0099] As the preferred technical measures:

[0100] The two classification models of the intent and slot joint recognition layer are generated based on the Softmax classifier;

[0101] The calculation formula of the Softmax classifier is as follows:

[0102]

[0103] where z i is the output value of the i-th node, C is the number of output nodes, that is, the number of classification categories;

[0104] The intent classification model is implemented based on the softmax classifier and is used to determine the classification of user intent. It includes a slot classification module and an entity linking module.

[0105] Slot classification module, used to express the basic semantic position of natural language sentences;

[0106] Slot is a basic concept in NLP. Based on the idea of ​​slot, the present invention designs a slot-based question template to express the semantics of user input; for example, "Which manufacturer manufactured the #1 main transformer?" The slots that can be extracted from this sentence include [manufacturer], [manufacturing], and [equipment type]. The present invention completes the fitting of input data to output classification based on the Softmax classifier through the output of the bidirectional long short-term memory network BILSTM network layer;

[0107] The entity linking module is used to find the slots in the user input through the slot classification model, extract the slot information, and align the slot information with the transformer name in the atlas library through the text similarity model, that is, transformer linking.

[0108] For example, "Changchun Station" is aligned to "Changchun 220kV Substation", and "No. 1 Transformer" is aligned to "No. 1 Main Transformer". The present invention adopts a text similarity model based on cosine similarity to solve the transformer link problem.

[0109] The text similarity model is a text similarity model based on cosine similarity;

[0110] Cosine similarity, also known as cosine distance, uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals; the closer the cosine value is to 1, the closer the angle is to 0 degrees, indicating that the two vectors are more similar, that is, "cosine similarity";

[0111] The formula for calculating the cosine similarity of the vector angle is as follows:

[0112]

[0113] Among them, A and B are the sentence vectors of short texts respectively;

[0114] A and B are n-dimensional vectors, respectively. i is the i-th value of vector A, B i is the i-th value of the B vector;

[0115] The cosine similarity value range is [0, 1], and the larger the value, the higher the similarity.

[0116] The present invention represents the sentence vectors of two short text sentences A and B based on the language model BERT, and the dimension n of the vector is 202 dimensions. The cosine similarity of the sentence vectors of the two short texts A and B is calculated to determine the similarity between the two.

[0117] As the preferred technical measures:

[0118] The answer display layer adopts different display strategies for different categories;

[0119] 1) Collection type, whose type data includes numbers and collections;

[0120] The query on which collection type the user asked;

[0121] For example, "Which transformers are there in Chitouqing 220kV substation?" needs to return the number of sets and the specific number of sets;

[0122] 2) Transformer type, whose value is a specific transformer, showing the detailed properties of the transformer;

[0123] For example, if you input "Who is the manufacturer of the main transformer of Chitouqing 220kV substation #1?", the specific transformer "Daur Yellow Rat Co., Ltd." will be displayed, along with detailed information of the transformer;

[0124] 3) Transformer attribute value type, whose value is the name of a specific transformer or the attribute value of the transformer;

[0125] For example, if a user asks "Chitouqing 220kV substation #1 main transformer voltage level", the returned value is the value of the voltage level attribute of the #1 main transformer.

[0126] As the preferred technical measures:

[0127] A semantic matching system for knowledge retrieval and question answering of power transformers, comprising:

[0128] one or more processors;

[0129] A storage device for storing one or more programs;

[0130] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned semantic matching method for power transformer knowledge retrieval and question answering.

[0131] The corpus enhancement method based on a small sample transformer question template of the present invention can quickly construct transformer corpus data, and applies artificial neural network technology to the corpus data to jointly identify semantic intent and semantic slots of user input sentences and link entities.

[0132] The scheme of the present invention is detailed, feasible, easy to apply, easy to implement, has important engineering practice significance, can accurately identify the retrieval / question-answering semantics of grassroots operation and maintenance personnel, and effectively improve the retrieval / question-answering efficiency and operation and maintenance efficiency of grassroots operation and maintenance personnel.

[0133] Compared with the prior art, the present invention has the following beneficial effects:

[0134] After continuous exploration and experimentation, the present invention constructs a corpus enhancement model based on a small sample transformer question template, and combines the transformer knowledge graph and the synonym library to achieve transformer corpus text enhancement and data enhancement; then uses the transformer corpus data and artificial neural network technology to train and form an integrated recognition model; then the integrated recognition model is used to realize the joint recognition of user query intention and slot on the basis of the corpus, thereby accurately identifying the user's retrieval / question and answer semantics, effectively improving the user's operation and inspection efficiency, and is particularly suitable for application in power transformer knowledge retrieval and question and answer.

[0135] Furthermore, the scheme of the present invention is detailed, feasible, easy to apply, and easy to implement, and has important engineering practice significance, and can effectively improve the retrieval / question-and-answer efficiency and operation and inspection efficiency of grassroots operation and inspection personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0136] Figure 1 It is a diagram showing the semantic parsing and matching process of user input in the present invention;

[0137] Figure 2 This is a diagram showing the structure of a long short-term neural network LSTM network unit of the present invention;

[0138] Figure 3 This is a diagram showing the structure of the long short-term neural network LSTM of the present invention;

[0139] Figure 4 It is a diagram showing the cell state structure of the present invention;

[0140] Figure 5 It is a structural diagram of the memory gate of the present invention;

[0141] Figure 6 This is a diagram showing the structure of the output gate of the present invention;

[0142] Figure 7 This is a flowchart of obtaining the information of the No. 1 main transformer according to the ES query statement of the present invention;

[0143] Figure 8 This is a flowchart of obtaining manufacturer information according to an ES query statement in the present invention;

[0144] Fig. 9 It is a diagram showing the interaction process of the model of the present invention;

[0145] Fig.10 It is a structural diagram of the door assembly of the present invention. DETAILED DESCRIPTION

[0146] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0147] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0148] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0149] A semantic matching method for knowledge retrieval and question answering of power transformers.

[0150] The first step is to build a transformer-based knowledge graph KG;

[0151] In the second step, based on the knowledge graph KG and synonym library in the first step, and using the small sample transformer question template, a corpus enhancement model is established to quickly construct transformer corpus data;

[0152] The third step is to use artificial neural network technology to train the transformer corpus data in the second step to form an integrated recognition model;

[0153] The integrated recognition model uses the bidirectional long short-term memory network BILSTM and the attention mechanism Attention to judge the transformer question-answering intention and effectively identify the slot of the input sentence according to the question-answering intention;

[0154] The identified slot information is linked with the transformer of the knowledge graph KG in the first step to form an effective recognition system;

[0155] The fourth step is to use the integrated recognition model in the third step to effectively match the semantics of the search and / or question-answer statements input by the user.

[0156] After continuous exploration and experimentation, the present invention constructs a corpus enhancement model based on a small sample transformer question template, and combines the transformer knowledge graph and the synonym library to achieve transformer corpus text enhancement and data enhancement; then uses the transformer corpus data and artificial neural network technology to train and form an integrated recognition model; then the integrated recognition model is used to realize the joint recognition of user query intention and slot on the basis of the corpus, thereby accurately identifying the user's retrieval / question and answer semantics, effectively improving the user's operation and inspection efficiency, and is particularly suitable for application in power transformer knowledge retrieval and question and answer.

[0157] A preferred embodiment of the present invention:

[0158] Based on the transformer knowledge graph KG and synonym library, the present invention proposes a corpus enhancement method based on a small sample transformer question template, quickly constructs transformer corpus data, and applies artificial neural network technology to the corpus data to train a transformer question-answering intention and slot integrated recognition model based on the bidirectional long short-term memory network BILSTM+Attention mechanism. The model jointly recognizes the semantic intent and semantic slot of the user input sentence, judges the user's query intention, identifies the sentence slot, extracts the slot information, and links the extracted slot information with the entity in the graph library, thereby realizing semantic matching of the retrieval / question-answering sentence input by the user.

[0159] A specific embodiment of the present invention for enhancing corpus in the field of small sample of power transformer:

[0160] To address the problem of insufficient corpus data in the field of small samples of power transformers, the present invention proposes a corpus rapid enhancement mechanism based on knowledge graph KG+semantic template. Based on the idea of ​​slots, a slot-based question template is designed to express the semantics of user input.

[0161] The question template consists of the sentence input by the user and the semantic annotation of the sentence. For example, in the question templates "Which factory [ / C manufacturer] manufactures [ / R relationship] #1 main transformer [ / C transformer]", "What is the voltage [attribute, named "voltage level"] of #1 main transformer [ / equipment type]", the "[]" part is the semantic annotation part, which is divided into three categories:

[0162] Concepts begin with the letter C, followed by the specific concept name.

[0163] Attributes start with the letter P, followed by the specific attribute name.

[0164] Relationships begin with the letter R, followed by the specific relationship name.

[0165] Through such an annotation structure, the user's question template can be expressed in a semantic structure, and the position of the vocabulary corresponding to the semantic annotation part is the slot, that is, the meaning slot, and different vocabulary can be filled in the slot.

[0166] For example, "Which factory [ / C manufacturer] manufactured [ / R relationship] #1 main transformer [ / C equipment type]" can be generalized into the following types:

[0167] Which manufacturer manufactured the #1 main transformer?

[0168] #1Where is the main transformer manufactured?

[0169] #1Which manufacturer produces the main transformer?

[0170] The present invention combines the knowledge graph KG, fills the question template with the data of the knowledge graph KG, generates a large number of training sentences, constitutes the training set, and completes the training set preparation. It supports users to input question and answer intentions and keyword slots of intentions to complete the relevant training template configuration.

[0171] According to the configured training template, a large amount of corpus data is dynamically generated. Based on the generated training corpus, the training of the bidirectional long short-term memory network BILSTM model + attention model can be quickly carried out.

[0172] like Figure 1 As shown, a specific embodiment of the present invention constructs an integrated recognition model:

[0173] An integrated recognition model is used to parse and match the semantics of user input statements. The model can be divided into four layers from top to bottom, and the output of the upper layer is used as the input of the lower layer, including a question input layer, a semantic analysis and matching layer, a query retrieval layer, and an answer display layer.

[0174] 1) Question Input Layer

[0175] The question input layer is positioned to receive user input from the system interface and segment the user input statement to extract statement vocabulary. User input presents problems such as ambiguity, colloquialism, abbreviation, various near-synonymous expressions for the same question, and multiple interactive inputs. For example, when a user wants to query "Where is the manufacturer of the No. 1 main transformer at Changchun Station?", this question may also be input as "Which manufacturer produces the No. 1 main transformer of the 220kV substation in Changchun", "Which factory manufactures the No. 1 transformer at Changchun Station", etc., near-synonymous questions.

[0176] The present invention segments the input statement based on the jieba Chinese word segmenter. The word segmentation strategy is the longest vocabulary strategy, and stop words and meaningless words are removed. For example, in the sentence "Which factory manufactures the No. 1 transformer at Changchun Station", meaningless words such as "of" need to be removed.

[0177] 2) Semantic Analysis and Matching Layer

[0178] The semantic analysis and matching layer is positioned to perform semantic analysis on the statements passed from the question input layer. In this layer, the present invention proposes an integrated recognition model of intention and slot based on a bidirectional long short-term memory network BILSTM + attention to achieve semantic analysis and matching of input statements. First, vector representations are made for the vocabulary formed by the word segmenter. Based on a pre-trained model, the word vectors of each vocabulary are calculated, and the features of the vectors are encoded through a bidirectional long short-term memory network BILSTM + attention layer. The encoded vectors are sent to a semantic intention classifier and a semantic slot classifier, and through entity linking operations, information such as the standard entity name / attribute name / relationship name of the query is output. The entire processing process can be further subdivided into a word vector layer, a bidirectional long short-term neural network LSTM encoding layer, an attention layer, and an intention and slot joint recognition layer from top to bottom. Among them:

[0179] Word Vector Layer: Based on the pre-trained model BERT, vectors of vocabulary are represented, thereby converting text into a computable mathematical model. Through the word vector model, each word can be converted from a one-dimensional space to a mathematical embedding in a continuous vector space. In this paper, based on the pre-trained model BERT, the vocabulary input by the user is calculated, the vector representation of the vocabulary is calculated, and the dimension of the calculated word vector is 204.

[0180] Bidirectional long short-term neural network LSTM encoding layer: It is positioned to extract the features of word vectors and encode them to form new vectors. Bidirectional long short-term neural network LSTM is the superposition of two layers of long short-term memory networks with different directions. Each layer of long short-term neural network LSTM contains a group of long short-term neural network LSTM artificial neural network units with a chain structure. The number of long short-term neural network LSTM network units in each layer used in the present invention is 300. Figure 2 shown.

[0181] Among them, the long short-term neural network LSTM, as a typical structure of the RNN network, includes a cell state component, a forget gate, a memory gate, and an output gate. Through their interaction, long-distance information processing of sequence data is realized. The structure of the entire model is as follows Figure 3 shown.

[0182] The components are described as follows:

[0183] The cell state is a bit like a conveyor belt that goes straight through the entire chain with only a few minor linear interactions. The information carried on it can easily flow through without change, such as Figure 4 shown.

[0184] Forget gate, long short-term neural network LSTM through the forgetting factor f t Determines what information to forget / discard from the cell state, the forgetting factor f t The calculation formula is shown in the following formula, where W is the weight matrix, which can be calculated through subsequent model training.

[0185] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0186] In the above formula, σ is an activation function, also called Logistic function, which is used for hidden layer neuron output and has a value range of (0,1). It can map a real number to the interval of (0,1) and can be used for binary classification.

[0187] The forgetting factor is h t-1 and x t As input, the σ neural network is used for calculation and outputs a number between 0 and 1, where 1 means "totally retain the information" and 0 means "totally discard the information". The cell state C t-1 Each element in C will be judged by the forget gate to determine whether to keep or forget C t-1 which part of.

[0188] For example, in a natural language model, if we want to predict the next word from a word, the unit module C t-1 If an attribute contained in is not needed in the next module unit, this attribute data can be discarded through the forget gate.

[0189] Memory gate, this step is to decide what new information will be stored in the cell state. The structure of the memory gate is as follows Figure 5 shown.

[0190] The calculation of the memory gate is divided into three steps:

[0191] First, a σ neural network layer is used to decide which information needs to be updated, using i t To express. t The formula is defined as follows, where W is the weight matrix, which can be calculated through subsequent data training.

[0192] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0193] Secondly, a new candidate value is created in a tanh layer. Tanh is one of the hyperbolic functions. Tanh() is the hyperbolic tangent. The function passes through the origin and the range of the generated value is (-1,1). The value generated by the tanh layer may be added to the cell state. The new value is used To represent, the input is h t-1 and x t , the calculation formula is as follows:

[0194]

[0195] Finally, in the old cell state C t-1 Based on this, update its value to get the new C t The value of C t The calculation formula is as follows:

[0196]

[0197] The output gate, based on the current cell state, adds some filtering to output the desired information. The structure of the output gate is as follows Figure 6 shown.

[0198] First, we establish a gate of the σ neural network layer to determine which parts of the output cell will be output. The output information is represented by o t Indicates that t The calculation formula is as follows, where W is the weight matrix, which can be calculated through subsequent training.

[0199] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0200] Then the cell state is processed by the tanh activation function and then compared with o t Multiply them together, so that you can output the part you want to output h t ,h t The calculation formula is as follows:

[0201] h t =o t ×tanh(C t ).

[0202] After encoding by the bidirectional long short-term memory network BILSTM encoding layer, the full feature of the input sentence is encoded and the global feature vector is output. The global feature vector includes the text value, part of speech of the current word, the text value, part of speech, position of the first 1-50 words, and the text value, part of speech, position of the next 50 words. After the global feature vector is input, the subsequent attention layer extracts local key features.

[0203] Attention layer, the location of this layer is to determine the key features of the user input.

[0204] For example, the query sentences "Who is the manufacturer of Changchun Station No. 1 main transformer?", "Which manufacturer is Changchun 220kV substation No. 1 main transformer?", "Which factory manufactured Changchun Station No. 1 transformer", although the vocabulary after word segmentation is different, the key points of the three sentences are "manufacturer" and "transformer". This key point is the so-called "attention". The role of the attention layer is to form a local feature vector.

[0205] Define X 1 ,X 2 ,…,X T is the vector sequence output by the bidirectional long short-term memory network BILSTM layer, a tj The weight assigned by the attention layer to all feature vectors is calculated as follows:

[0206]

[0207] In the above formula, e ij is the associated energy of the input and output and is defined as follows:

[0208] e ij =v a T tanh(w a Ct-1 +u a h j )

[0209] Among them, v a is the global eigenvector, c t-1 is the state vector of the attention model at the previous moment, h i is the feature vector sequence output by the bidirectional long short-term memory network BILSTM layer, u a is the weight of the eigenvector at the previous moment, w a is the weight of the attention mechanism at the previous moment.

[0210] After processing by the attention layer, the final state of the output is C t , and its calculation formula is as follows:

[0211]

[0212] The intent and slot joint recognition layer is located based on the feature vector calculated by the attention layer, which is decoded and classified. It includes two classification models, the intent classification model and the slot classification model. Both classification models are based on the Softmax classifier.

[0213]

[0214] where z i is the output value of the ith node, and C is the number of output nodes, that is, the number of classification categories.

[0215] The intent classification model, implemented based on the softmax classifier, is used to determine the classification of user intent.

[0216] Slot classification model, slot is a basic concept in NLP, which is used to express the basic semantic position of natural language sentences. Based on the idea of ​​slot, the present invention designs a slot-based question template to express the semantics of user input.

[0217] For example, the sentence "Which manufacturer manufactured the #1 main transformer?" can extract slots including [manufacturer], [manufacture], and [equipment type]. The present invention completes the fitting of input data to output classification based on the Softmax classifier by the output of the bidirectional long short-term memory network BILSTM network layer.

[0218] Entity linking, through the slot classification model, can find the slots in the user input, extract the slot information, and complete the alignment of the slot information with the entity name in the atlas library through the text similarity model. This process is called entity linking, such as aligning "Changchun Station" to "Changchun 220kV Substation" and "No. 1 Transformer" to "No. 1 Main Transformer". The present invention uses a text similarity model based on cosine similarity to solve the entity linking problem.

[0219] Cosine similarity, also known as cosine distance, uses the cosine value of the angle between two vectors in vector space as a measure of the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0 degrees, that is, the more similar the two vectors are, which is called "cosine similarity".

[0220] Assume that A and B are two n-dimensional vectors, A i is the i-th value of vector A, B i is the i-th value of vector B, then the formula for calculating the cosine of the angle between A and B is as shown below.

[0221]

[0222] In the above formula, the value range of the cosine similarity value is [0, 1], and the larger the value, the higher the similarity. The present invention represents the sentence vectors of two short text sentences A and B based on the language model BERT, and the dimension n of the vector is 202. The cosine similarity of the sentence vectors of the two short texts A and B is calculated to determine the degree of similarity between the two.

[0223] 3》Query retrieval layer

[0224] The query retrieval layer builds a query subgraph based on the concepts, entities, and attributes passed by the intent recognition and slot extraction layers, converts the query into a search engine (Elasticsearch, ES) query statement, and returns potential triples or Q / A answers.

[0225] Taking the above example of "Which factory manufactured the No. 1 main transformer of Changchun Station" as an example, after identification by the intent recognition model and the slot recognition model. Based on "Changchun 220kV substation-No. 1 main transformer-manufacturer", first follow the ES query statement of "Changchun 220kV substation-No. 1 main transformer" to obtain detailed information of No. 1 main transformer, such as Figure 7 shown.

[0226] Then, based on "Main Transformer No. 1 – Manufacturer", according to the ES query statement, obtain the manufacturer of Main Transformer No. 1, such as Figure 8 shown.

[0227] 4》Answer display layer

[0228] This layer is positioned to display different answers according to different types. The answer types can be divided into three categories: collection type query, entity type query, and entity attribute query. Example statements are shown in Table 1 below:

[0229] Table 1

[0230]

[0231] Different display strategies are adopted for different categories.

[0232] 1) Set type, which includes numbers and sets. If the user asks a set type query, such as "What transformers are there in Chitouqing 220kV substation", the number of sets and the specific number of sets need to be returned.

[0233] 2) Entity type, the value is a specific entity, and the detailed attributes of the entity are displayed. For example, if you enter "Where is the manufacturer of the main transformer of Chitouqing 220kV substation #1?", the specific entity "Daur Yellow Rat Co., Ltd." will be displayed, and the detailed information of the entity will be displayed.

[0234] 3) Entity attribute value type, the value is the name of a specific entity or the attribute value of the entity. For example, if the user asks "The voltage level of the #1 main transformer of Chitouqing 220kV substation", the returned value is the value of the voltage level attribute of the #1 main transformer entity.

[0235] In summary, the present invention proposes a corpus enhancement model based on a small sample transformer question template. Based on the small sample question template, combined with the transformer knowledge graph and the synonym library, a large amount of transformer corpus text is automatically realized, and data enhancement in a small sample space is achieved, providing training corpus for generating intent recognition models and slot recognition models through deep learning training.

[0236] Furthermore, the present invention proposes an integrated recognition model of intent and slot based on the bidirectional long short-term memory network BILSTM+attention model. Based on this model, the joint recognition of user query intent and slot can be quickly realized based on the corpus.

[0237] like Fig. 9 As shown, a specific embodiment of the present invention is applied:

[0238] The following example, "Which factory produced the No. 1 main transformer of Changchun Station?", is used to illustrate the interaction process of the above models.

[0239] Step 1: First, segment the sentence input by the user and extract the key words: Changchun Station, No. 1 main transformer, production, and factory.

[0240] Step 2: Input the proposed keywords into the intent recognition model and slot recognition model to recognize the classification and slot information of the intent.

[0241] For example, the sentence input by the user, "Which factory manufactured the No. 1 main transformer of Changchun Station", contains three slots and one relationship slot. The three concept slots are "substation", "transformer", and "manufacturer", and the relationship slot is "production".

[0242] The slot values ​​are shown in Table 2 below.

[0243] Table 2

[0244] Serial number Slot Type Slot Name Specific slot value 1 concept Substation Changchun Station 2 concept transformer No. 1 main transformer 3 concept Manufacturer factory 4 relation Production manufacture

[0245] Input the slot value into the entity linking model, and output the entity name or attribute name under the concept corresponding to the specific word. Entity linking is to perform cosine similarity judgment between the value extracted from the slot and the name of the existing entity in the knowledge base. For example, in the example above "Which factory manufactured the No. 1 main transformer of Changchun Station", the entity name corresponding to "Changchun Station" in the library is "Changchun 220kV Substation", and the entity name corresponding to "No. 1 Main Transformer" in the library is "No. 1 Main Transformer".

[0246] Step 3: Generate an ES query script based on the extracted slot information, execute the ES query script, obtain the query results, the query result type is the entity type, and call the entity type interface for display.

[0247] The corpus enhancement method based on a small sample transformer question template proposed in the present invention can quickly construct transformer corpus data, and applies artificial neural network technology to the corpus data to jointly identify the semantic intent and semantic slot of user input sentences and link entities. The method has the advantages of clear physical meaning, simple actual operation, and easy operation, and has important engineering practice significance.

[0248] An embodiment of a device applying the method of the present invention:

[0249] A computer device comprising:

[0250] one or more processors;

[0251] A storage device for storing one or more programs;

[0252] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned semantic matching method for power transformer knowledge retrieval and question answering.

[0253] Explanation of professional terms involved in the present invention:

[0254] LSTM, Long Short Term Memory networks

[0255] Long short-term memory network, a special kind of neural network that can learn long-term dependencies.

[0256] Door components

[0257] A method of selectively allowing information to pass through. The gate component consists of a Sigmoid neural network layer and an element-wise multiplication operation. The Sigmoid neural network outputs a value between 0 and 1, and each value indicates whether the corresponding part of the information should pass through. A value of 0 means that the information is not allowed to pass through, and a value of 1 means that all information is allowed to pass through. Fig.10 shown.

[0258] BILSTM Bi-directional Long Short-Term Memory

[0259] The bidirectional long short-term neural network LSTM is composed of a forward long short-term neural network LSTM and a backward long short-term neural network LSTM. Both are often used to model contextual information in natural language processing tasks.

[0260] Attention Mechanism

[0261] The attention mechanism imitates the internal process of biological observation behavior, that is, a mechanism that aligns internal experience and external sensation to increase the observation precision of some areas. For example, when processing a picture, human vision will quickly scan the global image to obtain the target area that needs to be focused on, that is, the focus of attention. Then more attention resources are invested in this area to obtain more detailed information about the target that needs to be focused on and suppress other useless information. Based on the attention mechanism, the effect of the Encoder+Decoder model based on RNN (LSTM or GRU) can be improved.

[0262] Semantic intent search intent

[0263] When a user inputs information, the real search target is, for example, "Dayao basketball". The search target is not the two words "Dayao" and "basketball". The combination of these two words expresses the intended target "basketball player Yao Ming". The semantic intent can be divided into retrieval intent and question-answering intent according to the different forms of information interaction.

[0264] Slot

[0265] It can be understood as the attributes of the entity that have been clearly defined. For example, in the taxi-hailing scenario, the attributes of the departure location slot, destination slot, and departure time slot are "departure location", "destination", and "departure time" respectively.

[0266] Semantic Matching

[0267] The process of semantically measuring the similarity of the text and matching the user's phrase vocabulary with the name and attributes of the knowledge point.

[0268] Jieba Chinese word segmenter

[0269] An open source word segmentation software that can segment sentences and paragraphs and convert sentences into word sequences.

[0270] Word embedding

[0271] A general term for a set of language modeling and feature learning techniques in natural language processing (NLP) in which words or phrases from a vocabulary are mapped to vectors of real numbers.

[0272] BERT, Bidirectional Encoder Representations from Transformers

[0273] Refers to the language model proposed by Google, which can provide tasks such as word vector calculation, reading comprehension, and classification.

[0274] Entity Links

[0275] Refers to the process of correctly and unambiguously pointing the recognized entity objects (such as names of people, places, and institutions) in free text to the target entities in the knowledge base, for example, pointing "Dayao" to "Yao Ming".

[0276] Knowledge graph

[0277] The knowledge graph, referred to as KG, is essentially a semantic network, a graph-based data structure consisting of nodes (Point) and edges (Edge). In the knowledge graph, each node represents an "entity" that exists in the real world, and each edge is a "relationship" between entities. The database that stores knowledge graph data is referred to as the graph library.

[0278] Full text search engine

[0279] The mainstream search engine that is widely used at present. Its working principle is that the computer indexing program scans each word in the article and creates an index for each word, indicating the number and position of the word in the article. When the user queries, the search program searches according to the pre-established index and feeds the results back to the user. Commonly used components include Elastic search (abbreviated as ES).

[0280] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0281] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0282] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A semantic matching method for power transformer knowledge retrieval and question answering, characterized in that: The first step is to build a transformer-based knowledge graph KG; In the second step, based on the knowledge graph KG and synonym library in the first step, and using the small sample transformer question template, a corpus enhancement model is established to quickly construct transformer corpus data; The third step is to use artificial neural network technology to train the transformer corpus data in the second step to form an integrated recognition model; The integrated recognition model uses the bidirectional long short-term memory network BILSTM and the attention mechanism Attention to judge the transformer question-answering intention and effectively identify the slot of the input sentence according to the question-answering intention; The identified slot information is linked with the transformer of the knowledge graph KG in the first step to form an effective recognition system; The attention mechanism in the third step quickly scans the semantics of the search and / or question-answer sentences input by the user to obtain the target area that needs to be focused on, i.e., the focus of attention. Then, more attention resources are invested in the focus of attention to obtain more detailed information of the target that needs attention and suppress other useless information; Integrated recognition model, including question input layer, semantic analysis and matching layer, query retrieval layer, and answer display layer; The output of the previous layer serves as the input of the next layer; The question input layer is used to receive user input, segment the sentences input by the user, and extract sentence vocabulary; Problems with user input that are ambiguous, colloquial, abbreviated, with multiple synonyms for the same question, and with multiple interactive inputs; Use the Chinese word segmenter Jieba to segment the input sentence. The segmentation strategy is the longest word strategy, and stop words and meaningless words are removed. The semantic analysis and matching layer is used to perform semantic analysis on the sentences transmitted by the question input layer; Use the bidirectional long short-term memory network BILSTM and the attention mechanism Attention to analyze and match the semantics of the input sentence; It specifically includes the following steps: S1, the vocabulary formed by the word segmenter is represented by a vector, and the word vector of each word is calculated based on the pre-trained model BERT; S2, encodes the features of the word vector in S1 through the bidirectional long short-term memory network BILSTM and the attention mechanism Attention; S3, inputs the word vector encoded in S2 into the semantic intent classifier and the semantic slot classifier, and outputs the queried standard transformer name and / or attribute name and / or relationship name information through the transformer link operation; The query retrieval layer is used to build a query subgraph based on the concepts, transformers, and attributes passed by the intent recognition and slot extraction layer; and convert the query into a search engine query statement, returning potential triples or answers Q / A; The answer display layer is used to display different answers. The answer types are divided into three categories: collection type query, transformer type query, and transformer attribute query; The fourth step is to use the integrated recognition model in the third step to effectively match the semantics of the search and / or question-answer statements input by the user.

2. A semantic matching method for power transformer knowledge retrieval and question answering as claimed in claim 1, characterized in that: The knowledge graph in the first step is a graph-based data structure, which uses triple RDF to describe the attribute values ​​of transformers and the relationship between transformers, and includes several nodes and edges; The node is used to represent various attribute values ​​of a transformer; The edge is used to represent the relationship between transformers; The attribute values ​​include transformer inventory, transformer faults, and detection standards.

3. A semantic matching method for power transformer knowledge retrieval and question answering as claimed in claim 1, characterized in that: The small sample transformer question template in the second step is a slot-based question template, which is used to express the semantics of user input, and includes the sentence input by the user and the semantic annotation of the sentence; The method of constructing the corpus enhancement model is as follows: Use deep learning methods to train question-and-answer related data, perform small sample data enhancement, and increase the amount of corpus data in small samples by professionally enhancing the universal vocabulary of the data set.

4. A semantic matching method for power transformer knowledge retrieval and question answering as claimed in claim 3, characterized in that: The semantic analysis and matching layer includes a word vector layer, a bidirectional long short-term neural network BILSTM encoding layer, an attention layer, and an intention and slot joint recognition layer; The word vector layer is a vector representing vocabulary based on the pre-trained model BERT, which is used to convert text into a computable word vector model; Through the word vector model, each word is converted from a one-dimensional space to a multi-dimensional space with a continuous vector of embedded numbers; The bidirectional long short-term memory network BILSTM encoding layer is composed of two layers of long short-term neural network LSTM with different directions; Each layer of the long short-term neural network LSTM includes a group of artificial neural network units with a chain structure; The attention layer is used to determine the key features of the user input, that is, to calculate the local feature vector; The intent and slot joint recognition layer is used to decode and classify the word vector features calculated based on attention. It includes the intent classification model and the slot classification model.

5. A semantic matching method for power transformer knowledge retrieval and question answering as claimed in claim 4, characterized in that: The long short-term neural network LSTM includes at least one cell state component, a forget gate, a memory gate, and an output gate. Through the interaction of the cell state component, the forget gate, the memory gate, and the output gate, long-distance information processing of sequence data is realized; The cell state component is a data link, which is used to carry information and enable the information to be transmitted unchanged; The forget gate is passed through the forgetting factor f t Decide what information to forget or discard from the cell state; Forgetting factor f t The calculation formula is as follows: f t =σ(W f ·[h t-1 ,x t ]+b f ) In the above formula, W is the weight matrix, which is calculated through subsequent model training. σ is an activation function, namely the ogistic function, which is used for the output of hidden layer neurons and has a value range of (0,1). It maps a real number to the interval of (0,1) for binary classification. The forgetting factor is h t-1 and x t As input, it is calculated through the σ neural network and outputs a number between 0 and 1, where 1 means "completely retain the information" and 0 means "completely discard the information"; Cell state C t-1 Each element in C will be judged by the forget gate to determine whether to keep or forget C t-1 which part of; Memory gates, which decide what new information will be stored in the cell state; The memory gate calculation method is as follows: First, a σ neural network layer is used to decide which information needs to be updated, using i t To express; i t The calculation formula is as follows: i t =σ(W i ·[h t-1 ,x t ]+b i ) Among them, W i is the weight matrix, which is calculated through subsequent data training; Secondly, a new candidate value is created at the tanh layer, where tanh is one of the hyperbolic functions; tanh() is the hyperbolic tangent. The function passes through the origin and the range of the generated values ​​is (-1,1). The value produced by the tanh layer is added to the cell state, and the new value is expressed as To represent, the input is h t-1 and x t , the calculation formula is as follows: Among them, W c is the weight matrix obtained through training, b c is the linear offset; Finally, in the old cell state C t-1 Based on this, update its value to get the new C t The value of C t The calculation formula is as follows: The output gate is used to add some filtering based on the current cell state and output the desired information; The calculation formula of the output gate is as follows: First, we establish a gate of the σ neural network layer to determine which parts of the output cell will be output. The output information is represented by o t Indicates that t The calculation formula is as follows: the t =σ(W o ·[h t-1 ,x t ]+b o ) Among them, W o is the weight matrix, calculated through subsequent training, b o is the linear offset; Then the cell state is activated by the function tanh and then t Multiply, and then output the part you want to output h t ,h t The calculation formula is as follows: h t =o t ×tanh(C t ) After being encoded by the bidirectional long short-term memory network BILSTM encoding layer, the full features of the input sentence are encoded and the global feature vector is output; The global features include the text value, part of speech of the current word, the text value, part of speech, position of the first 1-50 words, and the text value, part of speech, and position features of the next 50 words; Then, after the global feature vector is input, the subsequent attention layer extracts local key features.

6. A semantic matching method for power transformer knowledge retrieval and question answering as claimed in claim 4, characterized in that: The local feature vector of the attention layer is C t , which is calculated as follows: e ij =v a T tanh(w a C t-1 +u a h j ); Where T is the number of vector sequences output by the bidirectional long short-term memory network BILSTM layer, a tj is the weight given by the attention layer to all feature vectors, e ij is the associated energy of input and output, v a is the global eigenvector, c t-1 is the state vector of the attention model at the previous moment, h j is the feature vector sequence output by the bidirectional long short-term memory network BILSTM layer, u a is the weight of the eigenvector at the previous moment, w a is the weight of the attention mechanism at the previous moment.

7. A semantic matching method for power transformer knowledge retrieval and question answering as claimed in claim 4, characterized in that: The two classification models of the intent and slot joint recognition layer are generated based on the Softmax classifier; The calculation formula of the Softmax classifier is as follows: where z i is the output value of the i-th node, C is the number of output nodes, that is, the number of classification categories; The intent classification model is implemented based on the softmax classifier and is used to determine the classification of user intent. It includes a slot classification module and an entity linking module. Slot classification module, used to express the basic semantic position of natural language sentences; The entity linking module is used to find the slots in the user input through the slot classification model, extract the slot information, and align the slot information with the transformer name in the atlas library through the text similarity model, that is, transformer linking. The text similarity model is a text similarity model based on cosine similarity; Cosine similarity, also known as cosine distance, uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals; the closer the cosine value is to 1, the closer the angle is to 0 degrees, indicating that the two vectors are more similar; The calculation formula of the cosine similarity value of the vector angle is as follows: Among them, A and B are the sentence vectors of short texts respectively; A and B are n-dimensional vectors, respectively. i is the i-th value of vector A, B i is the i-th value of the B vector; The cosine similarity value range is [0, 1], and the larger the value, the higher the similarity.

8. A semantic matching method for power transformer knowledge retrieval and question answering as described in any one of claims 4 to 7, characterized in that: The answer display layer adopts different display strategies for different categories; 1) Collection type, whose type data includes numbers and collections; The query on which collection type the user asked; 2) Transformer type, whose value is a specific transformer, showing the detailed properties of the transformer; 3) Transformer attribute value type, whose value is the name of a specific transformer or the attribute value of the transformer.

9. A semantic matching system for knowledge retrieval and question answering of power transformers, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a semantic matching method for power transformer knowledge retrieval and question answering as described in any one of claims 1 to 8.

Images