Electric power material named entity identification method and system

Abstract

The invention provides an electric power material named entity identification method and system, and relates to the technical field of artificial intelligence. According to the specific implementation scheme, an attention mechanism is adopted, semantic learning is conducted on all characters in an electric power material target text, length coding vectors of all sequence text segments are fused, and intermediate vector representation of all the sequence text segments is obtained; based on a plurality of convolutional layers, respectively carrying out residual summation on the second vector representation of each sequence text segment to obtain a target vector representation of each sequence text segment; based on the target vector representation of each sequence text segment and the target vector representation of each entity category, calculating a semantic matching degree between each sequence text segment and each entity category; and based on the semantic matching degree of each sequence text segment and each entity category, determining the entity category to which each sequence text segment belongs. According to the invention, the identification accuracy of the electric power material named entity can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for identifying named entities of electric power materials. Background Art

[0002] Entity extraction is widely used in knowledge graphs, knowledge discovery, question-answering systems, etc. Entity extraction in the field of power material supply chain is entity extraction for specific fields.

[0003] At present, the research on named entity recognition is mainly based on supervised learning algorithms. The core idea is to train the model on large-scale annotated text so that the model can recognize the positive and negative features of entities of a specified category. In traditional machine learning methods, the most critical part is feature engineering, that is, how to design how to convert word-level features, document-level features, and corpus features into feature vectors, so as to abstractly express the target sample. Then, based on the feature vector, traditional machine learning algorithms are used for named entity recognition, for example, the named entity recognition system IdentiFinder based on the hidden Markov model (HMM) is used to recognize entities such as names, times, and numbers. In addition, researchers also handle named entity recognition tasks based on conditional random fields (CRF) and support vector machines (SVM). Compared with rule-based named entity recognition methods, the entity recognition accuracy of machine learning-based named entity recognition methods has been greatly improved.

[0004] However, such methods are affected by the subjectivity of feature selection in the feature engineering stage, and inappropriate feature selection may lead to a decline in algorithm performance. Summary of the invention

[0005] The present invention provides a method, device, system and storage medium for identifying named entities of electric power materials, which can solve at least one of the above technical problems.

[0006] According to one aspect of the present invention, a method for identifying named entities of electric power materials is provided, comprising:

[0007] Using the attention mechanism, semantic learning is performed on each character in the target text of electric power materials to obtain the first vector representation of each sequence text segment in the target text of electric power materials;

[0008] Based on the length encoding vectors of each of the sequence text segments, respectively fuse the length information of the first vector representations of each of the sequence text segments to obtain a second vector representation of each of the sequence text segments;

[0009] Based on multiple convolutional layers, respectively summing the residuals of the second vector representations of each of the sequence text segments to obtain a target vector representation of each of the sequence text segments;

[0010] Based on the target vector representation of each of the sequence text segments and the target vector representation of each of the entity categories, calculating the semantic matching degree between each of the sequence text segments and each of the entity categories;

[0011] Based on the semantic matching degree between each of the sequence text segments and each of the entity categories, the entity category to which each of the sequence text segments belongs is determined.

[0012] According to another aspect of the present invention, there is provided a device for identifying named entities of electric power materials, comprising:

[0013] An attention processing module, used for using an attention mechanism to perform semantic learning on each character in the target text of electric power materials, and obtaining a first vector representation of each sequence text segment in the target text of electric power materials;

[0014] A length fusion module, used for fusing the length information of the first vector representations of the respective sequence text segments based on the length encoding vectors of the respective sequence text segments, so as to obtain the second vector representations of the respective sequence text segments;

[0015] A convolution processing module, configured to perform residual summation on the second vector representation of each of the sequence text segments based on multiple convolution layers to obtain a target vector representation of each of the sequence text segments;

[0016] A matching degree calculation module, used for calculating the semantic matching degree between each of the sequence text segments and each of the entity categories based on the target vector representation of each of the sequence text segments and the target vector representation of each of the entity categories;

[0017] The entity category determination module is used to determine the entity category to which each of the sequence text segments belongs based on the semantic matching degree between each of the sequence text segments and each of the entity categories.

[0018] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any electric power material named entity recognition method in the embodiments of the present invention.

[0022] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method for identifying named entities of electric power materials according to an embodiment of the present invention.

[0023] By adopting the technical solution of the present invention, the electric power material target text is first encoded by using the attention mechanism to obtain the first vector representation of each sequence text segment in the electric power material target text. Based on the length encoding vector of each of the sequence text segments, the length information of the first vector representation of each of the sequence text segments is fused to obtain the second vector representation of each of the sequence text segments, and the sequence vector representation with fused length information is obtained. Through multiple convolutional layers, the residuals of the second vector representation of the sequence text segment are summed up respectively to obtain the final vector representation of the sequence text segment. Then, based on the target vector representation of each of the sequence text segments and the target vector representation of each entity category, the semantic matching degree between each of the sequence text segments and each of the entity categories can be calculated. Therefore, based on the semantic matching degree, the entity category to which each of the sequence text segments belongs can be accurately determined.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0026] Figure 1 is a flow chart of a method for identifying named entities of electric power materials according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a named entity recognition process according to an embodiment of the present invention;

[0028] Figure 3 It is a framework structure diagram of a named entity recognition model according to an embodiment of the present invention;

[0029] Figure 4 It is a structural block diagram of a device for identifying named entities of electric power materials according to an embodiment of the present invention;

[0030] Figure 5 The block diagram is a block diagram of an electronic device for implementing the method according to the embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] Terminology explanation:

[0033] Flat means that any character in a sentence can belong to at most one entity word.

[0034] Nesting means that characters in a sentence can belong to multiple entity words, that is, there is a hierarchical relationship between named entities.

[0035] Figure 1 The present invention is a flowchart of a method for identifying named entities of electric power materials according to an embodiment of the present invention.

[0036] like Figure 1 As shown, the method for identifying named entities of electric power materials may include:

[0037] S110, using an attention mechanism to perform semantic learning on each character in the target text of electric power materials, and obtaining a first vector representation of each sequence text segment in the target text of electric power materials;

[0038] S120, based on the length encoding vectors of the respective sequence text segments, respectively fuse the length information of the first vector representations of the respective sequence text segments to obtain second vector representations of the respective sequence text segments;

[0039] S130, based on multiple convolutional layers, respectively summing the residuals of the second vector representations of each sequence text segment to obtain a target vector representation of each sequence text segment;

[0040] S140, calculating the semantic matching degree between each sequence text segment and each entity category based on the target vector representation of each sequence text segment and the target vector representation of each entity category;

[0041] S150, determining the entity category to which each sequence text segment belongs based on the semantic matching degree between each sequence text segment and each entity category.

[0042] Exemplarily, for any target text of electric power materials, the above steps S110 to S130 may be used for encoding processing to obtain target vector representations of each sequence text segment in the target text of electric power materials.

[0043] It can be understood that the power material target text may be a sentence, a paragraph or an article, etc. The power material target text may include multiple characters, one or more consecutive characters form a sequence text segment, and the same one or more characters may exist between each sequence text segment.

[0044] It can be understood that the first vector representation and the second vector representation are intermediate representations of the sequence text segment. The target vector representation of the sequence text segment is an embedded representation, that is, any corresponding character in the target vector representation can appear in one or more sequence text segments.

[0045] It can be understood that the target vector representation of the entity category is also an embedded representation, that is, any corresponding character in the target vector representation can appear in one or more entity categories.

[0046] For example, a model corresponding to the attention mechanism can be used to perform semantic learning on each character in the target text of electric power materials to obtain the first vector representation of each sequence text segment in the target text of electric power materials. The parameters of the model can be pre-trained.

[0047] Exemplarily, the length encoding vector of the sequence text segment and the first vector representation of the sequence text segment may be concatenated to obtain a second vector representation of the sequence text segment. In this way, the second vector representation of the sequence text segment incorporates the length information of the sequence text segment.

[0048] Exemplarily, the convolution kernel size of each convolution layer in the multiple convolution layers is the same, so that the lattice embedding shape of the sequence text segment can be guaranteed not to change, and the spatial relationship between each sequence text segment can be captured. With respect to the spatial relationship, in each sequence text segment in the target text, there are three spatial relationships between different sequence text segments: separation, intersection, and inclusion. Separation means that there are no common characters between two sequence text segments. Intersection means that two sequence text segments have some common characters. Inclusion means that one sequence text segment is a text segment in another sequence text segment.

[0049] Exemplarily, based on the semantic matching degree between the sequence text segment and each entity category, the entity category with the highest semantic matching degree with the sequence text segment is determined among the entity categories as the entity category to which the sequence text segment belongs.

[0050] Figure 2 It is a schematic diagram of a named entity recognition process of an example of the present invention.

[0051] like Figure 2 As shown, the named entity recognition task belongs to the information extraction task. The goal of this task is to extract (For example, ), for the input sentence T = [t1,…,ti ,…,t n ] satisfies the category set The entity fragments required by the category are located and classified.

[0052] In this task, the input is the target category and the target sentence T. The goal of the task is to semantically match each sequence segment in sentence T with the target category, so as to identify the entities in the target sentence. For example, for the sentence "There are low-price bids in the transformer procurement process", the recognition result of its named entity is: the sequence segment that semantically matches the entity category "risk" is "low-price bid". The recognition process is: first, map the entity category x to the prompt word p(x), and for the subsequence T of sentence T i,j =[t i ,…,t j ]; Then, the sequence fragment encoding module is used to encode the subsequence T i,j Coded as f target (T,i,j), use the category prompt word encoding module to encode the prompt word p(x) into f in (p(x)); Finally, the similarity between the two is calculated through the semantic matching module. If the similarity measurement algorithm is S and the threshold is ρ, if Sf in (p(x)),f target (T,i,j) / ≥ρ, then the sequence T i,j It is an entity of type x.

[0053] According to the above implementation, the electric power material target text is first encoded using an attention mechanism to obtain the first vector representation of each sequence text segment in the electric power material target text. Based on the length encoding vector of each of the sequence text segments, the length information of the first vector representation of each of the sequence text segments is fused to obtain the second vector representation of each of the sequence text segments, and the sequence vector representation of the fused length information is obtained. Through multiple convolutional layers, the residuals of the second vector representations of the sequence text segments are summed up respectively to obtain the final vector representation of the sequence text segment. Then, based on the target vector representation of each of the sequence text segments and the target vector representation of each entity category, the semantic matching degree between each of the sequence text segments and each of the entity categories can be calculated. Therefore, based on the semantic matching degree, the entity category to which each of the sequence text segments belongs is determined, which can improve the recognition accuracy of named entities.

[0054] Figure 3 is a framework diagram of a named entity recognition model according to an embodiment of the present disclosure;

[0055] like Figure 3As shown, the method of any embodiment of the present invention can be applied to a named entity recognition model, and the named entity recognition model can include a sequence segment encoding module, a category prompt word encoding module, and a semantic matching module. Among them, the sequence segment encoding module is used to perform the above steps S110 to S130. The category prompt word encoding module is used to pre-encode each entity category to obtain a target vector representation of each entity category. The semantic matching module is used to perform the above step S140.

[0056] The following is an example of encoding the target text by the sequence fragment encoding module.

[0057] In one embodiment, an attention mechanism is used to perform semantic learning on each character in the electric power material target text to obtain a first vector representation of each sequence text segment in the electric power material target text, including: performing a double affine transformation on the embedded vector representation of each character in the electric power material target text to obtain a third vector representation and a fourth vector representation of the electric power material text, wherein the third vector representation includes the embedded representation when each character is a sequence head character, and the fourth vector representation includes the embedded representation when each character is a sequence tail character; multiple attention heads are used to perform semantic learning on the third vector representation and the fourth vector representation of the electric power material target text to obtain a first vector representation of each sequence text segment in the electric power material target text.

[0058] For example, take the target text of electric power materials as an input sentence, for the input sentence T = [t1, ..., t i ,…,t n ] to perform contextual semantic encoding. Where n represents the total number of characters in the sentence. Sentence T is input into the BERT (Bidirectional Encoder Representations from Transformers) model to obtain the embedding representation of sentence T Among them, d t Indicates the encoding dimension size of BERT. The details are as follows:

[0059] E text =f BERT (T)

[0060] For example, through a multi-head attention mechanism, the embedding representations of all sequence fragments in a sentence are calculated.

[0061] For example, in the attention mechanism, the attention score is calculated by double affine transformation, so that the correlation between each pair of elements in the input sequence can be obtained. text , and obtain the intermediate representation through two affine transformations Es represents the embedding representation of each character in sentence T as the sequence head character, that is, the third vector representation mentioned above. e Represents the embedding representation of each character in sentence T as the end character of the sequence, that is, the fourth vector representation mentioned above. The details are as follows:

[0062] E s =E text W s +b s ;

[0063] E e =E text W e +b e ;

[0064] Among them, W s , b s , are the learnable parameters in the sequence segment encoding module.

[0065] Exemplarily, any one of the multiple attention heads is used to perform affine transformation on the third vector representation and the fourth vector representation of the electric power material target text to obtain the sequence text segment encoding output by the attention head, and then based on the sequence text segment encoding output by the attention head, the first vector representation of each sequence text segment can be obtained.

[0066] According to the above implementation, the attention mechanism can be used to learn the contextual semantics between the sequence text segments to obtain the first vector representation of each sequence text segment.

[0067] In one embodiment, multiple attention heads are used to perform semantic learning on the third vector representation and the fourth vector representation of the electric power material target text to obtain the first vector representation of each sequence text segment in the electric power material target text, including: using m attention heads to perform affine transformation on the third vector representation and the fourth vector representation of the electric power material target text respectively to obtain the sequence text segment encoding vector output by each attention head; splicing the sequence text segment encoding vectors output by the m attention heads to obtain the lattice embedding representation of all the sequence text segments in the electric power material target text; based on the first character and the last character in each sequence text segment, in the lattice embedding representation of all the sequence text segments, respectively determine the first vector representation of each sequence text segment.

[0068] For example, for the i-th attention head, the intermediate representation E s With E e Perform double affine transformation to obtain the sequence text segment encoding of the i-th attention head The following formula, first The first two dimensions of E sDo matrix operations, then, The last two dimensions of E e Perform matrix operations. Finally, add the bias vector based on the matrix operations. Get the sequence text segment encoding of the i-th attention head

[0069]

[0070] in,

[0071] For example, the sequence text segment encodings output by m attention heads are concatenated to obtain the lattice embedding representation of all sequence segments in the input sentence T The specific formula is:

[0072] For example, from In, it can be extracted in, Represents the sequence fragment T in the corresponding input sentence T p,q =[t p ,…,t q ], i.e., the sequence fragment T p,q The first vector of p and q represent the sequence fragment T. p,q The sequence numbers of the first and last characters in sentence T.

[0073] In some embodiments, in order to further enhance the semantic representation of the sequence text segment, the first vector representation of the sequence text segment can be concatenated with the vector representation of the first character of the sequence text segment in the third vector representation and the vector representation of the last character of the sequence text segment in the fourth vector representation in the form of residual connection to obtain an adjusted first vector representation of the sequence text segment.

[0074] For example, the third vector is represented by E s , the fourth vector represents E e Input a Dropout (random inactivation) layer to get the fifth vector representation E s ′ , the sixth vector represents E e ′ . Sequence text segment T p,q =[t p ,…,t q The intermediate representation of ] and the vector representation E of character p in the fifth vector representation s ′ [p], the vector representation E of the character q in the sixth vector representation e ′[q] is concatenated to obtain the adjusted first vector representation of the sequence text segment Among them, d o =d G +2h. The specific formula is as follows:

[0075]

[0076] Among them, the final first vector of any sequence fragment in sentence T is expressed as f D Indicates the Dropout operation.

[0077] In some embodiments, since the attention calculation only considers the semantic association between the first and last characters of the sequence text segment and ignores the length information of the sequence segment, it is necessary to encode the length of the sequence segment according to the distance between the first and last characters. Therefore, a learnable length encoding parameter is introduced. Where L represents the maximum legal length of a sequence segment. p,q The length encoding vector of can be Then, the first vector representation of each sequence text segment is concatenated with the corresponding length encoding vector to obtain the intermediate representation of the fused length information of each sequence text segment. That is, the second vector representation mentioned above. r =d o +d D The specific formula is as follows: E GD [p,q]=[E G [p,q]; E D [|q-p+1|]].

[0078] In some embodiments, in each sequence text segment of sentence T, different sequence text segments have at least one of separation, intersection and inclusion spatial relationships. In order to capture the spatial relationship between sequence text segments and enhance the embedding representation of sequence text segments, the second vector representation E of the sequence text segment with integrated length information can be GD Input k convolutional layers in the form of residual connection. The convolution kernel size of each layer is the same. For example, the convolution kernel size is r×r, the padding size is p, and the number of movement steps of the convolution kernel is s. In order to ensure that the lattice embedding shape of the sequence text segment does not change, 2×p-r+s=0. For another example, the number of input channels is d r , the number of output channels is d s , and finally get the embedding representation of the sequence text segment That is, the target vector representation of the sequence text segment. The details are as follows:

[0079]

[0080] Among them, σ represents the Gelu activation function, Conv l represents the lth convolutional layer.

[0081] The following is an example of encoding the target text using the category cue word encoding module.

[0082] In one embodiment, the above method may also include: determining the corresponding prompt words based on each entity category; concatenating the prompt words to obtain a query instruction; encoding the query instruction to obtain an embedded representation of the query instruction; masking each character representation in the embedded representation of the query instruction to obtain a mask instruction; and performing maximum aggregation on the embedded representation of the query instruction according to the mask instruction to obtain a target vector representation of each entity category.

[0083] It is understandable that the target vector representation for the entity category can be predetermined and stored in the database. Each time an entity in the input target text is recognized, the target vector representation of each entity category is directly obtained from the database. This operation can be performed when applying the above solution or model training.

[0084] Understandably, the target vector of the entity category is represented as an embedding representation.

[0085] Exemplarily, a mapping relationship p between a pre-established entity category label and a text representation containing the entity category concept is obtained. For an entity category set Ω, any entity category v∈Ω in the set is obtained through the mapping relationship p. Specifically, it can be P v =[t v1 ,…,t vi ,…,t vz ]. Where z is the number of characters contained in the entity category.

[0086] Exemplarily, the prompt words of each entity category in the entity category set Ω are organized into query instructions.

[0087] Exemplarily, each prompt word is concatenated by a separator s. For example, if the separator s is “,”, the query instruction may be “name entity, location entity, organization entity”.

[0088] For example, by using the pre-trained language model BERT to perform context encoding on the query instruction Q, an embedded representation of the query instruction can be obtained. Since BERT adds special identifiers [CLS] and [SEP] at the beginning and end of the sentence during the word segmentation process of the input sentence, the following query instruction Q is finally obtained. Where Q = [[CLS], s, …, s, t v1 ,…,t vz ,s,…,s,[SEP]].

[0089] For example, if the number of characters (tokens) included in the query instruction Q is c, then through the pre-trained language model BERT, the embedding representation of the query instruction Q can be obtained as

[0090] For example, in order to obtain the embedding representation of the entity category hint word granularity, it is necessary to embed the query instruction E Q The aggregation operation is performed. Among them, the special identifiers [CLS], [SEP] and the prompt word separator s contained in the query instruction are regarded as a meaningless entity category prompt word.

[0091] Exemplarily, before aggregation, the query instruction needs to be masked to treat the special identifiers [CLS], [SEP] and the prompt word separator s contained in the query instruction as meaningless entity category prompt words, and enhance the entity category prompt words with specific meanings.

[0092] In one embodiment, each character representation in the embedded representation of the query instruction is masked to obtain a mask instruction, including: when the character representation is a vector representation of any character in any prompt word in the prompt word set, the value of the corresponding position identified in the mask instruction is 1; when the character representation is not a vector representation of any character in any prompt word in the prompt word set, the value of the corresponding position identified in the mask instruction is 0.

[0093] For example, a mask M is created for each prompt word v v . M v is a vector of length c consisting of elements 0 and 1. For the i-th character t in the query instruction Q i , if it is a constituent character of the entity category prompt word v, then M v [i] = 1, otherwise 0. In this way, the mask instruction M corresponding to the query instruction Q can be obtained. Q .

[0094] For example, according to the mask instruction M Q , for E Q The characters of each entity category prompt word in the maximum value aggregation (max pooling) operation is performed. The details are as follows:

[0095] e v =f MaxPooling (M Q E Q );

[0096] Among them, the hint word embedding of entity category v is expressed as

[0097] Thus, the embedding of the entity category hint word granularity of the query instruction Q is expressed as Among them, n p =|Ω|+3.

[0098] According to the above implementation, the target vector representation of each entity category can be accurately encoded.

[0099] The following is an example of the semantic matching module performing semantic matching on each sequence text segment in the target text and each entity category.

[0100] In one embodiment, based on the target vector representation of each sequence text segment and the target vector representation of each entity category, the semantic matching degree between each sequence text segment and each entity category is calculated, including: performing an inner product operation on the target vector representation of the sequence text segment and the target vector representation of the entity category to obtain the semantic matching score of the sequence text segment and the entity category; normalizing the semantic matching score to obtain the semantic matching degree between the sequence text segment and the entity category.

[0101] For example, after obtaining the embedding representation E of each sequence text segment of the input sentence T span and the hint word embedding representation E consisting of each entity category Prompt Finally, the relevance score between each sequence text segment and each entity category prompt word is obtained through inner product operation.

[0102] For example, through a fully connected layer, the embedding representation of the entity category is projected into the same dimensional space as the embedding representation of the sequence text segment, and we get in, is a learnable parameter. The details are as follows:

[0103] E p ′ rompt =E Prompt W p +b p .

[0104] For example, the embedding representation E of the sequence text segment is span and the embedding representation E of each entity category p ′ rompt Perform inner product operation to obtain the semantic matching scores between the sequence text segment and each entity category Among them, n p Represents the number of target entity categories. Both S[p,q,j] and S[q,p,j] can be represented as sequence text segments T p,q The matching score with the j-th entity category. The details are as follows:

[0105] S=E span ⊙E p ′ rompt T .

[0106] For example, the matching probability between each sequence text segment of the input sentence T and each entity analogy is calculated P[p,q,j]∈(0,1) represents the sequence text segment T p,q The probability of matching with the j-th entity category is the semantic matching degree mentioned above. The details are as follows:

[0107]

[0108] According to the above implementation, the entity categories of each sequence text segment in the target text can be accurately identified.

[0109] The above process can be applied to both the actual named entity recognition process and the training process of the named entity model. During the training, the target text can be a sample text, for example, a sample sentence.

[0110] The following describes a model training process according to an embodiment of the present invention.

[0111] In some embodiments, for training samples <T=[t1,…,t i ,…,t n ], Ents = {(s, e, v) ...}>, T is a sample sentence, Ents is the set of real entity triples of the sample, (s, e, ν) represents the sequence text segment T s,e It is a v-type entity word.

[0112] For example, the training samples are constructed into a tensor of true semantic matching between sequence text segments and entity category words Then when the sample sentence T contains the entity category (s, e, v), Y[s, e, v] = Y[e, s, v] = 1, otherwise Y[s, e, v] = Y[e, s, v] = 0.

[0113] An exemplary training objective is:

[0114] Exemplarily, the model parameter setting includes the following steps:

[0115] 1. The present invention adopts the adaptive momentum estimation algorithm AdamW with decoupled weight decay as the learner, and the weight decay rate is 1e-5.

[0116] 2. When encoding sentences and entity category prompt words, the pre-trained language model used in the present invention adopts the same version of BERT, so as to achieve parameter sharing. For example, the context encoding dimension of the input sentence and the query instruction is set to 768.

[0117] 3. The present invention can set the convolution kernel size of the convolution layer to 3×3, the padding size of the convolution layer to 1, and the moving step number of the convolution kernel to 1.

[0118] 4. The model training process of the present invention may adopt a linear learning rate warmup algorithm, wherein the learning rate warmup ratio may be 0.1. The loss function used during training may be a binary cross entropy loss function.

[0119] 5. Use the entity recognition model obtained through the above training to extract entities from the target text.

[0120] Figure 4 It is a structural block diagram of a device for identifying named entities of electric power materials according to an embodiment of the present invention.

[0121] like Figure 4 As shown, the electric power material named entity recognition device may include:

[0122] An attention processing module 410 is used to use an attention mechanism to perform semantic learning on each character in the electric power material target text to obtain a first vector representation of each sequence text segment in the electric power material target text;

[0123] A length fusion module 420 is used to fuse the length information of the first vector representations of the respective sequence text segments based on the length encoding vectors of the respective sequence text segments, so as to obtain the second vector representations of the respective sequence text segments;

[0124] A convolution processing module 430 is used to perform residual summation on the second vector representation of each of the sequence text segments based on multiple convolution layers to obtain a target vector representation of each of the sequence text segments;

[0125] A matching degree calculation module 440, configured to calculate the semantic matching degree between each of the sequence text segments and each of the entity categories based on the target vector representation of each of the sequence text segments and the target vector representation of each of the entity categories;

[0126] The entity category determination module 450 is used to determine the entity category to which each of the sequence text segments belongs based on the semantic matching degree between each of the sequence text segments and each of the entity categories.

[0127] In one embodiment, the attention processing module includes:

[0128] A double affine transformation unit, used for performing a double affine transformation on the embedded vector representation of each character in the electric power material target text to obtain a third vector representation and a fourth vector representation of the electric power material text, wherein the third vector representation includes the embedded representation when each character is a sequence head character, and the fourth vector representation includes the embedded representation when each character is a sequence tail character;

[0129] The semantic learning unit is used to use multiple attention heads to perform semantic learning on the third vector representation and the fourth vector representation of the electric power material target text to obtain the first vector representation of each sequence text segment in the electric power material target text.

[0130] In one embodiment, the semantic learning unit is specifically used for:

[0131] Using m attention heads, respectively performing affine transformation on the third vector representation and the fourth vector representation of the electric power material target text, to obtain the sequence text segment encoding vector output by each of the attention heads;

[0132] splicing the sequence text segment encoding vectors output by the m attention heads to obtain a lattice embedding representation of all sequence text segments in the electric power material target text;

[0133] Based on the first character and the last character in each of the sequence text segments, a first vector representation of each of the sequence text segments is determined in the lattice embedding representation of all the sequence text segments.

[0134] In one embodiment, the above device further includes an entity category encoding module, which specifically includes:

[0135] A prompt word determination unit, used for determining the prompt words corresponding to the respective entity categories based on the respective entity categories;

[0136] A prompt word concatenation unit, used for concatenating the prompt words to obtain a query instruction;

[0137] An instruction encoding unit, used for encoding the query instruction to obtain an embedded representation of the query instruction;

[0138] An instruction masking unit, used for masking each character representation in the embedded representation of the query instruction to obtain a masked instruction;

[0139] A maximum aggregation unit is used to perform maximum aggregation on the embedded representation of the query instruction according to the mask instruction to obtain a target vector representation of each entity category.

[0140] In one implementation, the instruction encoding unit is specifically used to:

[0141] In the case where the character representation is a vector representation of any character in any prompt word in the prompt word set, the value identifying the corresponding position in the mask instruction is 1;

[0142] In the case where the character representation is not the vector representation of any character in any prompt word in the prompt word set, the value identifying the corresponding position in the mask instruction is 0.

[0143] In one implementation, the matching degree calculation module 440 includes:

[0144] a matching score determination unit, configured to perform an inner product operation on a target vector representation of the sequence text segment and a target vector representation of the entity category to obtain a semantic matching score between the sequence text segment and the entity category;

[0145] The score normalization unit is used to normalize the semantic matching score to obtain the semantic matching degree between the sequence text segment and the entity category.

[0146] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present invention, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0147] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0148] According to an embodiment of the present invention, the present invention also provides a system and a readable storage medium.

[0149] Figure 5 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0150] like Figure 5As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0151] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0152] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the electric power material named entity recognition method. For example, in some embodiments, the electric power material named entity recognition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the electric power material named entity recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the electric power material named entity recognition method in any other appropriate manner (e.g., by means of firmware).

[0153] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0154] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0157] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0158] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0159] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0160] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying named entities of electric power materials, characterized in that: include: Using the attention mechanism, semantic learning is performed on each character in the target text of electric power materials to obtain the first vector representation of each sequence text segment in the target text of electric power materials; Based on the length encoding vectors of each of the sequence text segments, respectively fuse the length information of the first vector representations of each of the sequence text segments to obtain a second vector representation of each of the sequence text segments; Based on multiple convolutional layers, respectively summing the residuals of the second vector representations of each of the sequence text segments to obtain a target vector representation of each of the sequence text segments; Based on the target vector representation of each of the sequence text segments and the target vector representation of each of the entity categories, calculating the semantic matching degree between each of the sequence text segments and each of the entity categories; Based on the semantic matching degree between each of the sequence text segments and each of the entity categories, the entity category to which each of the sequence text segments belongs is determined.

2. The method according to claim 1, characterized in that: The attention mechanism is used to perform semantic learning on each character in the target text of electric power materials to obtain the first vector representation of each sequence text segment in the target text of electric power materials, including: Performing a double affine transformation on the embedded vector representation of each character in the electric power material target text to obtain a third vector representation and a fourth vector representation of the electric power material text, wherein the third vector representation includes the embedded representation when each character is a sequence head character, and the fourth vector representation includes the embedded representation when each character is a sequence tail character; Multiple attention heads are used to perform semantic learning on the third vector representation and the fourth vector representation of the electric power material target text to obtain the first vector representation of each sequence text segment in the electric power material target text.

3. The method according to claim 2, characterized in that The method of using multiple attention heads to perform semantic learning on the third vector representation and the fourth vector representation of the electric power material target text to obtain the first vector representation of each sequence text segment in the electric power material target text includes: Using m attention heads, respectively performing affine transformation on the third vector representation and the fourth vector representation of the electric power material target text, to obtain the sequence text segment encoding vector output by each of the attention heads; splicing the sequence text segment encoding vectors output by the m attention heads to obtain a lattice embedding representation of all sequence text segments in the electric power material target text; Based on the first character and the last character in each of the sequence text segments, a first vector representation of each of the sequence text segments is determined in the lattice embedding representation of all the sequence text segments.

4. The method according to claim 1, characterized in that: Also includes: Based on each of the entity categories, determining the corresponding prompt words; Connect the prompt words to obtain a query instruction; Encoding the query instruction to obtain an embedded representation of the query instruction; Masking each character representation in the embedded representation of the query instruction to obtain a masked instruction; According to the mask instruction, maximum value aggregation is performed on the embedding representation of the query instruction to obtain a target vector representation of each entity category.

5. The method according to claim 4, characterized in that The step of masking each character representation in the embedded representation of the query instruction to obtain a mask instruction includes: In the case where the character representation is a vector representation of any character in any prompt word in the prompt word set, the value identifying the corresponding position in the mask instruction is 1; In the case where the character representation is not the vector representation of any character in any prompt word in the prompt word set, the value identifying the corresponding position in the mask instruction is 0.

6. The method according to any one of claims 1 to 5, characterized in that The calculating the semantic matching degree between each of the sequence text segments and each of the entity categories based on the target vector representation of each of the sequence text segments and the target vector representation of each of the entity categories comprises: Performing an inner product operation on a target vector representation of the sequence text segment and a target vector representation of the entity category to obtain a semantic matching score between the sequence text segment and the entity category; The semantic matching scores are normalized to obtain the semantic matching degrees of the sequence text segments and the entity categories.

7. A device for identifying named entities of electric power materials, characterized in that: include: An attention processing module, used for using an attention mechanism to perform semantic learning on each character in the target text of electric power materials, and obtaining a first vector representation of each sequence text segment in the target text of electric power materials; A length fusion module, used for fusing the length information of the first vector representations of the respective sequence text segments based on the length encoding vectors of the respective sequence text segments, so as to obtain the second vector representations of the respective sequence text segments; A convolution processing module, configured to perform residual summation on the second vector representation of each of the sequence text segments based on multiple convolution layers to obtain a target vector representation of each of the sequence text segments; A matching degree calculation module, used for calculating the semantic matching degree between each of the sequence text segments and each of the entity categories based on the target vector representation of each of the sequence text segments and the target vector representation of each of the entity categories; The entity category determination module is used to determine the entity category to which each of the sequence text segments belongs based on the semantic matching degree between each of the sequence text segments and each of the entity categories.

8. The device according to claim 7, characterized in that The attention processing module comprises: A double affine transformation unit, used for performing a double affine transformation on the embedded vector representation of each character in the electric power material target text to obtain a third vector representation and a fourth vector representation of the electric power material text, wherein the third vector representation includes the embedded representation when each character is a sequence head character, and the fourth vector representation includes the embedded representation when each character is a sequence tail character; The semantic learning unit is used to use multiple attention heads to perform semantic learning on the third vector representation and the fourth vector representation of the electric power material target text to obtain the first vector representation of each sequence text segment in the electric power material target text.

9. A named entity recognition system for electric power materials, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

Images