Named entity recognition method and named entity recognition model construction method

Abstract

The embodiment of the invention provides a named entity recognition method and a named entity recognition model construction method. The named entity recognition method comprises the following steps: processing a coding feature tensor by adopting a double affine network to obtain an initial fusion tensor; processing the initial fusion tensor comprising the first attention mechanism network and the second attention mechanism network to obtain a final fusion tensor; and processing the final fusion tensor by using a prediction network to obtain an identified named entity, the identified named entity being a named entity in which the minimum semantic unit is positively sorted. The named entity recognition method provided by the embodiment of the invention is excellent in processing nested entities and cross-sentence entity blocks, overcomes the limitation of recognizing a complex text structure in the prior art, and remarkably improves the recognition capability of named entities in a complex text.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, and in particular to a named entity recognition method and a named entity recognition model construction method. Background Art

[0002] As a key step in information extraction and its subsequent technical applications, named entity recognition has become a basic technology in many fields such as knowledge graph construction and information retrieval. However, current named entity recognition technology still faces many challenges when processing complex texts, including poor recognition accuracy of nested entities, resulting in insufficient parsing capabilities for complex texts. Summary of the invention

[0003] In order to solve the problem that existing named entity recognition methods are insufficient in parsing complex texts, the embodiments of the present disclosure provide a new named entity recognition method and a named entity recognition model construction method.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for named entity recognition, including: Using an encoding network to encode the text to be recognized, and obtaining an encoding feature tensor, wherein the encoding feature tensor includes encoding tensors that are sorted according to the minimum semantic units in the text to be recognized and correspond to the minimum semantic units; A dual affine network is used to process the encoding feature tensor to obtain a representation tensor associated with all candidate named entities with each minimum semantic unit as the starting semantic unit and the ending semantic unit, and the representation tensors corresponding to each candidate named entity are sorted and combined in two dimensions to obtain an initial fusion tensor; wherein the two-dimensional sorting includes sorting the representation tensor in a first dimension according to the order of each minimum semantic unit in the text to be recognized, with each minimum semantic unit as the starting semantic unit of the candidate named entity and the number of minimum semantic units included in each candidate named entity changing in sequence as a constraint, and sorting the representation tensor in a second dimension with each minimum semantic unit as the ending semantic unit of the candidate named entity and the number of minimum semantic units included in each candidate named entity changing in sequence as a constraint; The following S1-S3 are executed in a loop until the final fused tensor is obtained, where the first attention mechanism network and the second attention mechanism network used in each loop are different: S1: using a first attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a first aggregate tensor associated with each candidate named entity; and using a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a second aggregate tensor associated with each candidate named entity; S2: re-aggregate the first aggregation tensor and the second aggregation tensor corresponding to each candidate entity to obtain a re-representation tensor; S3: Recombining the re-representation tensors corresponding to the candidate named entities according to the sorting strategy of the two-dimensional sorting to obtain an intermediate fusion tensor or the final fusion tensor; The final fused tensor is processed by using a prediction network to obtain a recognized named entity, wherein the recognized named entity is a named entity in which the smallest semantic unit is positively sorted.

[0005] Optionally, before executing S1, the method further includes: Deleting the representation tensor corresponding to the super-long candidate named entity in the initial fused tensor; The super-long candidate named entity is a candidate named entity including a number of minimum semantic units exceeding a preset length, and the preset length is determined according to the windows of the first attention mechanism network and the second attention mechanism network.

[0006] Optionally, before each cyclic execution of S1, the method further includes: For the initial fusion tensor or the intermediate fusion tensor, align and sort the representation tensors corresponding to the candidate named entities composed of a minimum semantic unit in the second dimension, and keep the sorting of all the representation tensors in the initial fusion tensor or the representation tensors in the first dimension in the intermediate fusion tensor to adjust the position of the initial fusion tensor or the intermediate fusion tensor to obtain a first adjusted fusion tensor; The first attention mechanism network is used to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the intermediate fusion tensor, including: A first attention mechanism network is used to perform attention mechanism calculation on the representation tensors associated with candidate named entities having the same starting semantic unit in the first adjusted fusion tensor.

[0007] Optionally, before each execution of S1, the method further includes: Transposing the initial fused tensor or the intermediate fused tensor to obtain a transposed fused tensor; For the transposed fused tensor, align and sort the representation tensors or re-representation tensors corresponding to the candidate named entity composed of a minimum semantic unit on the second dimension, and keep the sorting of all representation tensors or re-representation tensors in the first dimension of the transposed fused tensor to adjust the position of the transposed fused tensor to obtain a second adjusted fused tensor; The use of a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor includes: using a second attention mechanism network to perform attention mechanism calculation on the representation tensor or re-representation tensor associated with the candidate named entities having the same ending and starting semantic units in the second adjusted fusion tensor.

[0008] Optionally, S3 includes: recombining the re-representation tensors corresponding to each candidate named entity according to the sorting strategy of the two-dimensional sorting to obtain a combined tensor; Processing the combined tensor using a convolutional neural network to obtain a re-representation tensor after convolution processing; The re-characterized tensors obtained after the convolution process are combined according to the sorting strategy of the two-dimensional sorting to form the intermediate fused tensor or the final fused tensor; The convolutional neural network used in each cycle of executing S3 is different.

[0009] Optionally, the encoding network is used to encode the text to be recognized to obtain an encoding feature tensor, including: Transcode each minimum semantic unit in the text to be recognized to obtain the corresponding transcoded tensor; According to the order of each minimum semantic unit in the text to be recognized, the corresponding transcoded tensor is sorted and the CLS tensor is added at the start or end position of the sorting to obtain a sorted tensor; The following S4-S7 are executed in a loop until the encoded feature tensor is obtained, wherein the global attention mechanism network and the local attention mechanism network used in each loop are different: S4: for the global coding tensor in the CLS tensor or the re-sorting tensor, a global attention mechanism network is used to perform global attention mechanism calculation on the sorting tensor or the re-sorting tensor to obtain a global coding tensor corresponding to the CLS tensor; S5: Perform sliding window sampling on the part of the sorted tensor except the CLS tensor to obtain a sampling tensor, and add the CLS tensor to the starting or ending position of the sampling tensor to obtain a rejoined tensor; or, perform sliding window sampling on the part of the rejoined tensor except the global encoding tensor to obtain a sampling tensor, and add the global encoding tensor to the starting or ending position of the sampling tensor to obtain a rejoined tensor; S6: using a local attention mechanism network to perform attention mechanism calculation on the re-concatenated tensor to obtain an intermediate encoding tensor corresponding to each minimum semantic unit; S7: Sort the intermediate encoding tensors corresponding to each minimum semantic unit and the global encoding tensor according to the order of the transcoded tensors and CLS tensors corresponding to each minimum semantic unit in the sorted tensor to obtain a re-sorted tensor; or, sort the intermediate encoding tensors corresponding to each minimum semantic unit to obtain the encoding feature tensor.

[0010] Optionally, S4 includes: using a LogN-Scaling mechanism when using a global attention mechanism network to perform attention mechanism calculation on the sorted tensor or the re-sorted tensor.

[0011] Optionally, the adopting a prediction network to process the final fused tensor to obtain an identified named entity includes: Processing the final fused tensor using a prediction network to obtain a prediction probability corresponding to each of the candidate named entities; The mean probability of two predicted probabilities corresponding to the two candidate named entities whose minimum semantic units are sorted in opposite directions is calculated, and the identified named entity is determined among the candidate named entities whose minimum semantic units are sorted in a positive direction according to all the mean probabilities.

[0012] Second method, the embodiment of the present disclosure provides a method for constructing a named entity recognition model, wherein the named entity recognition model includes an encoding network, a bi-affine network, a first attention mechanism network, a second attention mechanism network, and a prediction network; the model construction method includes: The sample text is encoded by using an encoding network to obtain an encoding feature tensor, wherein the encoding feature tensor includes encoding tensors that are sorted according to the minimum semantic units in the sample text and correspond to the minimum semantic units; A dual affine network is used to process the encoding feature tensor to obtain a representation tensor associated with all candidate named entities with each minimum semantic unit as the starting semantic unit and the ending semantic unit, and the representation tensors corresponding to each candidate named entity are sorted and combined in two dimensions to obtain an initial fusion tensor; wherein the two-dimensional sorting includes sorting the representation tensor in a first dimension according to the order of each minimum semantic unit in the sample text, with each minimum semantic unit as the starting semantic unit of the candidate named entity and making the number of minimum semantic units included in each candidate named entity change in sequence as a constraint, and sorting the representation tensor in a second dimension with each minimum semantic unit as the ending semantic unit of the candidate named entity and making the number of minimum semantic units included in each candidate named entity change in sequence as a constraint; The following S1-S3 are executed in a loop until the final fused tensor is obtained, where the first attention mechanism network and the second attention mechanism network used in each loop are different: S1: using a first attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a first aggregate tensor associated with each candidate named entity; and using a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a second aggregate tensor associated with each candidate named entity; S2: re-aggregate the first aggregation tensor and the second aggregation tensor corresponding to each candidate entity to obtain a re-representation tensor; S3: Recombining the re-representation tensors corresponding to the candidate named entities according to the sorting strategy of the two-dimensional sorting to obtain an intermediate fusion tensor or the final fusion tensor; Processing the final fused tensor using a prediction network to obtain a model recognition probability of each candidate named entity; Based on the model recognition probability of each candidate named entity and the label of the actual named entity of the sample text, the named entity recognition model is trained, and the above steps are repeated until the final named entity recognition model is obtained.

[0013] In a third aspect, an embodiment of the present disclosure provides a computing device, comprising a processor and a memory, wherein the memory is used to store a computer program; when the computer program is loaded by the processor, the processor executes the named entity recognition method as described above and / or the training method of the named entity recognition model as described above.

[0014] The disclosed embodiment scheme performs attention mechanism calculation on the representation tensor or re-representation tensor corresponding to the candidate named entities with the same starting semantic unit and the same ending semantic unit to obtain the corresponding first aggregation tensor and second aggregation tensor, and re-fuse the first aggregation tensor and the second aggregation tensor associated with each candidate named entity to obtain an intermediate fusion tensor until the final fusion tensor is obtained, and uses a prediction network to process the final fusion tensor to obtain the identified named entity. The aforementioned method enables the fusion processing network to mine the information of the candidate named entity from the perspective of forward reading and reverse reading, so that the final fusion tensor contains more accurate actual named entity information, and the corresponding accuracy of identifying the named entity can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work, including: Figure 1 is a schematic diagram of the architecture of the named entity recognition model adopted by the embodiment of the present disclosure; Figure 2 is a flow chart of a named entity recognition method provided by an embodiment of the present disclosure; Figure 3 is a flowchart of some embodiments of using an encoding network to process a text to be recognized to obtain an encoding feature tensor; Figure 4 is a flow chart of a method for constructing a named entity recognition model provided by an embodiment of the present disclosure; Figure 5 It is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] The term "including" and its variations used in this document are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. In this document, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0019] The embodiment of the present disclosure provides a new named entity recognition method. The new named entity recognition method processes the text to be recognized based on a named entity recognition model with a new architecture to obtain recognized named entities. The following first analyzes the named entity recognition model provided by the embodiment of the present disclosure, then analyzes the named entity recognition method based on the aforementioned named entity recognition model, and finally analyzes how to train and construct the named entity recognition model.

[0020] Figure 1 FIG. 1 is a schematic diagram of the architecture of the named entity recognition model 100 used in the embodiment of the present disclosure. Figure 1 As shown, the named entity recognition model 100 provided by the embodiment of the present disclosure includes an encoding network 101, a fusion processing network 102 and a prediction network 103, wherein the fusion processing network 102 includes a dual affine network 1021, a plurality of sub-fusion processing networks consisting of a first attention mechanism network 1022, a second attention mechanism network 1023, and an aggregation network 1024. The aforementioned networks are arranged according to Figure 1 The arrows shown in the figure are connected. In addition to the special sorting of the intermediate tensors processed by the corresponding network, the data is sorted according to Figure 1 The data flow direction shown implements the determination of the named entities to be recognized in the text to be recognized.

[0021] Figure 2 is a flow chart of the named entity recognition method provided by the embodiment of the present disclosure. Figure 2 As shown, the named entity recognition method provided by the embodiment of the present disclosure includes S210 to S250. The named entity recognition method provided by the embodiment of the present disclosure is executed by a computing device that deploys the aforementioned named entity recognition model.

[0022] S210: Use a coding network to encode the text to be recognized to obtain a coding feature tensor.

[0023] After receiving the text to be recognized, the computing device can encode the text to be recognized according to a pre-trained encoding network to obtain an encoding feature tensor, so that the encoding feature tensor includes the encoding tensors of each minimum semantic unit in the text to be recognized, and each encoding tensor is sorted according to the sorting order of the minimum semantic unit in the text to be recognized.

[0024] The aforementioned minimum semantic unit refers to a semantic unit that can no longer be segmented. For example, when the text to be recognized is composed of phonetic characters such as English, the minimum semantic unit is a word such as an English word that can no longer be segmented; when the text to be recognized is composed of pictographic characters such as Chinese characters, the minimum semantic unit is a single word such as a Chinese character that can no longer be segmented.

[0025] In a specific implementation, the encoding network can be any type of available encoding network, as long as the encoding feature tensor that meets the description in the previous paragraph can be obtained. For example, the encoding network can be an existing network based on a multi-head attention mechanism, such as the encoding network used in the Transformer architecture.

[0026] S220: Using a dual affine network to process the encoded feature tensor, obtain a representation tensor associated with all candidate named entities with each minimum semantic unit as the starting semantic unit and the ending semantic unit.

[0027] The bi-affine network constructs a tensor representation of all possible candidate named entities for the text passed in. For details on how the bi-affine network processes the encoding feature tensor and obtains the representation tensor associated with all candidate named entities, please refer to relevant technical literature such as "Named Entity Recognition as Dependency Parsing", which will not be analyzed here.

[0028] In order to more intuitively understand the operation result of S220 and to more conveniently understand the following content, an example of the result obtained by S220 is given here.

[0029] Assume that the text to be recognized is the six Chinese characters "entity tensor representation", and the smallest semantic units therein are respectively "real", "body", "tensor", "quantity", "table", and "representation". All candidate named entities composed of the aforementioned 6 smallest semantic units include "real", "entity", "entity tensor", "entity tensor representation", "body", "body tensor", "body tensor representation", "tensor", "tensor representation", "quantity", "quantity representation", "table", "table representation", "representation", "body real", "tensor body real", "quantity tensor body real", "table quantity tensor body real", "representation table quantity tensor body real", "tensor body", "quantity tensor body", "table quantity tensor body", "representation table quantity tensor body", "quantity tensor", "table quantity tensor", "representation table quantity tensor", "table quantity", "representation table quantity", "representation table" (a total of 36).

[0030] It should be noted here that when determining all candidate named entities with each smallest semantic unit as the ending semantic unit, the candidate named entities are determined in a reverse sorting manner. However, the aforementioned candidate named entities determined in reverse sorting will not be used as the recognized named entities determined in the last step, and their role is to better determine the recognized named entities through the operations in subsequent steps.

[0031] Here, the effect of this operation is analyzed: In practical applications, for nested entities, it may not be possible to well determine whether a candidate named entity nested by multiple entity names is a recognized entity through forward positioning (from the perspective of a person, the forward reading perspective). However, by jumping out of the human forward reading thinking habit and combining reverse positioning (from the perspective of a person, the reverse reading perspective), it is relatively easy to determine whether a candidate named entity nested by multiple entity names is a recognized entity. For example, for the nested entity "entity tensor", from the forward positioning perspective, it may be understood as three candidate named entities: "entity", "tensor", and "entity tensor", and it is not easy to determine which candidate entity is the truly recognized named entity from the forward positioning perspective. Correspondingly, by combining the reverse positioning method, the credibility (i.e., the recognition probability) of the candidate named entity "quantity tensor body real" is relatively high. It is precisely by combining the result that the credibility of the aforementioned reverse named entity is relatively high that it is easier to confirm that the forward named entity "entity tensor" is the recognized named entity. (This is only an example here and does not indicate that the example is correct and reasonable). Just like this, the embodiments of the present disclosure better realize the recognition of recognized named entities through the combined processing of forward candidate named entities and reverse candidate named entities, and specifically need to be combined with S240 - S250 in the following text. For the corresponding content, please refer to the following text.

[0032] It should also be noted that the above example of "entity tensor representation" including only 6 (very few) minimum semantic units is only for the convenience of illustrating the steps of the solution of this embodiment. In actual applications, the number of minimum semantic units included in the text to be recognized is much larger than the above example, and the number of corresponding candidate named entities will also increase in a quadratic manner.

[0033] Because the sorting order of each minimum semantic unit in the text to be recognized is 1, 2, 3, 4, 5, and 6, respectively, for convenience, the sorting order of the starting semantic unit and the ending semantic unit of each candidate named entity is used to represent the representation tensor of each candidate named entity. The representation tensors of the aforementioned candidate named entities are <1,1>, <1,2>, <1,3>, <1,4>, <1,5>, <1,6>, <2,2>, <2,3>, <2,4> respectively. , <2,5>, <2,6>, <3,3>, <3,4>, <3,5>, <3,6>, <4,4>, <4,5>, <4,6>, <5,5>, <5,6>, <6,6>, <2,1>, <3,1>, <4,1>, <5,1>, <6,1>, <3,2>, <4,2>, <5,2>, <6,2>, <4,3>, <5,3>, <6,3>, <5,4>, <6,4>, <6,5>.

[0034] S230: Perform two-dimensional sorting and combination on the representation tensors corresponding to each candidate named entity to obtain an initial fusion tensor.

[0035] In the disclosed embodiment, the representation tensors corresponding to the candidate named entities are sorted and combined in two dimensions according to the following sorting strategy: the representation tensors are sorted in the first dimension with each minimum semantic unit as the starting semantic unit of the candidate named entity and the number of minimum semantic units included in each candidate named entity is constrained to change sequentially; the representation tensors are sorted in the second dimension with each minimum semantic unit as the ending semantic unit of the candidate named entity and the number of minimum semantic units included in each candidate named entity is constrained to change sequentially.

[0036] The initial fusion tensor is a tensor formed by splicing the representation tensors in a specific order according to the aforementioned rule sorting strategy, and it is dimensionally expanded relative to the representation tensor.

[0037] Here, the rows in the two-dimensional table are taken as the first dimension mentioned above, and the columns in the two-dimensional table are taken as the second dimension mentioned above. An example is given of how to sort the 36 (6×6) representation tensors mentioned above to obtain the initial fusion tensor. Table 1 is a schematic table of the initial fusion tensor determined based on the 36 representation tensors. As shown in Table 1: (1) Taking each minimum semantic unit as the starting semantic unit of the candidate named entity as a constraint, the representation tensor corresponding to the candidate named entity included in each row is determined; (2) Taking each minimum semantic unit as the ending semantic unit of the candidate named entity as a constraint, the representation tensor corresponding to the candidate named entity included in each column is determined; (3) According to the constraint that the number of minimum semantic units included in the candidate named entity changes in the row and column in sequence, the representation tensors corresponding to the candidate named entity are sorted in rows and columns; (4) In order to avoid the fact that the number of minimum semantic units in the text to be recognized will not affect the determination of the sorting, the representation tensors on the diagonal of the two-dimensional table are the representation tensors corresponding to the candidate named entity consisting of only one minimum semantic unit, and the rows and columns of the diagonal representation tensors are the sorting order numbers of the corresponding minimum semantic units in the text to be recognized.

[0038] Table 1 is a schematic diagram of the initial fusion tensor determined based on 36 two-dimensional tensors Of course, in practical applications, the initial fused tensor obtained by performing two-dimensional rule sorting on the representation tensors corresponding to each candidate named entity is not limited to the results in the aforementioned Table 1, but can also be other results. However, considering that it is easier to understand to determine the starting semantic unit of each row and the ending semantic unit of each column according to the sorting order of each minimum semantic unit in the text to be recognized as shown in Table 1, it is preferred to determine the initial fused tensor as shown in Table 1 or the transposed method of Table 1 in practical applications.

[0039] S240: Execute S241-S243 in a loop until the final fused tensor is obtained.

[0040] It should be noted here that the number of times S241-S243 is executed in a loop is determined according to the number of fusion processing sub-networks included in the fusion processing network, and the first attention mechanism network and the second attention mechanism network used in each execution of S241-S243 are the networks of the corresponding fusion processing sub-models, and the same attention mechanism network is not always used.

[0041] S241: Using a first attention mechanism network to perform attention mechanism calculations on the representation tensors associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensors associated in the intermediate fusion tensor, to obtain a first aggregate tensor associated with each candidate named entity; and using a second attention mechanism network to perform attention mechanism calculations on the representation tensors associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor or the re-representation tensors associated in the intermediate fusion tensor, to obtain a second aggregate tensor associated with each candidate named entity.

[0042] With respect to the aforementioned Table 1, the first attention mechanism network is used to perform attention mechanism calculation on the representation tensors associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor, which may be performed by using the first attention mechanism network to perform attention mechanism calculation on each row; correspondingly, the second attention mechanism network is used to perform attention mechanism calculation on the representation tensors associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor, which may be performed by using the second attention mechanism network to perform attention mechanism calculation on each column.

[0043] In one embodiment, the window size of the first attention mechanism network is 5, that is, for one representation tensor, it uses five adjacent representation tensors in the same column, including itself, to perform attention calculations to obtain a first aggregate tensor; similarly, the window size of the second attention mechanism network is also 5, that is, for one representation tensor, it also uses five adjacent representation tensors in the same column, including itself, to perform attention calculations to obtain a second aggregate tensor.

[0044] S242: Re-aggregate the first aggregation tensor and the second aggregation tensor corresponding to each candidate entity to obtain a re-representation tensor.

[0045] In a specific implementation, after obtaining the first aggregation tensor and the second aggregation tensor corresponding to each candidate named entity, a multi-layer perceptron network can be used to process the first aggregation tensor and the second aggregation tensor to obtain a re-aggregated tensor.

[0046] S243: Recombining the re-representation tensors corresponding to the candidate named entities according to the two-dimensional sorting strategy to obtain an intermediate fusion tensor or a final fusion tensor.

[0047] The sorting strategy of the two-dimensional sorting is as analyzed in the previous S230, and will not be expanded here. After the re-aggregated tensors corresponding to each candidate named entity are sorted and combined in two dimensions, an intermediate fusion tensor or a final fusion tensor can be obtained.

[0048] Please note here that, when the aforementioned S241-S243 operations are executed for the first time, the first attention mechanism network and the second attention mechanism network process the representation tensor in the fusion tensor, and the intermediate fusion tensor is obtained by combining the representation tensors; except for the first and last time the aforementioned S241-S243 operations are executed, the first attention mechanism network and the second attention mechanism network process the re-representation tensor in the intermediate fusion tensor obtained last time, and the intermediate fusion tensor is obtained by combining the re-representation tensors; when the aforementioned S241-S243 operations are executed for the last time, the first attention mechanism network and the second attention mechanism network process the re-representation tensor in the intermediate fusion tensor obtained last time, and the final fusion tensor is obtained by combining the re-representation tensors.

[0049] S250: Use the prediction network to predict and process the final fused tensor to obtain the recognized named entity.

[0050] After obtaining the final fused tensor, the computing device may then process the final fused tensor using a prediction network to obtain the probabilities of each candidate named entity. Subsequently, based on the probabilities of each candidate named entity, it may be determined which candidate entities are used as recognized named entities. In a specific implementation, the prediction network may be a multi-layer perceptron network (it should be noted that the multi-layer perceptron network used here is not the same as the multi-layer perceptron network used in step S242 above). It should be noted here that the finally determined recognized named entities are the named entities in which the smallest semantic units are sorted in a forward direction, and the candidate named entities in which the smallest semantic units are sorted in a reverse direction are excluded.

[0051] In some embodiments, when the computing device determines that the confidence probability of a candidate named entity reaches a set probability standard, and the candidate named entity does not have a boundary conflict with a candidate named entity that meets the set probability standard, the candidate named entity can be determined as a recognized named entity. In the case where two candidate named entities that meet the set probability standard have a boundary conflict, the candidate named entity with a higher confidence probability can be determined as the recognized named entity.

[0052] As analyzed above, after obtaining the representation tensors of all candidate entities, the embodiment of the present disclosure performs attention mechanism calculation on the representation tensors or re-representation tensors corresponding to the candidate named entities with the same starting semantic unit and the same ending semantic unit to obtain the corresponding first aggregation tensor and second aggregation tensor, and re-fuse the first aggregation tensor and the second aggregation tensor associated with each candidate named entity to obtain an intermediate fusion tensor until the final fusion tensor is obtained, and uses a prediction network to process the final fusion tensor to obtain the identified named entity.

[0053] Since the candidate named entities identified in the embodiment of the present disclosure include candidate named entities in normal reading order and candidate named entities in reverse reading order, the representation tensors corresponding to the aforementioned candidate named entities are combined to form an initial fusion tensor and an intermediate fusion tensor, so that the fusion processing network mines the information of the candidate named entities from the perspective of forward reading and reverse reading, so that the final fusion tensor contains more accurate actual named entity information, and the corresponding accuracy of identifying named entities can also be improved.

[0054] It is precisely because of the aforementioned use of the first attention mechanism network and the second attention mechanism network to process the representation tensor in the initial fusion tensor and the re-representation tensor in the intermediate fusion tensor to obtain the final fusion tensor, and the identified named entities based on the final fusion tensor, that the named entity recognition method provided in the embodiment of the present disclosure performs well in processing nested entities and cross-sentence entity blocks, overcomes the limitations of the prior art in recognizing complex text structures, and significantly improves the ability to recognize named entities in complex texts.

[0055] Figure 3 Flowchart of some embodiments using the encoding network to process the text to be recognized to obtain the encoding feature tensor. Figure 3 As shown, in some embodiments, the computing device may process the text to be recognized using the following S211-S216 to obtain an encoded feature tensor.

[0056] S211: Transcode each minimum semantic unit in the text to be recognized to obtain a corresponding transcoded tensor.

[0057] The computing device pre-stores a one-to-one correspondence between most widely used minimum semantic units and the associated transcoding tensors. After obtaining the text to be recognized, the computing device determines each minimum semantic unit according to the distinction rules of each minimum semantic unit therein, and determines the transcoding tensor corresponding to each minimum semantic unit according to the aforementioned one-to-one correspondence. It should be noted here that the dimensions of the transcoding tensors corresponding to all minimum semantic units are the same.

[0058] S212: Sort the corresponding transcoded tensors according to the order of each minimum semantic unit in the text to be recognized and add the CLS tensor to the start or end position of the sort to obtain a sorted tensor.

[0059] The CLS tensor is a predefined tensor that represents the global information of the sorted tensor that needs to be aggregated. It should be noted here that the CLS tensor is either added before the sorted transcoded tensor or after the sorted transcoded tensor.

[0060] S213: Loop through S2131-S2134 until the encoded feature tensor is obtained.

[0061] It should be noted here that the number of times S2131-S2134 is executed in a loop is determined according to the number of sub-networks included in the encoding network, and the global attention mechanism network and the local attention mechanism network used in each execution of S2131-S2134 are not the same.

[0062] S2131: For the global encoding tensor in the CLS tensor or the re-sorted tensor, a global attention mechanism network is used to perform global attention mechanism calculation on the sorted tensor or the re-sorted tensor to obtain the global encoding tensor corresponding to the CLS tensor.

[0063] During the first execution of S2131, the computing device uses the global attention mechanism network in the first sub-network in the encoding network to perform global attention mechanism calculations on the sorting tensor to obtain a global encoding tensor corresponding to the CLS tensor. During the subsequent repeated execution of S2131, for the global encoding tensor, the computing device uses the global attention mechanism network in the corresponding sub-network to perform global attention mechanism calculations on the sorting tensor to obtain a subsequent global encoding tensor.

[0064] In practical applications, because the text length of the text to be recognized is random, the corresponding sorting tensor and re-sorting tensor length (relative to the sorting tensor and re-sorting tensor corresponding to other texts) are also random. In order to avoid the instability of the entropy of the attention score caused by the different lengths of the text to be recognized, that is, to keep the entropy of the attention score stable under different lengths of the text to be recognized (essentially, the attention mechanism score is not sensitive to the length of the sorting tensor or the re-sorting tensor, even if a few minimum semantic units are added to the text to be recognized, the calculated attention score will not change much), the global attention mechanism network can adopt the LogN-Scaling technology. Based on this, in the specific implementation, texts of various lengths to be recognized can be processed without considering the structural requirements of the model for the change in the number of minimum semantic units in the text to be recognized, thereby improving the encoding efficiency and accuracy. Considering that the LogN-Scaling technology is already a public technology, it will not be analyzed here. For specific content, please refer to the analysis of the prior art literature.

[0065] It should be noted that the use of LogN-Scaling technology in the global attention mechanism network is to ensure the stability and robustness of the model (that is, robustness), rather than being mainly used for length extrapolation of large language models as in the prior art. Of course, the use of LogN-Scaling in the global attention mechanism during the training phase of the named entity recognition model has a more obvious actual effect in ensuring the stability and robustness of the model.

[0066] S2132: Perform sliding window sampling on the part of the sorted tensor except the CLS tensor to obtain a sampling tensor, and add the CLS tensor to the starting or ending position of the sampling tensor to obtain a re-joined tensor; or, perform sliding window sampling on the part of the re-sorted tensor except the global encoding tensor to obtain a sampling tensor, and add the global encoding tensor to the starting or ending position of the sampling tensor to obtain a re-joined tensor.

[0067] During the first execution of S2132, a sliding window sampling is performed on the part of the sorted tensor except the CLS tensor to obtain a sampling tensor, and then the CLS tensor is added to the corresponding starting or ending position to obtain a rejoined tensor. During the subsequent execution of S2132, a sliding window sampling is performed on the part of the rejoined tensor except the global encoding tensor to obtain a sampling tensor, and the global encoding tensor is added to the starting or ending position of the sampling tensor to obtain a rejoined tensor.

[0068] S2133: Use the local attention mechanism network to perform attention mechanism calculation on the re-concatenated tensor to obtain the intermediate encoding tensor corresponding to each minimum semantic unit.

[0069] The local attention mechanism network is a network used to achieve local attention aggregation. It aggregates the attention of a preset window size for the transcoded tensor corresponding to a certain minimum semantic unit, and then obtains the local tensor corresponding to a single minimum semantic unit. According to the above idea, the size of the attention sliding window of the local attention mechanism network (the dimension of the transcoded tensor is not considered here, and only one transcoded tensor is treated as 1 unit) is smaller than the number of minimum semantic units included in the text to be recognized in most cases, and only in rare cases is it equal to or even smaller than the number of minimum semantic units included in the text to be recognized. In addition, in order to obtain local tensors for all minimum semantic units, when using the local attention mechanism network for attention mechanism calculation, the sliding window step size for the sorting tensor is 1 unit. S2314: Sort the intermediate encoding tensor and the global encoding tensor corresponding to each minimum semantic unit according to the order of the transcoded tensor and the CLS tensor corresponding to each minimum semantic unit in the sorted tensor to obtain a re-sorted tensor; or, sort the intermediate encoding tensor corresponding to each minimum semantic unit to obtain a coding feature tensor.

[0070] In the process except the last execution of S2314, the order of the transcoded tensor and the CLS tensor corresponding to each minimum semantic unit in the sorted tensor is compared, and the intermediate coding tensor and the global coding tensor corresponding to each minimum semantic unit are sorted to obtain a re-sorted tensor. In the process of the last execution of S2314, only the intermediate coding tensor corresponding to each minimum semantic unit is sorted to obtain the coding feature tensor.

[0071] Using the aforementioned method, the global attention mechanism is calculated only for the CLS tensor or the global encoding tensor, without performing global attention mechanism operations on each candidate named entity, thereby reducing the amount of calculation (also reducing the training cost during model training).

[0072] Of course, the embodiments of the present disclosure are not limited to the encoding network of the aforementioned architecture to process the text to be recognized, and existing technologies such as the encoding network in the Transformer architecture can also be used to process the text to be recognized to obtain the encoded feature tensor.

[0073] In actual applications, real named entities do not include too many minimum semantic units (that is, real named entities are not too long. If they are too long, they will be difficult to read). Based on the above ideas, named entities that contain too many minimum semantic units in candidate named entities are most likely not recognized named entities. Performing the above-mentioned first attention mechanism operation and second attention mechanism operation on the representation tensor corresponding to such candidate named entities will not only fail to obtain the corresponding correct results, but also cause a waste of computing resources.

[0074] To solve the problem mentioned in the previous paragraph, in some embodiments, before executing the aforementioned S240, the computing device may further execute the following S260.

[0075] S260: Delete the representation tensor corresponding to the super-long candidate named entity in the initial fusion tensor.

[0076] An over-long candidate named entity is a candidate named entity including a minimum semantic unit whose number exceeds a preset length, and the aforementioned preset length is determined according to the window length of the first attention mechanism network and the second attention mechanism network.

[0077] It is assumed here that the preset length is 3. For the 36 candidate named entities determined for "entity tensor representation", the deleted over-long candidate named entities include "entity tensor", "entity tensor table", "entity tensor representation", "body tensor table", "body tensor representation", "tensor representation", "quantity-to-body-real", "table-quantity-to-body-real", "table-table-quantity-to-body-real", "table-quantity-to-body-real", "table-quantity-to-body", "table-quantity-to-body", and "table-quantity-to-table".

[0078] Table 2 Schematic diagram of the initial fusion tensor obtained after deleting super-long named entities Correspondingly, S240 performs attention mechanism calculation on the initial fused tensor from which part of the representation tensor is deleted. In practical applications, the first attention mechanism network and the second attention mechanism network can be used to enter the attention mechanism budget only for the remaining representation tensors in the initial fused tensor. This can reduce the number of representation tensors that are actually calculated by the attention mechanism, thereby reducing the complexity of data calculation.

[0079] In some embodiments, the window sizes of the first attention mechanism network and the second attention mechanism network are set to a preset length. Assume that the number of the minimum semantic units in the text to be recognized is , the aforementioned preset length is , the computational complexity when executing the aforementioned S240 can be obtained from Reduce to In practical applications, due to Much greater than , which significantly reduces the computational complexity and reduces the computational cost.

[0080] In practical applications, the sizes of the initial fused tensor in the first and second dimensions are retained as In the case of , in order to be able to use the first attention mechanism network to process only the remaining representation tensors in the initial fusion tensor, different attention mechanism windows need to be used for different rows to calculate the representation tensors or re-representation tensors of each row, which complicates the calculation of the attention mechanism and is not conducive to the implementation of streaming processing. To solve this problem, in some embodiments of the present disclosure, after executing the aforementioned S280 and before executing S2140, the computing device can also execute the following S270 and S280.

[0081] S270: For the initial fusion tensor or the intermediate fusion tensor, align and sort the representation tensors corresponding to the candidate named entities composed of a minimum semantic unit in the second dimension, and keep the sorting of all representation tensors in the initial fusion tensor or the first dimension of the representation tensor in the intermediate fusion tensor to adjust the position of the initial fusion tensor or the intermediate fusion tensor to obtain a first adjusted fusion tensor.

[0082] In the above operation, before executing S241 for the first time, the above adjustment is performed on the representation tensor in the initial fusion tensor to obtain the first adjusted fusion tensor; before the subsequent execution of S241, the above adjustment is performed on the re-representation tensor in the intermediate fusion tensor to obtain the first adjusted fusion tensor.

[0083] As shown in Table 2, the corresponding representation tensors corresponding to the candidate named entities composed of a minimum semantic unit are aligned in the second dimension, that is, <1,1>, <2,2>, <3,3>, <4,4>, <5,5>, <6,6> are aligned to form the same column; the initial fusion tensor is adjusted while keeping the order of the first dimension of all representation tensors in the initial fusion tensor, that is, the row position adjustment is performed while keeping the same row order for each other representation tensor. Table 3 is a schematic table of the first adjustment fusion tensor.

[0084] Table 3 Schematic diagram of the first adjustment fusion tensor When the aforementioned first adjusted fusion tensor is obtained, the corresponding S241 uses the first attention mechanism network to perform attention mechanism calculations on the representation tensors associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensors associated in the intermediate fusion tensor to obtain the first aggregate tensor associated with each candidate named entity. Specifically, the first attention mechanism network is used to perform attention mechanism calculations on the representation tensors or re-representation tensors associated with the candidate named entities having the same starting semantic unit in the first adjusted fusion tensor to obtain the first aggregate tensor associated with each candidate named entity.

[0085] S280: For the initial fusion tensor or the intermediate fusion tensor, align and sort the representation tensor or re-representation tensor corresponding to the candidate named entity composed of a minimum semantic unit on the first dimension, and keep the sorting of all representation tensors in the second dimension in the initial fusion tensor, or keep the sorting of all representation tensors in the second dimension in the intermediate fusion tensor, adjust the position of the initial fusion tensor or the intermediate fusion tensor, and obtain a third adjusted fusion tensor.

[0086] As shown in Table 2, the corresponding representation tensors corresponding to the candidate named entities composed of a minimum semantic unit are aligned in the second dimension, that is, <1,1>, <2,2>, <3,3>, <4,4>, <5,5>, <6,6> are aligned to form the same row; the initial fusion tensor is adjusted while keeping the order of the first dimension of all representation tensors in the initial fusion tensor, that is, the row position adjustment is performed while keeping the order of the same column for other representation tensors. Table 4 is a schematic table of the third adjustment of the fusion tensor.

[0087] Table 4 Schematic diagram of the third adjustment fusion tensor When the aforementioned third adjusted fusion tensor is obtained, the corresponding S240 adopts a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same end semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor to obtain the second aggregate tensor associated with each candidate named entity. Specifically, the second attention mechanism network is used to perform attention mechanism calculation on the representation tensor or re-representation tensor associated with the candidate named entities having the same end semantic unit in the third adjusted fusion tensor, that is, the attention mechanism calculation is performed on each column separately to obtain the second aggregate tensor associated with each candidate named entity.

[0088] In the case of adopting the aforementioned S270, S280 and the corresponding improvement of S241, when executing the aforementioned S242 operation, it is also necessary to determine the reverse-ordered named candidate entities corresponding to each named candidate entity through tensor position retrieval, and then determine the spliced ​​first aggregate tensor and second aggregate tensor, resulting in a poor speed of the data splicing operation. To solve this problem, in some embodiments, the computing device does not execute the aforementioned 300, but executes the following S290-S300.

[0089] S290: Transpose the initial fused tensor to obtain a transposed fused tensor.

[0090] Table 5 is a schematic table of transposed fusion tensors. As shown in Table 5, the size of its rows and columns is the same as that of Table 1, and only the positions of each representation tensor have changed.

[0091] Table 5 Transposed fusion tensor diagram S300: for the transposed fused tensor, align and sort the representation tensors or re-representation tensors corresponding to the candidate named entity composed of a minimum semantic unit in the second dimension, and adjust the position of the transposed fused tensor while keeping the sorting of all the representation tensors or re-representation tensors in the first dimension in the transposed fused tensor to obtain a second adjusted fused tensor; The step of S300 processing the transposed fused tensor to obtain the second adjusted fused tensor is the same as the aforementioned S270, which will not be repeated here. Table 6 is a schematic table of the second adjusted fused tensor.

[0092] Table 6 Schematic diagram of the second adjustment fusion tensor When executing the aforementioned S290-S300, the aforementioned S241 uses a second attention mechanism network to perform attention mechanism calculations on the representation tensors associated with the candidate named entities having the same ending semantic units in the initial fusion tensor or the re-representation tensors associated in the intermediate fusion tensor, and obtains a second aggregate tensor associated with each candidate named entity, including: using a second attention mechanism network to perform attention mechanism calculations on the representation tensors or re-representation tensors associated with the candidate named entities having the same ending and starting semantic units in the second adjusted fusion tensor.

[0093] Comparing Table 3 and Table 6, the representation tensors at the same position of the first adjusted fusion tensor and the second adjusted fusion tensor are exactly the candidate named entities that are reversely sorted. The corresponding S243 respectively splices the first aggregate tensor associated with each candidate named entity and the second aggregate tensor associated with the corresponding reversely sorted candidate named entity, specifically: splicing the second aggregate tensor corresponding to each position of the second adjusted fusion tensor with the first aggregate tensor corresponding to each position of the first adjusted fusion tensor to obtain a spliced ​​tensor, and re-aggregating based on the spliced ​​tensor to obtain a re-representation tensor.

[0094] By transposing the initial fused tensor or the intermediate fused tensor into a transposed fused tensor and performing position adjustment processing to obtain a second adjusted fused tensor, the first aggregated tensor and the second aggregated tensor at the same position can be directly spliced ​​and aggregated to obtain a re-aggregated tensor. There is no need to retrieve the positions of the first aggregated tensor and the second aggregated tensor to be spliced ​​and aggregated, which facilitates streaming splicing processing and speeds up processing.

[0095] In some embodiments of the present invention, the re-characterization tensors of the corresponding steps may be combined according to the following S2431-S2432 to obtain an intermediate fusion tensor or a final fusion tensor.

[0096] S2431: Recombining the re-representation tensors corresponding to each candidate named entity according to the two-dimensional sorting strategy to obtain a combined tensor, S2432: Process the combined tensor using a convolutional neural network to obtain a re-representation tensor after convolution processing. S2433: The re-characterized tensors obtained after the convolution process are combined according to a two-dimensional sorting strategy to form an intermediate fusion tensor or a final fusion tensor.

[0097] By performing convolution processing on the combined tensor, the aggregate information of the re-representation tensors corresponding to the adjacent candidate named entities can be obtained, which is more conducive to the subsequent judgment of which candidate named entities are recognized named entities.

[0098] The aforementioned S270-S300 is more for processing from the perspective of accelerating streaming operations. In actual applications, the aforementioned S270-S300 may not be executed, but the final fused tensor may be obtained by processing the initial fused tensor or the intermediate fused tensor.

[0099] In some embodiments of the present disclosure, the computing device includes the following S251 - S252 when executing S250 .

[0100] S251: using the prediction network to process the final fusion tensor to obtain the prediction probability corresponding to each candidate named entity; S252: Calculate the mean probability of two predicted probabilities corresponding to two candidate named entities with opposite minimum semantic unit sorting, and determine the recognized named entity among the candidate named entities with positive minimum semantic unit sorting according to all mean probabilities.

[0101] As analyzed above, the candidate named entities whose minimum semantic units are sorted in reverse order will not be used as recognized named entities. However, they will still obtain corresponding prediction probabilities. However, from the perspective of the machine, the candidate named entities whose minimum semantic units are sorted in reverse order are equivalent to the candidate named entities whose minimum semantic units are sorted in forward order. Based on this, after obtaining the prediction probabilities corresponding to all candidate named entities, the embodiment of the present disclosure will cancel the two prediction probabilities corresponding to the two candidate named entities whose semantic units are sorted in opposite directions, and perform averaging to obtain the mean probability, and then determine the recognized named entity among the candidate named entities whose minimum semantic units are sorted in forward order based on all the mean probabilities.

[0102] In addition to providing the aforementioned named entity recognition method, the embodiment of the present disclosure also provides a method for constructing a named entity recognition model. Figure 4 is a flow chart of the method for constructing a named entity recognition model provided by an embodiment of the present disclosure. Figure 4 As shown, the method for constructing a named entity recognition model includes the following S410-S450.

[0103] S410: Use the encoding network to encode the sample text to obtain an encoding feature tensor.

[0104] The encoding feature tensor includes the encoding tensors of each minimum semantic unit in the sample text.

[0105] S420: Using a dual affine network to process the encoded feature tensor, obtain a representation tensor associated with all candidate named entities with each minimum semantic unit as a starting semantic unit and an ending semantic unit.

[0106] S430: Perform two-dimensional sorting and combination on the representation tensors corresponding to each candidate named entity to obtain an initial fusion tensor.

[0107] The aforementioned two-dimensional sorting includes sorting the representation tensor in the first dimension according to the order of each minimum semantic unit in the sample text, taking each minimum semantic unit as the starting semantic unit of the candidate named entity and making the number of minimum semantic units included in each candidate named entity change sequentially as a constraint, and sorting the representation tensor in the second dimension with each minimum semantic unit as the ending semantic unit of the candidate named entity and making the number of minimum semantic units included in each candidate named entity change sequentially as a constraint.

[0108] S440: Execute S441-S443 in a loop until the final fused tensor is obtained.

[0109] S441: using a first attention mechanism network to perform attention mechanism calculation on the representation tensors associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensors associated with the intermediate fusion tensor, to obtain a first aggregate tensor associated with each candidate named entity; and using a second attention mechanism network to perform attention mechanism calculation on the representation tensors associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor or the re-representation tensors associated with the intermediate fusion tensor, to obtain a second aggregate tensor associated with each candidate named entity; S442: re-aggregate the first aggregation tensor and the second aggregation tensor corresponding to each candidate entity to obtain a re-representation tensor; S443: recombining the re-representation tensors corresponding to the candidate named entities according to the sorting strategy of the two-dimensional sorting to obtain an intermediate fusion tensor or a final fusion tensor; S450: Using the prediction network to process the concatenated aggregation tensor in the final fusion tensor, obtain the model recognition probability of each candidate named entity.

[0110] The implementation process of the aforementioned S410-S450 is the same as the implementation process of the corresponding steps in the aforementioned recognition method, and the improved steps in the aforementioned named entity recognition method can also be replaced by the corresponding steps of the named entity recognition model construction method. It should be noted that the various networks used in the aforementioned S410-S450 are all untrained networks.

[0111] S460: Based on the model recognition probability of each candidate named entity and the label of the actual named entity in the sample text, the named entity recognition model is trained, and the above steps are repeated until the final named entity recognition model is obtained.

[0112] In the specific implementation, the process of model training based on the model recognition probability and the label of the actual named entity is the same as the prior art, and will not be analyzed here. In some embodiments, when executing S460, a low-rank adaptive method can be used to train the encoding network to reduce the computational cost of model training and prevent the model from overfitting and causing performance degradation.

[0113] In some embodiments, the aforementioned S410 may specifically include the following S411 - S413 .

[0114] S411: Perform transcoding processing on each minimum semantic unit in the text to be recognized to obtain a corresponding transcoded tensor.

[0115] S412: Randomly determine a selected semantic unit in the sample text, and use a mask tensor to randomly replace a transcoded tensor corresponding to the selected semantic unit as a new transcoded tensor corresponding to the selected semantic unit.

[0116] S413: Sort the corresponding transcoded tensors according to the sorting order of each minimum semantic unit in the text to be recognized and add the CLS tensor to the start or end position of the sorting to obtain a sorted tensor.

[0117] The aforementioned use of mask tensors to replace the corresponding transcoded tensors can make the trained model insensitive to noise and thus have better named entity recognition performance.

[0118] The embodiment of the present disclosure also provides a computing device for implementing the aforementioned method. Figure 5 Schematic diagram of the structure of the computing device provided by the embodiment of the present disclosure. Figure 5 , which shows a schematic diagram of the structure of a computing device 500 suitable for implementing the embodiments of the present disclosure. Figure 5 The computing device shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0119] like Figure 5 As shown, the computing device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory ROM 502 or a program loaded from a storage device 508 to a random access memory RAM 503. Various programs and data required for the operation of the computing device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output I / O interface 505 is also connected to the bus 504.

[0120] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a camera, a microphone, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication devices 509 may allow the computing device 500 to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The computing device 500 is shown with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0121] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0122] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable storage medium, a computer-readable signal medium, or any combination of the above two.

[0123] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0124] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] In some embodiments, the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (“LANs”). N ”), Wide Area Network (“WA N ”), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), and any network now known or developed in the future.

[0126] The computer-readable medium may be included in the computing device, or may exist independently without being incorporated into the computing device.

[0127] Computer program code for performing operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, S3, and Swift. m Alltalk, C++, and also conventional procedural programming languages, such as "C" or similar programming languages. The program code may execute entirely on the tester's computer, partially on the tester's computer, as a separate software package, partially on the tester's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the tester's computer through any type of network, including a local area network (LAN). N ) or Wide Area Network (WA N )—connects to the tester computer, or, can connect to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system based on hardware that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of the unit does not limit the unit itself in some cases. The functions described above in this article may be at least partially performed by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and hardware logic components such as CPUs, CPUs, and other hardware components. I C), application-specific standard products (ASSP), system on chip (SOC), complex programmable logic devices (CPLD), etc.

[0130] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for named entity recognition, characterized in that: include: Using an encoding network to encode the text to be recognized, and obtaining an encoding feature tensor, wherein the encoding feature tensor includes encoding tensors that are sorted according to the minimum semantic units in the text to be recognized and correspond to the minimum semantic units; A dual affine network is used to process the encoding feature tensor to obtain a representation tensor associated with all candidate named entities with each minimum semantic unit as the starting semantic unit and the ending semantic unit, and the representation tensors corresponding to each candidate named entity are sorted and combined in two dimensions to obtain an initial fusion tensor; wherein the two-dimensional sorting includes sorting the representation tensor in a first dimension according to the order of each minimum semantic unit in the text to be recognized, with each minimum semantic unit as the starting semantic unit of the candidate named entity and the number of minimum semantic units included in each candidate named entity changing in sequence as a constraint, and sorting the representation tensor in a second dimension with each minimum semantic unit as the ending semantic unit of the candidate named entity and the number of minimum semantic units included in each candidate named entity changing in sequence as a constraint; The following S1-S3 are executed in a loop until the final fused tensor is obtained, where the first attention mechanism network and the second attention mechanism network used in each loop are different: S1: using a first attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a first aggregation tensor associated with each candidate named entity; And, using a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same end semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a second aggregation tensor associated with each candidate named entity; S2: re-aggregate the first aggregation tensor and the second aggregation tensor corresponding to each candidate entity to obtain a re-representation tensor; S3: Recombining the re-representation tensors corresponding to the candidate named entities according to the sorting strategy of the two-dimensional sorting to obtain an intermediate fusion tensor or the final fusion tensor; The final fused tensor is processed by using a prediction network to obtain a recognized named entity, wherein the recognized named entity is a named entity in which the smallest semantic unit is positively sorted.

2. The method according to claim 1, characterized in that Before executing S1, the method further includes: Deleting the representation tensor corresponding to the super-long candidate named entity in the initial fused tensor; The super-long candidate named entity is a candidate named entity including a number of minimum semantic units exceeding a preset length, and the preset length is determined according to the windows of the first attention mechanism network and the second attention mechanism network.

3. The method according to claim 2, characterized in that Before each cyclic execution of S1, the method further includes: For the initial fusion tensor or the intermediate fusion tensor, align and sort the representation tensors corresponding to the candidate named entities composed of a minimum semantic unit in the second dimension, and keep the sorting of all the representation tensors in the initial fusion tensor or the representation tensors in the first dimension in the intermediate fusion tensor to adjust the position of the initial fusion tensor or the intermediate fusion tensor to obtain a first adjusted fusion tensor; The first attention mechanism network is used to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the intermediate fusion tensor, including: A first attention mechanism network is used to perform attention mechanism calculation on the representation tensors associated with candidate named entities having the same starting semantic unit in the first adjusted fusion tensor.

4. The method according to claim 3, characterized in that Before each execution of S1, the method further includes: Transposing the initial fused tensor or the intermediate fused tensor to obtain a transposed fused tensor; For the transposed fused tensor, align and sort the representation tensors or re-representation tensors corresponding to the candidate named entity composed of a minimum semantic unit on the second dimension, and keep the sorting of all representation tensors or re-representation tensors in the first dimension of the transposed fused tensor to adjust the position of the transposed fused tensor to obtain a second adjusted fused tensor; The use of a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor includes: using a second attention mechanism network to perform attention mechanism calculation on the representation tensor or re-representation tensor associated with the candidate named entities having the same ending and starting semantic units in the second adjusted fusion tensor.

5. The method according to claim 3, characterized in that S3 include: Recombining the re-representation tensors corresponding to the candidate named entities according to the two-dimensional sorting strategy to obtain a combined tensor; Processing the combined tensor using a convolutional neural network to obtain a re-representation tensor after convolution processing; The re-characterized tensors obtained after the convolution process are combined according to the sorting strategy of the two-dimensional sorting to form the intermediate fused tensor or the final fused tensor; The convolutional neural network used in each cycle of executing S3 is different.

6. The method according to any one of claims 1 to 5, characterized in that: The encoding network is used to encode the text to be recognized to obtain an encoding feature tensor, including: Transcode each minimum semantic unit in the text to be recognized to obtain the corresponding transcoded tensor; According to the order of each minimum semantic unit in the text to be recognized, the corresponding transcoded tensor is sorted and the CLS tensor is added at the start or end position of the sorting to obtain a sorted tensor; The following S4-S7 are executed in a loop until the encoded feature tensor is obtained, wherein the global attention mechanism network and the local attention mechanism network used in each loop are different: S4: for the global coding tensor in the CLS tensor or the re-sorting tensor, a global attention mechanism network is used to perform global attention mechanism calculation on the sorting tensor or the re-sorting tensor to obtain a global coding tensor corresponding to the CLS tensor; S5: Perform sliding window sampling on the part of the sorted tensor except the CLS tensor to obtain a sampling tensor, and add the CLS tensor to the starting or ending position of the sampling tensor to obtain a rejoined tensor; or, perform sliding window sampling on the part of the rejoined tensor except the global encoding tensor to obtain a sampling tensor, and add the global encoding tensor to the starting or ending position of the sampling tensor to obtain a rejoined tensor; S6: using a local attention mechanism network to perform attention mechanism calculation on the re-concatenated tensor to obtain an intermediate encoding tensor corresponding to each minimum semantic unit; S7: Sort the intermediate encoding tensors corresponding to each minimum semantic unit and the global encoding tensor according to the order of the transcoded tensors and CLS tensors corresponding to each minimum semantic unit in the sorted tensor to obtain a re-sorted tensor; or, sort the intermediate encoding tensors corresponding to each minimum semantic unit to obtain the encoding feature tensor.

7. The method according to claim 6, characterized in that S4 include: When a global attention mechanism network is used to perform attention mechanism calculation on the sorting tensor or the re-sorting tensor, a LogN-Scaling mechanism is used.

8. The method according to claim 1, characterized in that The adopting the prediction network to process the final fused tensor to obtain the recognized named entity includes: Processing the final fused tensor using a prediction network to obtain a prediction probability corresponding to each of the candidate named entities; The mean probability of two predicted probabilities corresponding to the two candidate named entities whose minimum semantic units are sorted in opposite directions is calculated, and the identified named entity is determined among the candidate named entities whose minimum semantic units are sorted in a positive direction according to all the mean probabilities.

9. A method for constructing a named entity recognition model, characterized in that: The named entity recognition model includes an encoding network, a bi-affine network, a first attention mechanism network, a second attention mechanism network and a prediction network; the model construction method includes: The sample text is encoded by using an encoding network to obtain an encoding feature tensor, wherein the encoding feature tensor includes encoding tensors that are sorted according to the minimum semantic units in the sample text and correspond to the minimum semantic units; A dual affine network is used to process the encoding feature tensor to obtain a representation tensor associated with all candidate named entities with each minimum semantic unit as the starting semantic unit and the ending semantic unit, and the representation tensors corresponding to each candidate named entity are sorted and combined in two dimensions to obtain an initial fusion tensor; wherein the two-dimensional sorting includes sorting the representation tensor in a first dimension according to the order of each minimum semantic unit in the sample text, with each minimum semantic unit as the starting semantic unit of the candidate named entity and making the number of minimum semantic units included in each candidate named entity change in sequence as a constraint, and sorting the representation tensor in a second dimension with each minimum semantic unit as the ending semantic unit of the candidate named entity and making the number of minimum semantic units included in each candidate named entity change in sequence as a constraint; The following S1-S3 are executed in a loop until the final fused tensor is obtained, where the first attention mechanism network and the second attention mechanism network used in each loop are different: S1: using a first attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same starting semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a first aggregate tensor associated with each candidate named entity; and using a second attention mechanism network to perform attention mechanism calculation on the representation tensor associated with the candidate named entities having the same ending semantic unit in the initial fusion tensor or the re-representation tensor associated in the intermediate fusion tensor, to obtain a second aggregate tensor associated with each candidate named entity; S2: re-aggregate the first aggregation tensor and the second aggregation tensor corresponding to each candidate entity to obtain a re-representation tensor; S3: Recombining the re-representation tensors corresponding to the candidate named entities according to the sorting strategy of the two-dimensional sorting to obtain an intermediate fusion tensor or the final fusion tensor; Processing the final fused tensor using a prediction network to obtain a model recognition probability of each candidate named entity; Based on the model recognition probability of each candidate named entity and the label of the actual named entity of the sample text, the named entity recognition model is trained, and the above steps are repeated until the final named entity recognition model is obtained.

10. A computing device, characterized in that It includes a processor and a memory, wherein the memory is used to store a computer program; when the computer program is loaded by the processor, the processor executes the named entity recognition method as described in any one of claims 1 to 8 and / or the training method of the named entity recognition model as described in claim 9.

Images