Named entity recognition method and apparatus, terminal device, and storage medium

Abstract

The application is suitable for the technical field of computers, and provides a named entity recognition method, including: acquiring a to-be-recognized text, and converting the to-be-recognized text into a first matrix of n*k dimensions; performing multi-layer convolution layer convolution on the first matrix, wherein the last convolution layer of a convolution kernel in the multi-layer convolution layer has a channel number of m, four convolution operations are performed on the last convolution layer, and four parallel second matrices of n*m dimensions are obtained; performing attention weight self-adaption on three second matrices of the four second matrices to obtain a third matrix of n*m dimensions, performing matrix addition on the third matrix and the remaining one second matrix, and outputting a fourth matrix of n*m dimensions; performing classification on the fourth matrix, and outputting an entity label corresponding to the to-be-recognized text; and outputting a named entity corresponding to the to-be-recognized text according to the entity label. By introducing an attention mechanism in the convolution layer, data redundancy is effectively reduced, the number of model parameters is reduced, and the recognition speed of the named entity recognition model is accelerated.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and particularly relates to a named entity recognition method and device, a terminal device, and a storage medium. BACKGROUND

[0002] Named entity recognition is a process of identifying entities with specific meanings from a piece of natural language text. At present, named entity recognition models mainly include traditional deep learning models and language models. The traditional deep learning model mainly uses a recurrent neural network, and the speed of this type of model is limited by its recurrent characteristics, resulting in slow recognition speed. The language model is a comprehensive language model trained through multiple tasks, but the large number of parameters of this type of model also leads to slow recognition speed.

[0003] Application Content

[0004] The present application provides a named entity recognition method, device, terminal device, and storage medium, which can solve the problem of slow recognition speed of existing named entity recognition models.

[0005] In a first aspect, the present application provides a named entity recognition method, comprising:

[0006] Obtaining a to-be-recognized text, and converting the to-be-recognized text into a first matrix of n x k dimensions, wherein n is the number of characters of the to-be-recognized text, and k is a preset dimension of a word vector corresponding to the to-be-recognized text;

[0007] Performing multi-layer convolution layer convolution on the first matrix, wherein the last convolution layer with a channel number of m in the multi-layer convolution layer is subjected to four convolution operations to obtain four parallel second matrices of n x m dimensions;

[0008] After performing attention weight self-adaption on three of the four second matrices, a third matrix of n x m dimensions is obtained, and the third matrix is subjected to matrix addition with the remaining one second matrix to output a fourth matrix of n x m dimensions;

[0009] Classifying the fourth matrix to output probabilities of multiple entity labels corresponding to the fourth matrix, and determining an entity label corresponding to the to-be-recognized text according to the multiple probabilities;

[0010] According to the entity label corresponding to the to-be-recognized text, a named entity corresponding to the to-be-recognized text is output.

[0011] The embodiment of the present application converts the to-be-recognized text into a first matrix, performs multi-layer convolution on the first matrix in a convolution layer to obtain four parallel second matrices, and introduces an attention mechanism in the convolution layer, to obtain a fourth matrix in a feature set according to the four second matrices, so that the attention mechanism is used to replace the pooling process, the to-be-recognized text is in the feature set, and the pooling process effectively reduces data redundancy, reduces the amount of model parameters, and speeds up the recognition speed of the named entity recognition model.

[0012] In a second aspect, the embodiment of the present application provides a named entity recognition device, comprising:

[0013] An acquisition module is configured to acquire a to-be-recognized text, and convert the to-be-recognized text into a first matrix of n x k dimensions, where n is the number of characters of the to-be-recognized text, and k is a preset dimension of a word vector corresponding to the to-be-recognized text.

[0014] A convolution module is configured to perform multi-layer convolution on the first matrix, where the last convolution layer with a channel number of m in the multi-layer convolution layer is subjected to four convolution operations to obtain four parallel second matrices of n x m dimensions.

[0015] An attention module is configured to obtain a third matrix of n x m dimensions by performing attention weight self-adaptation on three second matrices of the four second matrices, and perform matrix addition on the third matrix and the remaining one second matrix to output a fourth matrix of n x m dimensions.

[0016] A classification module is configured to classify the fourth matrix, output probabilities of multiple entity labels corresponding to the fourth matrix, and determine an entity label corresponding to the to-be-recognized text according to the multiple probabilities.

[0017] An output module is configured to output a named entity corresponding to the to-be-recognized text according to the entity label corresponding to the to-be-recognized text.

[0018] In a third aspect, the embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the named entity recognition method in any one of the first aspect.

[0019] In a fourth aspect, the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the named entity recognition method in any one of the first aspect.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, causes the terminal device to perform the named entity recognition method in any one of the first aspect.

[0021] It can be understood that the beneficial effects of the second aspect to the fifth aspect described above can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is a schematic diagram of a named entity recognition model provided by an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of a convolution operation applicable to the named entity recognition method provided by an embodiment of the present application;

[0025] Figure 3 is a flowchart of the named entity recognition method provided by an embodiment of the present application;

[0026] Figure 4 is a flowchart of the named entity recognition method provided by another embodiment of the present application;

[0027] Figure 5 is a flowchart of the named entity recognition method provided by another embodiment of the present application;

[0028] Figure 6 is a flowchart of the named entity recognition method provided by another embodiment of the present application;

[0029] Figure 7 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of an application scenario provided by another embodiment of the present application;

[0031] Figure 9 is a structural schematic diagram of a named entity recognition device provided by an embodiment of the present application;

[0032] Figure 10 is a structural schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0034] It is to be understood that the terminology "includes", "has", "holds", "contains" and / or "comprising", "including", "having" and their conjugates, as used herein, means "including but not limited to", and not to the exclusion of any other term or aspect.

[0035] It is also to be understood that the terminology "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items, and that the term "at least one of A and B" is equivalent to "A, B, or A and B".

[0036] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon [the described condition or event] being detected" or "in response to [the described condition or event] being detected", depending on the context.

[0037] In addition, the terms "first", "second", "third", etc. as used in the description of the application and the appended claims are not used to denote or imply relative importance but are used to distinguish one element from another.

[0038] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in other embodiments" or "in still other embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment, unless otherwise specified. Furthermore, the terms "comprise", "comprises", "comprising", "include", "includes", "including" and the like are synonymous with the term "contain", "contains", "containing" or "has", "has", "having" and are used in the sense of "including but not limited to", unless otherwise specified.

[0039] As introduced in the background, the named entity recognition model based on the recurrent neural network is limited by its recurrent characteristics, resulting in slow recognition speed of the named entity recognition method, and the parameter quantity of the named entity recognition model based on the language model is large, resulting in slow recognition speed.

[0040] Therefore, there is a need for a named entity recognition method to introduce an attention mechanism in a convolutional neural network model, so that the vector matrix corresponding to the text to be recognized has the characteristics of fast recognition speed and high recognition accuracy compared with the recurrent neural network, and has the characteristics of lightweight compared with the language model.

[0041] Figure 1 A schematic diagram of a named entity recognition model applied to the named entity recognition method is shown, and the named entity recognition model is based on a convolutional neural network. As shown in Figure 1 The model includes an input layer, a word embedding layer, a convolutional layer, and an output layer, wherein the convolutional layer includes multiple convolutional layers and an attention layer. Those skilled in the art can understand that Figure 1 The model structure shown in the above does not constitute the only limitation of the named entity recognition model of the embodiments of the present application, and it should be understood that there can be a fully connected layer and a softmax layer between the attention layer and the output layer, or there can be a CRF (Conditional Random Field) layer, or there can be other hidden layers, which will not be described here.

[0042] The word embedding layer is used to convert the text to be recognized into a vector matrix, which is used as the input of the convolutional layer. The model algorithm of the word embedding layer includes but is not limited to the word2vec algorithm or the GloVe algorithm.

[0043] The multiple convolutional layers are all ordinary convolutional layers, including a first convolutional layer, a second convolutional layer, …, and an i-th convolutional layer. The input of the first convolutional layer is the output of the word embedding layer, and the input of the second convolutional layer to the i-th convolutional layer is the output of the convolutional layer above. The multiple convolutional layers use convolution kernels to perform convolution operations on the input of the corresponding convolutional layer. Figure 2 A schematic diagram of the convolution operation provided by the embodiments of the present application is shown, as shown in Figure 2 A is a 3x3 convolution kernel, B is the input of the convolutional layer, i.e., a 6x6 vector matrix, each convolution kernel parameter (parameter in each small grid, such as 1 or -1) of the convolution kernel A is multiplied by the parameter at the corresponding position in the input B of the convolutional layer, and then the parameter at the corresponding position in the result C is obtained. After taking the average value of all parameters in the result C, a one-time sliding window result in the sliding window result D is obtained. After the convolution kernel A performs multiple sliding window operations on the input B of the convolutional layer, the convolution result of the convolutional layer is obtained.

[0044] The attention layer is an implicit layer into which the attention mechanism is introduced into the convolution layer, which is used to adaptively perform attention weight on the vector matrix output by the i-th layer to extract features and make the features concentrated.

[0045] The embodiment of the present application provides a named entity recognition method, which can be applied to a terminal device and can also be a separate application program, the application program can realize the process that the to-be-identified text is converted into a first matrix, the first matrix is subjected to multi-layer convolution, and then attention weight adaptation is performed to obtain a fourth matrix, and entity labeling is obtained according to the fourth matrix. Exemplarily, the terminal device can be a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a desktop computer, a stand-alone server, a cluster server or the like. The embodiment of the present application does not limit the specific type of the terminal device.

[0046] Figure 3 A schematic flowchart of the named entity recognition method provided by the present application is shown, which is used as an example but not limited to the method, which can be applied to the terminal device. As shown in Figure 3 the method includes S301-S305.

[0047] S301, obtaining a to-be-identified text, and converting the to-be-identified text into a first matrix of n*k dimensions, wherein n is the number of characters of the to-be-identified text, and k is a preset dimension of a word vector of the to-be-identified text.

[0048] The to-be-identified text contains named entities, and the named entities include names of persons, institutions, places and all other entities identified by names, such as names of plants, animals, medicines and other entities identified by names.

[0049] In a possible implementation manner, the word embedding layer in the named entity recognition model is pre-trained by using a word2vec algorithm based on a Chinese Wikipedia dataset, and then the to-be-identified text is converted into a word vector of k dimensions, i.e., a first matrix of n*k dimensions, by using the word2vec algorithm completed by the word vector pre-training, wherein the dimension k of the word vector can be preset.

[0050] S302, performing multi-layer convolution on the first matrix, wherein the last convolution layer in the multi-layer convolution layer is subjected to 4 convolution operations to obtain 4 parallel second matrices of n*m dimensions.

[0051] The input of each convolutional layer in the multi-layer convolutional layer is the output of the previous convolutional layer. The four second matrices are vector matrices of the same dimension. Alternatively, four parallel convolutional layers are constructed in the last layer of the multi-layer convolutional layer, the four parallel convolutional layers are convolved by convolution kernels of the same dimension, and each outputs one second matrix. It should be understood that each convolution kernel has the same dimension, but the convolution kernel parameters are different.

[0052] For example, the multi-layer convolutional layer is a 5-layer convolutional layer, and based on the output of the 4th layer convolutional layer, four parallel 5th layer convolutional layers are constructed, the input of the parallel 5th layer convolutional layers is the output of the 4th layer convolutional layer, and the four parallel 5th convolutional layers are convolved by convolution kernels of the same dimension. The output of the 4th layer convolutional layer is convolved, and finally four parallel second matrices are output.

[0053] S303, the attention weight of three of the four second matrices is adaptively obtained, and a third matrix of n x m dimension is obtained. The third matrix is added to the remaining one second matrix to output a fourth matrix of n x m dimension.

[0054] The second matrix is obtained by convolving the first matrix by convolution kernels with different parameters and the same dimension, so as to obtain second matrices with different attention. Then, the attention weight of three of the second matrices is adaptively obtained, so that the third matrix contains the attention features of the three second matrices, so as to concentrate the attention features.

[0055] S304, classifying the fourth matrix, outputting the probability of multiple entity labels corresponding to the fourth matrix, and determining the entity label corresponding to the to-be-identified text according to the multiple probabilities.

[0056] The classification is to predict the probability of different entity label results output by the fourth matrix, and to determine the output entity label result according to the predicted probability. Specifically, the fourth matrix can be input into a classifier, and the classifier outputs the entity label corresponding to the to-be-identified text. The classifier includes but is not limited to a Logistic classifier, a softmax classifier or a conditional random field classifier.

[0057] Alternatively, the output of the entity label is in the form of BIOES, where B is the beginning of the entity, I is the middle of the entity, O is the non-entity, E is the end of the entity, and S is the single word entity. In practical applications, there are L types of named entities, so there are 5 x L possible entity label results.

[0058] S305, according to the entity label corresponding to the to-be-identified text, output the named entity corresponding to the to-be-identified text.

[0059] The entity annotation corresponds to the named entity. For example, the text to be identified is "Sanxia Travel Agency organizes Xiaoming and his family to travel to Sanxia", and the named entity is obtained according to the entity annotation, as shown in the following table:

[0060]

[0061] ORG represents an organization name entity, PER represents a person name entity, and LOC represents a place name entity. It should be understood that when an entity is identified by other names, other characters can be used to represent the corresponding entity, such as PLA representing an entity identified by a plant name. It should also be understood that the above characters such as ORG are only used for explanation and do not limit the specific implementation means of the present application.

[0062] The embodiment of the present application uses a convolutional neural network model as a basic model, so that the named entity recognition model has the characteristics of light weight, and the recognition speed of the named entity recognition method is faster. After converting the text to be identified into a first matrix, a plurality of layers of convolution are performed on the first matrix in the convolution layer to obtain four parallel second matrices, and an attention mechanism is introduced in the convolution layer, and a fourth matrix in the feature set is obtained according to the four second matrices, which effectively reduces data redundancy, reduces the number of model parameters, and speeds up the recognition speed of the named entity recognition model.

[0063] On the basis of the embodiment shown in Figure 3 The present application also provides another embodiment of a named entity recognition method. The above S301 further includes S3011. It should be noted that the same steps as the Figure 2 embodiment are not repeated here, please refer to the foregoing.

[0064] S3011, a preset g-dimensional position vector is added to the n*k-dimensional first matrix to obtain an n*(k+g)-dimensional first matrix with added position vectors.

[0065] The embodiment adds a g-dimensional position vector to the n*k-dimensional first matrix to supplement the sequence interaction information of the Chinese text, so that the convolutional neural network can extract the sequence position relationship of the first matrix. Figure 7 A schematic diagram of adding a position vector is shown, as shown in Figure 7 A 2-dimensional position vector is added to the 3*3-dimensional first matrix to obtain a 3*5-dimensional first matrix.

[0066] On the basis of the embodiment shown in Figure 3 Figure 4 A flowchart of another named entity recognition method of the embodiment of the present application is shown. As shown in Figure 4 The above step S302 specifically includes steps S401 and S402. It should be noted that the same steps as the Figure 4 ​The same steps as in the embodiments are not described here again, please refer to the foregoing.

[0067] S401, after the i-layer convolution layer convolution of the first matrix, a fifth matrix of n*m dimensions is obtained, and the channel number of the convolution kernel of each convolution layer in the i-layer convolution layer is m;

[0068] S402, the fifth matrix is convolved 4 times by a preset first convolution kernel of j*m dimensions, and 4 parallel n*m dimensional second matrices are output, and the channel number of the first convolution kernel is m.

[0069] In this embodiment, the first matrix is subjected to i-layer convolution by different convolution kernels, and a fifth matrix of n*m dimensions is output, and the fifth matrix is subjected to 4 times of convolution by a first convolution kernel of j*m dimensions, and 4 second matrices of the same dimension are output. Wherein, the vector dimension m of the first convolution kernel is the same as the vector dimension m of the first matrix, the convolution kernel parameters of the first convolution kernel are different each time, i>=2, for example, i=3, too large i value is easy to cause the calculation amount to increase, too small i value may lead to the characteristics of the obtained second matrix not concentrated, j can be 3, and the dimension of the first convolution kernel is 3*m.

[0070] The n*m dimensional matrix is convolved by the m channel first convolution kernel of 3*m dimensions, and m n*1 dimensional matrices are obtained, and a n*m dimensional matrix is obtained after splicing. Figure 8 A schematic diagram of a first convolution kernel performing sliding window convolution on a fifth matrix is shown. As shown in Figure 8 The dimension of the first matrix b is 6*6, the first convolution kernel has 6 channels (it should be understood that the convolution kernel parameters of each channel can be different or the same, and the specific parameters are obtained by training), a is one of the channels of the first convolution kernel, and one result "1" of the sliding window result c is obtained after one sliding window calculation of the first matrix b by one of the channels of the first convolution kernel a. After 6 times of downward sliding window calculation, a convolution result is finally obtained, and then 6 convolution results of 6*1 dimensions are obtained by 6 channels, and a second matrix of 6*6 dimensions is obtained after splicing.

[0071] On the basis of the embodiment shown in Figure 4 As shown in the embodiment, Figure 5 A flowchart of another named entity recognition method according to an embodiment of the present application is shown. As shown in Figure 5 The step S401 specifically includes steps S501 and S502. It should be noted that the same steps as in the embodiments are not described here again, please refer to the foregoing. Figure 4 The same steps as in the embodiments are not described here again, please refer to the foregoing.

[0072] S501, performing convolution on the first matrix by a preset second convolution kernel of 1xk dimensions, outputting a sixth matrix of nxm dimensions, a channel number of the second convolution kernel being m;

[0073] The second convolution kernel has dimensions of 1xk and a channel number of m, and after the first matrix of n x k dimensions is convolved, m convolution results of n x 1 dimensions are obtained, and the sixth matrix of n x m dimensions is obtained after splicing. It should be understood that when the first matrix is the first matrix of n x (k+g) dimensions after the position vector is added, the dimensions of the second convolution kernel are 1x(k+g).

[0074] S502, performing convolution on the fifth matrix by a second convolution layer to an i-th convolution layer, outputting a fifth matrix of n x m dimensions, wherein the convolution kernels of the second convolution layer to the i-th convolution layer are all the first convolution kernel.

[0075] The input of the second convolution layer to the i-th convolution layer is the output of the previous convolution layer, that is, the input of the second convolution layer is the fifth matrix, and the input of the third convolution layer is the output of the second convolution layer.

[0076] In Figure 3 on the basis of the embodiment shown in the figure, Figure 6 Another named entity recognition method of the embodiment of the present application is shown. As Figure 6 shown, the above-mentioned step S303 specifically includes steps S601 to S603. It should be noted that the same steps as the embodiment are not described here, please refer to the foregoing. Figure 3 The same steps as the embodiment are not described here, please refer to the foregoing.

[0077] S601, performing matrix multiplication on the first second matrix and the transpose matrix of the second second matrix, obtaining a seventh matrix of n x n dimensions;

[0078] S602, performing matrix multiplication on the seventh matrix and the third second matrix, obtaining the third matrix of n x m dimensions;

[0079] S603, performing matrix addition on the third matrix and the fourth second matrix, obtaining the fourth matrix of n x m dimensions.

[0080] In the embodiment, the attention mechanism is introduced in the convolution layer, that is, the matrix operation is performed on the four parallel second matrices in the i+1 convolution layer. Specifically, the first second matrix is multiplied by the transposed matrix of the second second matrix to obtain a seventh matrix of n*n dimension, the seventh matrix is multiplied by the third second matrix to obtain a third matrix of n*m dimension, and the third matrix of n*m dimension is added with the fourth second matrix to obtain a fourth matrix, and the fourth matrix is the final matrix adjusted by the attention mechanism. By introducing the attention mechanism in the convolution layer, the model features are concentrated, thereby improving the recognition accuracy and speed of the model.

[0081] For example: four parallel second matrices, Y1, Y2, Y3 and Y4 (each second matrix has different matrix parameters and the same dimension), the fourth matrix is Y=(Y1*Y2 T )*Y3+Y4, wherein is the transposed matrix of the second n*m dimension second matrix, the obtained matrix has a dimension of n*n, the obtained matrix has a dimension of n*m. the attention of the matrix features is concentrated, Y3 is concentrated by the attention, the matrix obtained by adjusting the result is also concentrated. It should be understood that the meanings of the symbols "*" and "x" are the same.

[0082] Based on the embodiment shown in Figure 3 The application provides another embodiment of the named entity recognition method. The above step S304 specifically includes step S304a. It should be noted that the same steps as Figure 3 embodiments are not described here, please refer to the foregoing.

[0083] S304a, input the fourth matrix into the conditional random field model, output the probability of multiple entity labels corresponding to the fourth matrix, and determine the entity standard corresponding to the text to be recognized according to the multiple probabilities.

[0084] In the embodiment, the CRF layer is constructed on the basis of the output of the convolution layer. The above conditional random field model (CRF model) follows the Markovian probability graph model. In the embodiment, the input of the CRF is the fourth matrix, the output of the CRF is the entity label of the text to be recognized, and the model algorithm of the CRF model is:

[0085]

[0086] wherein, W and b are model parameters obtained by model training, y' is a possible entity annotation result, and p is a probability of outputting y'. It should be understood that the model algorithm of the CRF model is only used for explanation and does not limit the specific implementation means of the present application.

[0087] By classifying the fourth matrix through the CRF model, the recognition accuracy of the named entity recognition can be improved by using the context classification result of the CRF.

[0088] In the embodiment shown in the figure, the fourth matrix is input into the full connection layer, and then the softmax function is used to output the probabilities of a plurality of entity annotations corresponding to the fourth matrix. The entity annotation corresponding to the text to be recognized is determined according to the plurality of probabilities. Figure 3 Based on the embodiment shown in the figure, the present application provides another embodiment of a named entity recognition method. The above step S304 specifically includes step S304b. It should be noted that the same steps as the embodiment are not described here again, please refer to the foregoing. Figure 3 The same steps as the embodiment are not described here again, please refer to the foregoing.

[0089] S304b, the fourth matrix is input into the full connection layer, and then the softmax function is used to output the probabilities of a plurality of entity annotations corresponding to the fourth matrix. The entity annotation corresponding to the text to be recognized is determined according to the plurality of probabilities.

[0090] In the embodiment, on the basis of the convolution layer, a feedforward full connection layer is constructed according to the fourth matrix to complete full connection, and a softmax layer is constructed on the basis of the full connection layer to classify the output of the full connection layer.

[0091] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0092] Corresponding to the named entity recognition method described in the above embodiment, Figure 9 The structure block diagram of the named entity recognition device 900 provided by the embodiment of the present application is shown, only the part related to the embodiment of the present application is shown for the convenience of description.

[0093] Referring to Figure 9 The device comprises:

[0094] The acquisition module 901 is configured to acquire a text to be recognized, and convert the text to be recognized into a first matrix of n*k dimensions, wherein n is the number of characters of the text to be recognized, and k is a preset dimension of a word vector corresponding to the text to be recognized.

[0095] The convolution module 902 is configured to perform multi-layer convolution on the first matrix, wherein the last convolution layer in the multi-layer convolution layer is subjected to 4 times of convolution operation, and 4 parallel second matrices of n*m dimensions are obtained.

[0096] Attention module 903 is used to adaptively apply attention weights to three of the four second matrices to obtain an n×m dimensional third matrix, and then perform matrix addition on the third matrix and the remaining one second matrix to output an n×m dimensional fourth matrix.

[0097] The classification module 904 is used to classify the fourth matrix, output the probability of multiple entity labels corresponding to the fourth matrix, and determine the entity label corresponding to the text to be identified based on the multiple probabilities.

[0098] The output module 905 is used to output the named entity corresponding to the text to be identified based on the entity annotation corresponding to the text to be identified.

[0099] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0100] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0101] Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 10 As shown, the terminal device 100 of this embodiment includes: at least one processor 1000 ( Figure 10 (Only one is shown) a processor, a memory 1001, and a computer program 1002 stored in the memory 1001 and executable on the at least one processor 1000, wherein the processor 1000 executes the computer program 1002 to implement the steps in any of the above-described named entity recognition method embodiments.

[0102] The terminal device 100 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal device can include, but is not limited to, a processor 1000 and a memory 1001. Those skilled in the art can understand that Figure 10 The terminal device 100 is only an example and does not limit the terminal device 100, and can include more or fewer components than shown, or combine certain components, or include different components, for example, can also include an input / output device, a network access device, and the like.

[0103] The processor 1000 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0104] The memory 1001 can be an internal storage unit of the terminal device 100 in some embodiments, for example, a hard disk or a memory of the terminal device 100. The memory 1001 can also be an external storage device of the terminal device 100 in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the memory 1001 can include both an internal storage unit and an external storage device of the terminal device 100. The memory 1001 is used to store an operating system, an application program, a boot loader, data, and other programs, for example, program codes of the computer program, and the like. The memory 1001 can also be used to temporarily store data that has been output or will be output.

[0105] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement steps in each method embodiment.

[0106] The embodiment of the present application provides a computer program product, when the computer program product is run on a mobile terminal, so that the mobile terminal executes the steps in the above-mentioned various method embodiments.

[0107] The integrated unit, if in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods, which can be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps in the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, an executable file or some intermediate form. The computer-readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunications signal.

[0108] In the above-mentioned embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0109] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0110] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other manners. For example, the embodiments of the apparatus / network device described above are merely illustrative. For example, the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0111] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0112] The above-described embodiments are merely used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of named entity recognition, characterized by, The method includes: Obtain the text to be recognized and convert it into an n×k dimensional first matrix, where n is the number of characters in the text to be recognized and k is a preset dimension of the character vector corresponding to the text to be recognized; The first matrix is ​​subjected to multi-layer convolution, wherein the last convolutional layer with m channels in the multi-layer convolutional layer is subjected to 4 convolution operations to obtain 4 parallel n×m dimensional second matrices. After adaptive attention weighting of 3 out of the 4 second matrices, an n×m dimensional third matrix is ​​obtained. The third matrix is ​​then added to the remaining 1 second matrix to output an n×m dimensional fourth matrix. The fourth matrix is ​​classified, the probabilities of multiple entity labels corresponding to the fourth matrix are output, and the entity labels corresponding to the text to be identified are determined based on the multiple probabilities. Based on the entity annotations corresponding to the text to be identified, output the named entity corresponding to the text to be identified; The second matrix is ​​obtained by convolving the first matrix with convolution kernels of different parameters but the same kernel dimension, so as to obtain the second matrix with different attention. The n×m dimensional third matrix obtained by adaptively applying attention weights to three of the four second matrices is then added to the remaining one of the second matrices to obtain an n×m dimensional fourth matrix, comprising: Multiply the first second matrix by the transpose of the second second matrix to obtain an n×n dimensional seventh matrix; Perform matrix multiplication on the seventh matrix and the third second matrix to obtain the n×m dimensional third matrix; The third matrix is ​​added to the fourth second matrix to obtain the n×m dimensional fourth matrix.

2. The named entity recognition method of claim 1, wherein, After converting the text to be recognized into an n×k dimensional first matrix, the process further includes: Add a preset g-dimensional position vector to the n×k-dimensional first matrix to obtain an n×(k+g)-dimensional first matrix with the added position vector. 3.The named entity recognition method of claim 1, wherein, The first matrix is ​​subjected to multi-layer convolution, wherein the last convolutional layer with m channels in the multi-layer convolutional layer is subjected to 4 convolution operations to obtain 4 parallel n×m dimensional second matrices, including: After performing a preset i-th convolutional layer on the first matrix, an n×m dimensional fifth matrix is ​​obtained, wherein the number of channels of the convolution kernel in each i-th convolutional layer is m. The fifth matrix is ​​convolved four times by a convolutional layer with a preset first convolutional kernel of j×m dimensions, and four parallel second matrices of n×m dimensions are output. The number of channels of the first convolutional kernel is m.

4. The named entity recognition method of claim 3, wherein, After performing a preset i-th convolutional layer on the first matrix, an n×m dimensional fifth matrix is ​​obtained, including: The first matrix is ​​convolved by the first convolutional layer with a second convolutional kernel of 1×k dimensions, and the output is an n×m dimensional sixth matrix. The number of channels of the second convolutional kernel is m. After performing convolutions from the second convolutional layer to the i-th convolutional layer on the fifth matrix, an n×m dimensional fifth matrix is ​​output, wherein the convolution kernels of the second convolutional layer to the i-th convolutional layer are all the first convolutional kernel.

5. The named entity recognition method of claim 1, wherein, The step of classifying the fourth matrix, outputting the probabilities of multiple entity labels corresponding to the fourth matrix, and determining the entity label corresponding to the text to be identified based on the multiple probabilities includes: The fourth matrix is ​​input into the conditional random field model, and the probabilities of multiple entity labels corresponding to the fourth matrix are output. The entity labels corresponding to the text to be identified are determined based on the multiple probabilities.

6. The named entity recognition method of claim 1, wherein, The step of classifying the fourth matrix, outputting the probabilities of multiple entity labels corresponding to the fourth matrix, and determining the entity label corresponding to the text to be identified based on the multiple probabilities includes: After the fourth matrix is ​​input into the fully connected layer, the probability of multiple entity labels corresponding to the fourth matrix is ​​output through the softmax function, and the entity label corresponding to the text to be identified is determined based on the multiple probabilities.

7. A named entity recognition device, characterized in that, include: The acquisition module is used to acquire the text to be recognized and convert the text to be recognized into an n×k dimensional first matrix, where n is the number of characters in the text to be recognized and k is a preset dimension of the character vector corresponding to the text to be recognized; The convolution module is used to perform multi-layer convolution on the first matrix, wherein the last convolution layer in the multi-layer convolution is performed 4 times to obtain 4 parallel n×m-dimensional second matrices. The attention module is used to adaptively apply attention weights to three of the four second matrices to obtain an n×m dimensional third matrix. The third matrix is ​​then added to the remaining one of the second matrices to output an n×m dimensional fourth matrix. The classification module is used to classify the fourth matrix, output the probability of multiple entity labels corresponding to the fourth matrix, and determine the entity label corresponding to the text to be identified based on the multiple probabilities. The output module is used to output the named entity corresponding to the text to be identified based on the entity annotation corresponding to the text to be identified; Wherein, the second matrix is ​​obtained by convolving the first matrix with convolution kernels of different parameters but the same kernel dimension, so as to obtain second matrices with different attention; the attention module is specifically used to perform matrix multiplication on the transpose of the first second matrix and the second second matrix to obtain an n×n dimensional seventh matrix; and to perform matrix multiplication on the seventh matrix and the third second matrix to obtain the n×m dimensional third matrix; The third matrix is ​​added to the fourth second matrix to obtain the n×m dimensional fourth matrix.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images