Named entity recognition method, device, and apparatus

Abstract

The embodiment of the present invention provides a method, apparatus and device for identifying named entities, which includes: obtaining multiple information blocks contained in rich text; determining semantic feature vectors corresponding to each of the multiple information blocks and relative visual feature vectors between different information blocks; determining context feature vectors corresponding to each of the multiple information blocks according to the semantic feature vectors corresponding to each of the multiple information blocks and the relative visual feature vectors between different information blocks; identifying named entities contained in the rich text according to the context feature vectors corresponding to each of the multiple information blocks. In combination with the visual feature information of the rich text, the named entities of interest contained in the rich text can be more accurately identified.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a method, device and equipment for named entity recognition. Background Art

[0002] Named entity recognition is a basic problem in the field of natural language processing and belongs to the category of sequence labeling problems. Simply put, the problem of named entity recognition is to identify and classify the entities of interest contained in a text sequence, such as names of people, places, and institutions.

[0003] At present, most named entity recognition solutions are for plain text. However, there are many rich texts in real life, such as VAT invoices, insurance policies, customs declarations, etc. Therefore, the demand for named entity recognition of rich texts has become increasingly important and urgent. Rich text is a text format that is different from plain text and contains rich format attributes, such as font size, pictures, layout, etc. Summary of the invention

[0004] Embodiments of the present invention provide a method, apparatus, and device for identifying named entities contained in rich text.

[0005] In a first aspect, an embodiment of the present invention provides a method for named entity recognition, the method comprising:

[0006] Get multiple information blocks contained in rich text;

[0007] Determine the semantic feature vectors corresponding to each of the plurality of information blocks and the relative visual feature vectors between different information blocks;

[0008] Determining context feature vectors corresponding to each of the multiple information blocks according to the semantic feature vectors corresponding to each of the multiple information blocks and relative visual feature vectors between different information blocks;

[0009] The named entities included in the rich text are identified according to the context feature vectors corresponding to each of the multiple information blocks.

[0010] In a second aspect, an embodiment of the present invention provides a named entity recognition device, the device comprising:

[0011] An acquisition module, used to acquire multiple information blocks contained in the rich text;

[0012] A determination module, configured to determine the semantic feature vectors corresponding to each of the plurality of information blocks and the relative visual feature vectors between different information blocks; and to determine the context feature vectors corresponding to each of the plurality of information blocks according to the semantic feature vectors corresponding to each of the plurality of information blocks and the relative visual feature vectors between different information blocks;

[0013] The recognition module is used to recognize the named entities contained in the rich text according to the context feature vectors corresponding to each of the multiple information blocks.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the named entity recognition method in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the named entity recognition method in the first aspect.

[0016] In an embodiment of the present invention, for rich text that needs to be recognized by named entities, first, multiple information blocks contained in the rich text can be extracted, each of which contains multiple characters, which may or may not contain named entities. Secondly, for each information block, its corresponding semantic information is determined, and the semantic information is represented as a semantic feature vector, and for each information block, its visual information relative to other information blocks is determined, and the relative visual information between different information blocks is represented as a relative visual feature vector. Thirdly, for each information block, combined with its corresponding semantic feature vector and the relative visual feature vector with other information blocks, the context feature vector corresponding to the information block is determined, and the context feature vector contains the contextual semantic information and visual information between the information block and other information blocks. Finally, combined with the context feature vector corresponding to each information block, the named entities contained therein are identified from the multiple information blocks, that is, the named entities contained in the rich text are identified. In summary, combining the visual feature information of rich text can more accurately identify the named entities of interest contained therein. For example, to identify the tax-free amount in a value-added tax invoice, there are many amounts in the value-added tax invoice, such as the amount of a single item, the tax amount, the total amount and other numbers. Combining the visual feature information of the text can determine which position of the number is the tax-free amount, thereby achieving correct recognition of the named entity. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a method for named entity recognition provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a result of information block division;

[0020] Figure 3 A schematic diagram of a process of obtaining a semantic feature vector of an information block;

[0021] Figure 4 A schematic diagram of a rich text graph structure;

[0022] Figure 5 A schematic diagram of a process of obtaining a context feature vector of an information block;

[0023] Figure 6 Schematic diagram of the principle of named entity recognition process for rich text;

[0024] Figure 7 A schematic diagram of the structure of a named entity recognition device provided by an embodiment of the present invention;

[0025] Figure 8 For Figure 7 A schematic diagram of the structure of an electronic device corresponding to the named entity recognition device provided in the illustrated embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0028] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0029] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0030] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0031] The named entity recognition method provided in the embodiment of the present invention can be performed by an electronic device, which can be a terminal device such as a PC, a laptop, or a server. The server can be a physical server including an independent host, or a virtual server hosted by a host cluster, or a cloud server.

[0032] The named entity recognition method is suitable for identifying named entities of interest contained in rich text. The rich text may contain content with various format attributes such as text, pictures, tables, etc. In practical applications, the rich text may be various documents such as value-added tax invoices and insurance policies, but is not limited to this.

[0033] The following two examples are used to schematically illustrate the actual requirements of named entity recognition provided by the embodiments of the present invention.

[0034] For example, the goal of named entity recognition is to identify the invoice issue date. Since an invoice may contain various dates, such as the issue date and the bill occurrence date, it is often impossible to determine which is the issue date based on the date format alone.

[0035] For another example, the goal of named entity recognition is to identify the tax-free amount in a VAT invoice. Since there are many amounts in a VAT invoice, such as the amount of a single item, the tax-free amount, the total amount, etc., all in digital format, it is often impossible to determine which is the tax-free amount based on the data format alone.

[0036] Based on this, if the rich text contains multiple contents with the same format as the named entities to be identified, how to accurately identify the named entities of interest becomes an urgent problem to be solved, which is also the core purpose of the named entity recognition method provided in an embodiment of the present invention.

[0037] In summary, the key to solving this problem in the named entity recognition method provided by the embodiment of the present invention is to combine the visual information of rich text to assist in the accurate recognition of named entities. The visual information can be reflected as the difference in visual features such as position and size between the named entity of interest and other non-named entity content. For example, assuming that the identification object is an ID card, the purpose of named entity recognition is to identify the name on the ID card. In addition to the name, there is other information on the ID card, such as the ID card number, issuing authority and other information. Taking the name and the issuing authority as an example, since the area containing the name and the area containing the issuing authority information have obvious differences in size and position, the differences in these visual information can be combined to assist in the recognition of the name.

[0038] The execution process of the named entity recognition method is described below in conjunction with the following embodiments.

[0039] Figure 1 The flowchart of a method for identifying a named entity provided by an embodiment of the present invention is shown in FIG. 1 , and the method for identifying a named entity may be executed by a first wireless router. Figure 1 As shown, the method comprises the following steps:

[0040] 101. Obtain multiple information blocks contained in the rich text.

[0041] In practical applications, rich text is generally presented in picture format or PDF file format.

[0042] For rich text in image format, multiple information blocks contained in the rich text can be obtained through optical character recognition (OCR) software.

[0043] For rich text in PDF format, you can use tools such as the Apache pdfbox plug-in to obtain multiple information blocks contained in the rich text.

[0044] In this article, information blocks refer to various areas in rich text that contain independent text content. That is, non-text content such as graphics and background in the rich text is filtered out, and the remaining content is divided to obtain multiple information blocks.

[0045] In simple terms, an information block can be understood as a sentence. Therefore, the information blocks can be divided according to the layout information and semantic relevance of the text content.

[0046] The information blocks in this article can also be called text blocks or word blocks. Regardless of the name, the main emphasis is that the data content contained therein is text content such as words, numbers, special symbols, etc. in the respective languages.

[0047] To understand the meaning of the information block, Figure 2 The result of dividing a form into information blocks is shown in the figure. Figure 2 FIG. 4 illustrates multiple information blocks of different sizes and positions.

[0048] 102. Determine semantic feature vectors corresponding to each of the multiple information blocks and relative visual feature vectors between different information blocks.

[0049] For each information block, the semantic information represented by the text content contained in the information block is determined, and the semantic information is represented by a semantic feature vector.

[0050] In an optional embodiment, determining the semantic feature vectors corresponding to each of the multiple information blocks can be implemented as follows: inputting the words contained in each of the multiple information blocks into a first neural network model to extract the semantic feature vectors corresponding to each of the multiple information blocks through the first neural network model.

[0051] Among them, the first neural network model includes any one of the following: a recurrent neural network model (Recurrent Neural Network, referred to as RNN), a long short-term memory network model (Long Short-Term Memory, referred to as LSTM) or a bidirectional long short-term memory network model (Bi-directional Long Short-Term Memory, referred to as Bi-LSTM).

[0052] Combination Figure 3 To illustrate the process of obtaining the semantic feature vector of an information block. Figure 3 It is assumed that the first neural network model used is Bi-LSTM. Figure 3 The arrows between adjacent LSTMs in the figure indicate the hidden layer transmission direction of the forward LSTM and the backward LSTM, respectively. Taking any information block i among multiple information blocks as an example, assume that the information block i includes three words x1, x2, and x3, and these three words are input into the Bi-LSTM after word vector conversion. Assume that the hidden layer states are updated to Hf1, Hf2, and Hf3 after the forward LSTM encodes these three words in sequence, and the hidden layer states are updated to Hb1, Hb2, and Hb3 after the backward LSTM encodes these three words in sequence. After that, the hidden layer state Hf3 at the last moment of the forward LSTM and the hidden layer state Hb3 at the last moment of the backward LSTM can be spliced ​​together {Hf3⊕Hb3} as the semantic feature vector of the information block i, where ⊕ represents the splicing symbol.

[0053] In another optional embodiment, determining the semantic feature vectors corresponding to each of the multiple information blocks can be implemented as follows: encoding the words contained in each of the multiple information blocks to obtain corresponding word vectors; and calculating the mean of the multiple word vectors corresponding to each information block to obtain the semantic feature vector corresponding to each information block.

[0054] In practical applications, a large number of corpus samples can be segmented in advance to obtain a number of words, and then the words can be sorted according to the number of occurrences of each word, for example, sorted from most to least according to the number of occurrences. Assuming that there are N words in total, a word list consisting of N words arranged in sequence can be generated. In addition, for each word, a word vector conversion can be performed according to an existing word vector conversion algorithm. Each word is assumed to be represented as an M-dimensional row vector, so that an N*M-dimensional word vector matrix is ​​finally obtained, wherein the k-th row word vector of the word vector matrix corresponds to the k-th word in the word list.

[0055] Based on this, for any information block i among the multiple information blocks, each word included in the information block i can be converted into a corresponding word vector according to the correspondence between the word vector matrix and the word table.

[0056] For information block i, it generally includes multiple words, each word corresponds to a word vector, so information block i corresponds to multiple word vectors, and the average calculation result of the multiple word vectors can be used as the semantic feature vector of information block i.

[0057] For each information block, in addition to determining the semantic feature vector corresponding to the information block, it is also necessary to determine the visual information of the information block relative to other information blocks. The visual information is represented by a relative visual feature vector, where the word "relative" refers to the relative visual information of one information block relative to another information block.

[0058] Based on this, determining the relative visual feature vectors between different information blocks can be implemented as: determining the distance and / or relative size ratio between different information blocks, so that the relative visual feature vector includes the distance and / or relative size ratio.

[0059] The distance may include horizontal distance and vertical distance.

[0060] The relative size ratio may include: the aspect ratio of the same information block, the height ratio of different information blocks, and the aspect ratio of different information blocks.

[0061] Based on this, it can be understood that when obtaining multiple information blocks contained in the rich text, the position coordinates and size information of each information block can be determined at the same time. The position coordinates of each information block can be represented by the coordinates of the center point of the corresponding rectangular box, and the size information is determined according to the coordinates of the four vertices of the rectangular box.

[0062] Take any two information blocks: information block i and information block j as an example, assuming that the horizontal distance between the two information blocks is represented by Xij, and the vertical distance between the two information blocks is represented by Yij; assuming that the width of information block i is represented by Wi, and the height is represented by Hi; assuming that the width of information block j is represented by Wj, and the height is represented by Hj, then the aspect ratio of information block i is Wi / Hi, the height ratio of information block j to information block i is Hj / Hi, and the height ratio of information block j to information block i is Wj / Hi. Thus, optionally, the visual feature vector of information block i relative to information block j can be represented as: Rij = [Xij, Yij, Wi / Hi, Hj / Hi, Wj / Hi].

[0063] 103. Determine context feature vectors corresponding to each of the multiple information blocks according to the semantic feature vectors corresponding to each of the multiple information blocks and the relative visual feature vectors between different information blocks.

[0064] In fact, the process of dividing the rich text into information blocks, calculating the semantic information of each information block and the relative visual information between different information blocks can be understood as describing the rich text as a graph structure, that is, a graph structure composed of nodes and edges.

[0065] Among them, the semantic feature vector corresponding to each information block can be regarded as a node in the graph structure, and the relative visual feature vectors between different information blocks can be regarded as an edge in the graph structure.

[0066] For ease of understanding, combined Figure 4 To illustrate the graph structure of rich text. Assume that the rich text includes six information blocks, information block 1 to information block 6. Figure 4 In , the semantic feature vectors corresponding to the six information blocks are represented as t1 to t6. It can be assumed that there are connecting edges between each information block and the remaining information blocks. Therefore, for information block 1, it is assumed that the connecting edges between it and the other five information blocks are represented as: R12 to R16. Figure 4 Only the fully connected relationship between information block 1 and the remaining information blocks is illustrated.

[0067] Since rich text can be represented as the above-mentioned graph structure, when executing the above-mentioned step 103, the calculation of the upper and lower feature vectors corresponding to each information block can be realized by the graph convolutional neural network. Among them, the context feature vector corresponding to each information block is determined based on the semantic feature vector of the information block itself, the relative visual feature vector of the information block relative to other information blocks, and the semantic feature vectors of the other information blocks. Therefore, the context feature vector corresponding to the information block includes the contextual semantic information of the information block in the text dimension and the visual feature information in the visual dimension.

[0068] Taking any information block i as an example, the process of acquiring the context feature vector corresponding to information block i can be implemented as follows: acquiring multiple groups of feature vectors corresponding to information block i, wherein any group of feature vectors is composed of the semantic feature vector corresponding to information block i, the semantic feature vector corresponding to information block j, and the relative visual feature vector between information block i and information block j, and information block j is any information block among the multiple information blocks except information block i; inputting the acquired multiple groups of feature vectors into the graph convolutional neural network to output the context feature vector corresponding to information block i through the graph convolutional neural network.

[0069] Combination Figure 4 and Figure 5 To schematically illustrate the process of determining the context feature vector of the information block. As described above, Figure 4 In the figure, it is assumed that the semantic feature vectors corresponding to the six information blocks 1 to 6 are denoted as t1 to t6. For information block 1, it is assumed that the connecting edges between it and the other five information blocks are denoted as R12 to R16. R12 to R16 are the relative visual feature vectors of information block 1 relative to the other five information blocks.

[0070] based on Figure 4 Assumptions such as Figure 5 As shown in , for information block 1, the following five sets of feature vectors consisting of three elements can be obtained:

[0071] [t1,R12,t2],[t1,R13,t3],[t1,R14,t4],[t1,R15,t5],[t1,R16,t6]. These five sets of feature vectors are input into the graph convolutional neural network. Through the calculation of the graph convolutional neural network, five hidden state vectors will be output for these five sets of feature vectors respectively, which are assumed to be represented as: h12~h16. By calculating these five hidden state vectors, the context feature vector corresponding to information block 1 can be obtained, which is assumed to be represented as C1. Among them, the self-attention mechanism can be used to calculate these five hidden state vectors to obtain C1.

[0072] The calculation process of the context feature vectors corresponding to other information blocks contained in the rich text is consistent with the calculation process of information block 1, and will not be repeated here.

[0073] In practical applications, graph convolutional neural networks can be implemented as multi-layer perceptrons with one or more fully connected layers.

[0074] 104. Identify named entities contained in the rich text according to context feature vectors corresponding to each of the multiple information blocks.

[0075] In the embodiment of the present invention, an entity recognition model may be pre-trained to recognize named entities contained in rich text. The entity recognition model may also be called a sequence labeling model, because the named entity recognition problem is actually a sequence labeling problem.

[0076] Combination Figure 6 To illustrate the core process of named entity recognition.

[0077] The entity recognition model includes: a second neural network model, a fully connected layer (FC Layer) and a conditional random field model (CRF) connected in sequence. The second neural network model includes any of the following: LSTM, Bi-LSTM, Figure 6 The one shown in the figure is Bi-LSTM.

[0078] After the context feature vector corresponding to each information block is obtained, the input of the second neural network model can be determined according to the context feature vector corresponding to each information block.

[0079] Specifically, first, the words contained in each of the multiple information blocks in the rich text are encoded to obtain the corresponding word vectors. For any information block i, the information block i will correspond to multiple word vectors. Secondly, the multiple word vectors corresponding to the information block i are concatenated with the context feature vector corresponding to the information block i. Figure 6 As shown in , assuming that the words contained in information block i are x1, x2…, xm, the context feature vector Ci corresponding to information block i is concatenated with the word vector corresponding to each word, and m concatenated feature vectors will be obtained. Assume that the concatenation process is expressed as: Ui=Ci⊕e(xi), where ⊕ represents the concatenation symbol and e(xi) represents the word vector corresponding to word xi. Then, the concatenated m feature vectors are input into the entity recognition model to output the named entity recognition result corresponding to information block i through the entity recognition model. Among them, the named entity recognition result of information block i refers to the identification of whether it contains named entities and what the specific named entities are.

[0080] In summary, since the text of named entities and other non-named entities will show certain visual differences, the named entities of interest contained in the rich text can be more accurately identified by combining the visual feature information of the rich text. In addition, the rich text is divided into information blocks and represented as a graph structure. The visual information of each information block contained in the rich text can be automatically extracted based on the graph convolutional neural network, making the named entity recognition method have good universality and can be applied to various rich texts.

[0081] The previous article introduced the use of graph convolutional neural networks and entity recognition models. The following briefly introduces the training process of graph convolutional neural networks and entity recognition models.

[0082] First, a large number of training samples can be collected, and the training samples can be a large amount of rich text. Afterwards, multiple information blocks included in the training samples can be obtained, which are referred to as multiple information block samples here to distinguish them from the multiple information blocks in the previous text. The method of obtaining information block samples is consistent with the previous text, and when obtaining multiple information block samples included in the training samples, the position coordinates and size information of each information block sample can be determined at the same time. Since the purpose of training graph convolutional neural networks and entity recognition models is to recognize named entities, named entities can be labeled for each information block sample.

[0083] The task of named entity recognition is to identify one or more named entities included in rich text, such as names of people, time, numbers, place names, organization names, etc. In practical applications, the BIO method can be used to annotate named entities. Specifically, a label set can be defined according to the type of named entity to be recognized. For example, assuming that the named entities to be recognized include names of people and organization names, the label set LabelSet can be defined as:

[0084] LabelSet = {BA, IA, BP, IP, O},

[0085] Taking Chinese named entities as an example, the sentence that needs to be recognized by named entities is regarded as a sequence of Chinese characters.

[0086] Among them, BA represents that the Chinese character is the first letter of the organization name, IA represents that the Chinese character is the middle letter of the organization name; BP represents that the Chinese character is the first letter of a person's name, IP represents that the Chinese character is the middle letter of a person's name, and O represents that the Chinese character does not belong to a named entity.

[0087] With the input Chinese character sequence and label set, the next step is to use the trained entity recognition model to classify each Chinese character, that is, to label each Chinese character with a label from the label set, so that the named entities in the sentence are identified.

[0088] Based on the above introduction, it can be known that each information block sample obtained can be regarded as a sentence, and the named entity annotation of the information block sample is to label the text contained in each information block sample in combination with the defined label set. It can be understood that if there is no named entity in an information block sample, all the text in the information block sample is labeled as the letter O.

[0089] Then, the graph convolutional neural network and entity recognition model are trained based on the labeled information block samples. In the process of training the graph convolutional neural network and entity recognition model, the processing of each information block sample is consistent with the processing of information block i in the previous text, and will not be repeated. However, for each information block sample, the final entity recognition model outputs a prediction result of whether the information block sample contains a named entity and what kind of named entity it contains. It is necessary to compare the prediction result with the labeling result of the information block sample to determine the loss function of the model, so as to feedback and adjust the parameters of the entity recognition model and the graph convolutional neural network based on the loss function until the model converges.

[0090] The following will describe in detail one or more embodiments of the named entity recognition apparatus of the present invention. Those skilled in the art will appreciate that these named entity recognition apparatuses can be configured using commercially available hardware components through the steps taught in this solution.

[0091] Figure 7 A schematic diagram of a named entity recognition device provided by an embodiment of the present invention is shown in FIG. Figure 7 As shown, the named entity recognition device includes: an acquisition module 11, a determination module 12, and a recognition module 13.

[0092] The acquisition module 11 is used to acquire multiple information blocks contained in the rich text.

[0093] The determination module 12 is used to determine the semantic feature vectors corresponding to each of the multiple information blocks and the relative visual feature vectors between different information blocks; based on the semantic feature vectors corresponding to each of the multiple information blocks and the relative visual feature vectors between different information blocks, determine the context feature vectors corresponding to each of the multiple information blocks.

[0094] The identification module 13 is used to identify the named entities contained in the rich text according to the context feature vectors corresponding to each of the multiple information blocks.

[0095] Optionally, in the process of determining the semantic feature vectors corresponding to each of the multiple information blocks, the determination module 12 can be specifically used to: input the words contained in each of the multiple information blocks into a first neural network model, so as to extract the semantic feature vectors corresponding to each of the multiple information blocks through the first neural network model.

[0096] Among them, the first neural network model includes any one of the following: a recurrent neural network model, a long short-term memory network model or a bidirectional long short-term memory network model.

[0097] Optionally, in the process of determining the semantic feature vectors corresponding to each of the multiple information blocks, the determination module 12 can be specifically used to: encode the words contained in each of the multiple information blocks to obtain corresponding word vectors; and calculate the mean of the multiple word vectors corresponding to each information block to obtain the semantic feature vector corresponding to each information block.

[0098] Optionally, in the process of determining the relative visual feature vector between the different information blocks, the determination module 12 may be specifically used to: determine the distance and / or relative size ratio between the different information blocks, wherein the relative visual feature vector includes the distance and / or the relative size ratio.

[0099] The distance includes a horizontal distance and a vertical distance.

[0100] The relative size ratio includes: the aspect ratio of the same information block, the height ratio of different information blocks, and the aspect ratio of different information blocks.

[0101] Optionally, in the process of determining the context feature vectors corresponding to each of the multiple information blocks, the determination module 12 can be specifically used to: for information block i among the multiple information blocks, obtain multiple groups of feature vectors corresponding to the information block i, wherein any group of feature vectors is composed of a semantic feature vector corresponding to the information block i, a semantic feature vector corresponding to information block j, and a relative visual feature vector between the information block i and the information block j, the information block i is any information block among the multiple information blocks, and the information block j is any information block among the multiple information blocks except information block i; input the multiple groups of feature vectors into a graph convolutional neural network to output the context feature vector corresponding to the information block i through the graph convolutional neural network.

[0102] Optionally, the recognition module 13 can be specifically used to: encode the words contained in each of the multiple information blocks to obtain corresponding word vectors; for the information block i, splice the multiple word vectors corresponding to the information block i with the context feature vectors corresponding to the information block i; input the spliced ​​feature vectors into the entity recognition model to output the named entity recognition result corresponding to the information block i through the entity recognition model.

[0103] Among them, the entity recognition model includes: a second neural network model, a fully connected layer and a conditional random field model connected in sequence; the second neural network model includes any one of the following: a long short-term memory network model, a bidirectional long short-term memory network model.

[0104] Optionally, the device also includes: a training module, used to obtain training samples, the training samples are rich text; obtain multiple information block samples included in the training samples; label the named entities contained in the multiple information block samples; and train the graph convolutional neural network and the entity recognition model based on the labeled multiple information block samples.

[0105] Figure 7 The named entity recognition device shown can execute the methods provided in the aforementioned embodiments. For the parts not described in detail in this embodiment, reference can be made to the relevant descriptions of the aforementioned embodiments, which will not be repeated here.

[0106] In one possible design, the above Figure 7 The structure of the named entity recognition device shown can be implemented as an electronic device. Figure 8 As shown, the electronic device may include: a processor 21 and a memory 22. The memory 22 stores executable codes, and when the executable codes are executed by the processor 21, the processor 21 can at least implement the named entity recognition method provided in the above-mentioned embodiment.

[0107] The structure of the electronic device may further include a communication interface 23 for communicating with other devices or a communication network.

[0108] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of a wireless router, the processor executes the named entity recognition method provided in the aforementioned embodiments.

[0109] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0110] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for named entity recognition, characterized in that: include: Get multiple information blocks contained in rich text; Determine semantic feature vectors corresponding to each of the plurality of information blocks and relative visual feature vectors between different information blocks, wherein the relative visual feature vectors include distance and / or relative size ratio; For a first information block among the multiple information blocks, input a semantic feature vector corresponding to the first information block, a semantic feature vector corresponding to the second information block, and a relative visual feature vector between the first information block and the second information block into a graph convolutional neural network, so as to output a context feature vector corresponding to the first information block through the graph convolutional neural network, wherein the semantic feature vector corresponding to the first information block is used as a node in a graph structure, and the relative visual feature vector between the first information block and the second information block is used as an edge in the graph structure; The named entities included in the rich text are identified according to the context feature vectors corresponding to each of the multiple information blocks.

2. The method according to claim 1, characterized in that The step of determining the semantic feature vectors corresponding to each of the plurality of information blocks comprises: The words contained in each of the multiple information blocks are input into a first neural network model, so as to extract the semantic feature vectors corresponding to each of the multiple information blocks through the first neural network model.

3. The method according to claim 2, characterized in that The first neural network model includes any one of the following: a recurrent neural network model, a long short-term memory network model, or a bidirectional long short-term memory network model.

4. The method according to claim 1, characterized in that: The step of determining the semantic feature vectors corresponding to each of the plurality of information blocks comprises: Encoding the words contained in each of the plurality of information blocks to obtain a corresponding word vector; The average of multiple word vectors corresponding to each information block is calculated to obtain the semantic feature vector corresponding to each information block.

5. The method according to claim 1, characterized in that The distance includes a horizontal distance and a vertical distance.

6. The method according to claim 1, characterized in that The relative size ratio includes: the aspect ratio of the same information block, the height ratio of different information blocks, and the aspect ratio of different information blocks.

7. The method according to claim 1, characterized in that The identifying the named entities contained in the rich text according to the context feature vectors corresponding to each of the plurality of information blocks includes: Encoding the words contained in each of the plurality of information blocks to obtain a corresponding word vector; For any information block among the multiple information blocks, concatenate the multiple word vectors corresponding to the any information block with the context feature vector corresponding to the any information block; The concatenated feature vectors are input into an entity recognition model, so as to output a named entity recognition result corresponding to any one of the information blocks through the entity recognition model.

8. The method according to claim 7, characterized in that The entity recognition model includes: a second neural network model, a fully connected layer and a conditional random field model connected in sequence; The second neural network model includes any one of the following: a long short-term memory network model, a bidirectional long short-term memory network model.

9. The method according to claim 7, characterized in that: The method further comprises: Obtaining a training sample, where the training sample is a rich text; Acquire a plurality of information block samples included in the training sample; marking named entities included in the plurality of information block samples; The graph convolutional neural network and the entity recognition model are trained according to the labeled multiple information block samples.

10. A named entity recognition device, characterized in that: include: An acquisition module, used to acquire multiple information blocks contained in the rich text; A determination module, used to determine the semantic feature vectors corresponding to each of the multiple information blocks and the relative visual feature vectors between different information blocks, wherein the relative visual feature vector includes a distance and / or a relative size ratio; for a first information block among the multiple information blocks, the semantic feature vector corresponding to the first information block, the semantic feature vector corresponding to the second information block, and the relative visual feature vector between the first information block and the second information block are input into a graph convolutional neural network, so as to output a context feature vector corresponding to the first information block through the graph convolutional neural network, wherein the semantic feature vector corresponding to the first information block is used as a node in a graph structure, and the relative visual feature vector between the first information block and the second information block is used as an edge in the graph structure; The recognition module is used to recognize the named entities contained in the rich text according to the context feature vectors corresponding to each of the multiple information blocks.

11. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the named entity recognition method as described in any one of claims 1 to 9.

Images