Entity recognition model establishment method, device, electronic device and storage medium

Abstract

The present disclosure provides a method for establishing an entity recognition model, including: obtaining a labeled text sample set, the labeled text sample set including multiple paragraph categories and multiple entity categories; dividing the labeled text sample into segments, sentences and characters; obtaining a sentence feature vector group and a character feature vector group; obtaining a trained TextCNN model; obtaining a trained entity model; using the trained TextCNN model to perform paragraph category recognition on the vectorized text to obtain a paragraph category vector, using the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector; inputting the paragraph category vector and the entity category vector into the Softmax model for iterative training; and judging whether the training is stopped by the loss function value. The present disclosure also provides an entity recognition method, device, electronic device and readable storage medium.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of entity recognition, and in particular relates to a method, device, electronic device and storage medium for establishing an entity recognition model. Background Art

[0002] As entity recognition in the existing technology becomes more and more mature, its application scenarios are becoming more and more extensive, gradually expanding from short text entity recognition of a dozen or dozens of words to scenarios where it is recognized in super-long texts of thousands or tens of thousands of words. This complex long text extraction is often accompanied by diverse data definitions. In such new scenarios, two new problems that are difficult for short text entity recognition algorithms to solve will arise. The following uses the "privacy policy text" that is widely used in APPs as an example (the text length is usually 10,000 to 30,000 words) to introduce these two problems:

[0003] First, entity conflict and disambiguation. For example, the name of a company refers to the "APP operating company" in the "Company Information Introduction" section of the privacy policy, but refers to the "receiving company" of the information in the "Information Sharing" section. The same entity type is different, and such conflicts are difficult to distinguish using short text recognition methods.

[0004] Second, entity recognition errors. Entity recognition errors in long texts are not only related to samples, annotations, etc., but also to context. In very long texts, paragraph-level context beyond the sentence level is particularly important. For example, the two types of entities, "information storage region" and "storage period", only appear in the paragraph describing "information storage". Other expressions that mention locations do not belong to these two types. For example, the "registered address" of the operating entity company has the same expression as the "information storage region" but has completely different semantics. Making full use of semantic paragraphs can reduce such recognition errors, but it is difficult to recognize entities by sentence. From another perspective, finding the semantic paragraph of "information storage" first will also make the recognition of "information storage region" and "storage period" more accurate.

[0005] Among the existing methods, there are entity disambiguation methods specifically targeting the first difficulty, such as unsupervised methods based on contextual clustering and supervised methods that separately annotate ambiguous entities. The former is relatively ineffective, while the latter requires a large amount of annotations and samples, and requires an additional disambiguation model to be trained. As for the second difficulty, there is no good solution in the existing technology. Summary of the invention

[0006] In order to solve at least one of the above technical problems, the present disclosure provides a method and device for establishing an entity recognition model, and an entity recognition method and device, an electronic device, and a readable storage medium.

[0007] According to one aspect of the present disclosure, the present disclosure provides a method for establishing an entity recognition model, comprising:

[0008] S102, obtaining a marked text sample set, wherein the marked text sample set includes a plurality of preset paragraph categories and a plurality of preset entity categories;

[0009] S104, performing paragraph splitting on each text sample in the annotated text sample set, performing sentence splitting on each paragraph, and performing character splitting on each sentence;

[0010] S106, using the language representation model to vectorize each sentence of each paragraph to obtain a sentence feature vector group [CLS1, CLS2, ...] of each paragraph, and using the language representation model to vectorize each character of each sentence to obtain a character feature vector group [[token_embdedding1, token_embdedding2, ...], ...] of each sentence;

[0011] S108, training the TextCNN model based at least on the plurality of preset paragraph categories and the sentence feature vector groups of each paragraph to obtain a trained TextCNN model, including fine-tuning the TextCNN;

[0012] S110, training an entity model based at least on the plurality of preset entity categories and a character feature vector group of each sentence in each paragraph to obtain a trained entity model;

[0013] S112, using the trained TextCNN model to perform paragraph category recognition on the vectorized text sample to obtain a paragraph category vector, and using the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector; and

[0014] S114. Based on the paragraph category vector and the entity category vector, an input vector is constructed, and the input vector is input into a Softmax model for iterative training to obtain a trained Softmax model.

[0015] According to the entity recognition model establishment method of at least one embodiment of the present disclosure, by calculating the loss function, it is determined whether the threshold is met. If so, the iterative training is terminated to obtain the trained Softmax model.

[0016] According to the entity recognition model establishment method of at least one embodiment of the present disclosure, the step S106 includes:

[0017] The language representation model is fine-tuned so that some parameters of the language representation model are adjusted.

[0018] According to the entity recognition model establishment method of at least one embodiment of the present disclosure, the step S108 includes:

[0019] The TextCNN is fine-tuned so that some parameters of the TextCNN model are adjusted.

[0020] According to the entity recognition model establishment method of at least one embodiment of the present disclosure, the calculation of the loss function is based on the loss function of the training of the entity model and the loss function of the training of the Softmax.

[0021] According to another aspect of the present disclosure, the present disclosure further provides an entity recognition method, which performs entity recognition using an entity recognition model established by any of the above entity recognition model establishment methods, including:

[0022] Get the text to be recognized;

[0023] Vectorizing the text to be recognized through the language representation model to obtain a sentence feature vector group of each paragraph, and obtaining a character feature vector group of each sentence in the paragraph;

[0024] Using the trained TextCNN model to perform paragraph category recognition on the sentence feature vector group of the paragraph to obtain a paragraph category vector, and using the trained entity model to perform entity recognition on the character feature vector group of each sentence in the paragraph to obtain an entity category vector; and

[0025] An input vector is constructed based on the paragraph category vector and the entity category vector, and the input vector is input into the trained Softmax to perform entity classification to obtain the entity category.

[0026] According to another aspect of the present disclosure, the present disclosure also provides an entity recognition model establishment device, comprising:

[0027] A sample acquisition module is used to acquire a marked text sample set, wherein the marked text sample set includes a plurality of preset paragraph categories and a plurality of preset entity categories;

[0028] A sample processing module is used to split each text sample in the annotated text sample set into paragraphs, split each paragraph into sentences, and split each sentence into characters;

[0029] A vectorization module, which uses a language representation model to perform vectorization processing on each sentence in each paragraph to obtain a sentence feature vector group of each paragraph, and uses a language representation model to perform vectorization processing on each character of each sentence to obtain a character feature vector group of each sentence;

[0030] A TextCNN training module, which trains the TextCNN model based on at least the plurality of preset paragraph categories and the sentence feature vector groups of each paragraph to obtain a trained TextCNN model;

[0031] An entity model training module, which trains the entity model based on at least the plurality of entity preset categories and the character feature vector groups of each sentence in each paragraph to obtain a trained entity model;

[0032] A vector acquisition module, which uses the trained TextCNN model to perform paragraph category recognition on the vectorized text sample to obtain a paragraph category vector, and uses the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector;

[0033] The Softmax model training module constructs an input vector based on the paragraph category vector and the entity category vector, and inputs the input vector into the Softmax model for iterative training to obtain a trained Softmax model.

[0034] According to the entity recognition model establishment device provided by at least one embodiment of the present disclosure, a loss function is calculated to determine whether a threshold is met. If so, the iterative training is terminated to obtain the trained Softmax model.

[0035] According to the entity recognition model establishment device provided by at least one embodiment of the present disclosure, the vectorization module fine-tunes the language representation model so that some parameters of the language representation model are adjusted.

[0036] According to the entity recognition model establishment device provided by at least one embodiment of the present disclosure, the training of the TextCNN training module includes:

[0037] The TextCNN is fine-tuned so that some parameters of the TextCNN model are adjusted.

[0038] According to the entity recognition model building device provided by at least one embodiment of the present disclosure, the calculation of the loss function is based on the loss function of the training of the entity model and the loss function of the training of the Softmax.

[0039] According to another aspect of the present disclosure, the present disclosure further provides an entity recognition device, which uses an entity recognition model established by any of the above entity recognition model establishment methods to perform entity recognition, including:

[0040] A module for obtaining text to be recognized is used to obtain text to be recognized;

[0041] A vector group acquisition module is used to vectorize the text to be recognized through the language representation model to obtain a sentence feature vector group [CLS1, CLS2, ...] of each paragraph, and to obtain a character feature vector group [[token_embdedding1, token_embdedding2, ...], ...] of each sentence in the paragraph;

[0042] A category vector acquisition module, used to use the trained TextCNN model to perform paragraph category recognition on the sentence feature vector group of the paragraph to obtain a paragraph category vector, and use the trained entity model to perform entity recognition on the character feature vector group of each sentence in the paragraph to obtain an entity category vector; and

[0043] The entity recognition module constructs an input vector based on the paragraph category vector and the entity category vector, and inputs the input vector into the trained Softmax to perform entity classification to obtain the entity category.

[0044] According to another aspect of the present disclosure, the present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor, the processor executing the execution instructions stored in the memory, so that the processor executes any one of the methods described above.

[0045] According to another aspect of the present disclosure, the present disclosure further provides a readable storage medium, wherein the readable storage medium stores execution instructions, and the execution instructions are used to implement any of the above-mentioned methods when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0047] Figure 1 It is a flowchart of a method for establishing an entity recognition model according to an embodiment of the present disclosure.

[0048] Figure 2 It is a flowchart of an entity recognition method according to an embodiment of the present disclosure.

[0049] Figure 3 It is a schematic block diagram of an entity recognition model building device according to an embodiment of the present disclosure.

[0050] Figure 4 It is a schematic block diagram of an entity identification device according to one embodiment of the present disclosure.

[0051] Description of Reference Numerals

[0052] 1000 Entity recognition model building device

[0053] 1002 Get sample module

[0054] 1004 Sample Processing Module

[0055] 1006 Vectorization Module

[0056] 1008 Training Module

[0057] 1010 Physical Model Training Module

[0058] 1012 Vector Acquisition Module

[0059] 1014 Model training module

[0060] 1016 Loss function calculation module

[0061] 1100 Bus

[0062] 1200 processor

[0063] 1300 Memory

[0064] 1400 Other circuits

[0065] 2000 Entity Identification Device

[0066] 2002 Get the text module to be recognized

[0067] 2004 Vector Group Acquisition Module

[0068] 2006 Category vector acquisition module

[0069] 2008 Entity Recognition Module. DETAILED DESCRIPTION

[0070] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0071] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0072] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0073] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0074] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0075] For descriptive purposes, the present disclosure may use spatially relative terms such as "under," "beneath," "under," "down," "over," "upper," "above," "higher," and "side (e.g., in a "sidewall")," to describe the relationship of one component to another (other) component as shown in the accompanying drawings. The spatially relative terms are intended to encompass different orientations of the device in use, operation, and / or manufacture in addition to the orientation depicted in the accompanying drawings. For example, if the device in the accompanying drawings is turned over, components described as "under" or "beneath" other components or features would subsequently be positioned "over" the other components or features. Thus, the exemplary term "under" can encompass both the "above" and "below" orientations. Furthermore, the device may be otherwise oriented (e.g., rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein should be interpreted accordingly.

[0076] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0077] Figure 1 It is a flowchart of a method for establishing an entity recognition model according to an embodiment of the present invention.

[0078] like Figure 1 As shown, the entity recognition model establishment method S100 of this embodiment includes:

[0079] S102: Acquire a labeled text sample set, where the labeled text sample set includes a plurality of preset paragraph categories and a plurality of preset entity categories;

[0080] S104: splitting each text sample in the annotated text sample set into paragraphs, splitting each paragraph into sentences, and splitting each sentence into characters;

[0081] S106: using the language representation model to vectorize each sentence in each paragraph to obtain a sentence feature vector group of each paragraph, and using the language representation model to vectorize each character in each sentence to obtain a character feature vector group of each sentence;

[0082] S108: training the TextCNN model based on at least a plurality of preset paragraph categories and a sentence feature vector group of each paragraph to obtain a trained TextCNN model;

[0083] S110: training an entity model based on at least a plurality of entity preset categories and a character feature vector group of each sentence in each paragraph to obtain a trained entity model;

[0084] S112: Use the trained TextCNN model to perform paragraph category recognition on the vectorized text sample to obtain a paragraph category vector, and use the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector;

[0085] S114: Based on the paragraph category vector and the entity category vector, an input vector is constructed, and the input vector is input into the Softmax model for iterative training to obtain a trained Softmax model.

[0086] For the entity recognition model building method S100 of the above embodiment, preferably, by calculating the loss function, it is determined whether the threshold is met. If so, the iterative training is terminated to obtain the trained Softmax model.

[0087] An entity recognition model is established through the above entity recognition model establishment method, including a language representation model, a trained TextCNN model, a trained entity model, and a trained Softmax model.

[0088] Among them, the entity model described above is preferably a CRF model.

[0089] Each text sample may be a document.

[0090] The paragraph splitting described above may be paragraph splitting based on natural paragraphs.

[0091] Among them, the language representation model described above is preferably a BERT model.

[0092] Wherein, in the above step S114, an input vector is constructed based on the paragraph category vector and the entity category vector, and preferably, the input vector is constructed by concatenating the paragraph category vector and the entity category vector.

[0093] Combining the CRF model and the BERT model, the entity recognition model establishment method of each of the above implementations is as follows:

[0094] Among them, step S106 to step S108, the TextCNN training obtains the trained TextCNN model, specifically as follows: split the text sample into paragraphs, and mark the paragraph preset categories of the paragraphs; split the paragraphs into sentences, input the sentences into the BERT model encoding, obtain the [CLS] vectors of each sentence in the paragraph, and form the [CLS] vectors of each sentence into a sentence feature vector group, that is, sequence 1 [CLS1, CLS2, ...], sequence 1 is a paragraph-level [CLS] sequence; and input sequence 1 [CLS1, CLS2, ...] and the paragraph preset categories into TextCNN training to obtain the trained TextCNN model.

[0095] Among them, in step S110, the CRF is trained to obtain a trained CRF model, specifically as follows: the text sample is split into paragraphs, and the entity category in each sentence of the paragraph is marked; the paragraph is split into sentences, and the sentences are split into characters, and the characters are input into BERT encoding to obtain a character feature vector group, that is, sequence 2 is [[token_embdedding1, token_embdedding2, …], …], and sequence 2 is a paragraph-level [token_embdedding] sequence; and sequence 2 [[token_embdedding1, token_embdedding2, …], …] and the entity category are input into CRF for training to obtain the trained CRF model.

[0096] Wherein, step S112 includes:

[0097] Split the text sample into paragraphs, and mark the entity preset category in each sentence of the paragraph; split the paragraph into sentences, input the sentences into BERT encoding, obtain the [CLS] vectors of each sentence in the paragraph, and form the [CLS] vectors of each sentence into a sentence feature vector group, that is, sequence 1 [CLS1, CLS2, ...,]; and input sequence 1 [CLS1, CLS2, ...,] and the entity preset category into the trained TextCNN model for classification to obtain the paragraph category vector; and,

[0098] The text sample is divided into paragraphs, and the entity preset categories in each sentence of the paragraph are marked; the paragraphs are divided into sentences, the sentences are divided into characters, the characters are input into BERT encoding, and the character feature vector group is obtained, that is, sequence 2[[token_embdedding1, token_embdedding2, …], …]; and the sequence 2[[token_embdedding1, token_embdedding2, …], …] and the entity preset categories are input into the trained CRF for classification to obtain the entity category vector.

[0099] Wherein, step S106 includes:

[0100] The language representation model is fine-tuned so that some parameters of the language representation model are adjusted.

[0101] Wherein, step S108 includes:

[0102] TextCNN is fine-tuned so that some parameters of the TextCNN model are adjusted.

[0103] Among them, in step S116, the calculation of the loss function is based on the loss function of the training of the entity model and the loss function of the training of Softmax, which is expressed as CRF Loss+k*Softmax Loss, where k is a proportional coefficient, which is a number greater than 0 and can be preset and adjusted.

[0104] Among them, in step S116, the calculated value of the loss function is compared with the threshold. If the threshold is met, that is, the calculated value of the loss function is less than the threshold, the training is terminated to obtain the trained Softmax model.

[0105] The technical solution disclosed in the present invention, in terms of sample annotation, in addition to entity annotation, also predefines the semantic category of paragraphs, and performs paragraph category annotation for each paragraph of the article. BERT+CRF entity recognition is used as the baseline, and TextCNN is used to assist in semantic paragraph classification, and finally entity classification is achieved through Softmax to establish an entity recognition model. It can be seen that the entity recognition model established by the above method combines paragraph characteristics in the entity model (such as the CRF model), which helps to eliminate ambiguity when the same or similar entities appear in the text at the same time.

[0106] Figure 2 It is a flowchart of an entity recognition method according to an embodiment of the present invention.

[0107] like Figure 2 As shown, the entity recognition method S200 of this embodiment includes:

[0108] S202: Obtaining text to be recognized;

[0109] S204: vectorizing the text to be recognized through a language representation model to obtain a sentence feature vector group of each paragraph, and obtaining a character feature vector group of each sentence in the paragraph;

[0110] S206: using the trained TextCNN model to perform paragraph category recognition on the sentence feature vector group of the paragraph to obtain a paragraph category vector, and using the trained entity model to perform entity recognition on the character feature vector group of each sentence in the paragraph to obtain an entity category vector; and

[0111] S208: Based on the paragraph category vector and the entity category vector, an input vector is constructed, and the input vector is input into the trained Softmax for entity classification to obtain the entity category.

[0112] Taking entity recognition of privacy policy text as an example, the privacy policy text contains 27 entity categories and 25 title categories, such as "personal information", "company name", "contact information", "cookie principle", etc., and title categories include "business functions and information collected", "information storage", "information protection", "cookie introduction", etc. There are semantic constraints between entities and titles. For example, the "cookie principle" entity type will only appear under the "cookie introduction" title category.

[0113] The language representation model may be a BERT model, and the specific entity recognition method is:

[0114] Obtain the text of the above-mentioned privacy policy of the entity to be identified;

[0115] All sentences in the paragraph of the above text are encoded by BERT to obtain a sentence feature vector group, namely, sequence 1 [CLS1, CLS2, ...,], where sequence 1 is a paragraph-level [CLS] sequence;

[0116] All characters of the paragraph are encoded by BERT to obtain a character feature vector group, i.e., sequence 2 [[token_embdedding1, token_embdedding2, …], …], where sequence 2 is the paragraph-level [token_embdedding] sequence;

[0117] Input sequence 1 [CLS1, CLS2, ...,] into the trained TextCNN model of the text classification model for classification, and obtain the paragraph category vector;

[0118] Input sequence 2 [[token_embdedding1, token_embdedding2, …], …] into the trained BERT-CRF model of the entity recognition model for entity recognition to obtain an entity category vector; and

[0119] Based on the paragraph category vector and the entity category vector, an input vector is constructed, and the input vector is input into the Softmax of the entity recognition model for entity recognition to obtain the entity recognition result.

[0120] In view of the difficulty of paragraph-level entity recognition, the present invention discloses an entity recognition model constructed by BERT model + TextCNN model + CRF model + Softmax model, which combines paragraph semantic information and entity type information to effectively eliminate ambiguity and improve the accuracy of entity recognition.

[0121] According to an embodiment of the present disclosure, an entity recognition model building device 1000 includes:

[0122] A sample acquisition module 1002 acquires a labeled text sample set, wherein the labeled text sample set includes a plurality of paragraph preset categories and a plurality of entity preset categories;

[0123] The sample processing module 1004 performs paragraph splitting on each text sample in the annotated text sample set, sentence splitting on each paragraph, and character splitting on each sentence;

[0124] A vectorization module 1006, which uses the language representation model to perform vectorization processing on each sentence of each paragraph to obtain a sentence feature vector group of each paragraph, and uses the language representation model to perform vectorization processing on each character of each sentence to obtain a character feature vector group of each sentence;

[0125] A TextCNN training module 1008, which trains the TextCNN model based on at least a plurality of preset paragraph categories and a sentence feature vector group of each paragraph to obtain a trained TextCNN model;

[0126] An entity model training module 1010 is configured to train an entity model based on at least a plurality of entity preset categories and a character feature vector group of each sentence in each paragraph to obtain a trained entity model;

[0127] The vector acquisition module 1012 uses the trained TextCNN model to perform paragraph category recognition on the vectorized text sample to obtain a paragraph category vector, and uses the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector;

[0128] The Softmax model training module 1014 constructs an input vector based on the paragraph category vector and the entity category vector, and inputs the input vector into the Softmax model for iterative training to obtain a trained Softmax model.

[0129] Figure 3 An entity recognition model building device 1000 in the form of an electronic device is shown. Figure 3 As shown, the entity recognition model building device 1000 in the form of an electronic device includes a sample acquisition module 1002, a sample processing module 1004, a vectorization module 1006, a TextCNN training module 1008, an entity model training module 1010, a vector acquisition module 1012 and a vector acquisition module 1012.

[0130] For the entity recognition model building device 1000 of the above-mentioned embodiment, preferably, by calculating the loss function, it is determined whether the threshold is met. If so, the iterative training is terminated to obtain the trained Softmax model.

[0131] Among them, the vectorization module can fine-tune the language representation model so that some parameters of the language representation model are adjusted.

[0132] Among them, the training process of the TextCNN training module includes:

[0133] TextCNN is fine-tuned so that some parameters of the TextCNN model are adjusted.

[0134] The calculation of the loss function is based on the loss function of the training of the entity model (such as the CRF model) and the loss function of the training of the Softmax.

[0135] According to an entity recognition device 2000 of an embodiment of the present disclosure, entity recognition is performed using an entity recognition model established by the entity recognition model establishment method of any of the above embodiments, including:

[0136] The module 2002 for obtaining text to be recognized is used to obtain text to be recognized;

[0137] The vector group acquisition module 2004 is used to vectorize the text to be recognized through the language representation model to obtain the sentence feature vector group of each paragraph and the character feature vector group of each sentence in the paragraph;

[0138] A category vector acquisition module 2006 is used to use the trained TextCNN model to perform paragraph category recognition on the sentence feature vector group of the paragraph to obtain a paragraph category vector, and use the trained entity model to perform entity recognition on the character feature vector group of each sentence in the paragraph to obtain an entity category vector; and

[0139] The entity recognition module 2008 constructs an input vector based on the paragraph category vector and the entity category vector, and inputs the input vector into the trained Softmax to perform entity classification to obtain the entity category.

[0140] Figure 4 An entity identification device 2000 in the form of an electronic device is shown. Figure 4 As shown, the entity recognition device 2000 in the form of an electronic device includes a module 2002 for obtaining text to be recognized, a vector group acquisition module 2004, a category vector acquisition module 2006 and an entity recognition module 2008.

[0141] Figure 3 The entity recognition model building device 1000 in the form of an electronic device is shown. Figure 4The entity identification device 2000 in the form of an electronic device shown may include corresponding modules for executing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be executed by the corresponding module, and the device may include one or more of these modules. The module may be one or more hardware modules specifically configured to execute the corresponding steps, or implemented by a processor configured to execute the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0142] Figure 3 as well as Figure 4 The hardware structure shown can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0143] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0144] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0145] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0146] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0147] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0148] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0149] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0150] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments / methods or examples. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0151] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0152] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for establishing an entity recognition model, characterized in that: include: S102, obtaining a marked text sample set, wherein the marked text sample set includes a plurality of preset paragraph categories and a plurality of preset entity categories; S104, performing paragraph splitting on each text sample in the annotated text sample set, performing sentence splitting on each paragraph, and performing character splitting on each sentence; S106, using the language representation model to vectorize each sentence of each paragraph to obtain a sentence feature vector group [CLS1, CLS2, ...] of each paragraph, and using the language representation model to vectorize each character of each sentence to obtain a character feature vector group [[token_embdedding1, token_embdedding2, ...], ...] of each sentence; S108, training the TextCNN model based at least on the plurality of preset paragraph categories and the sentence feature vector groups of each paragraph to obtain a trained TextCNN model, including fine-tuning the TextCNN; S110, training an entity model based at least on the plurality of preset entity categories and a character feature vector group of each sentence in each paragraph to obtain a trained entity model; S112, using the trained TextCNN model to perform paragraph category recognition on the vectorized text sample to obtain a paragraph category vector, and using the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector; S114, constructing an input vector based on the paragraph category vector and the entity category vector, and inputting the input vector into a Softmax model for iterative training to obtain a trained Softmax model; The method further includes, by calculating the loss function, determining whether a threshold is satisfied, and if satisfied, ending the iterative training to obtain the trained Softmax model; The loss function is calculated based on the loss function of the training of the entity model and the loss function of the training of the Softmax model, and is expressed as CRF Loss+k*Softmax Loss, where k is a proportional coefficient and has a value greater than 0.

2. An entity recognition method, characterized in that: Using the entity recognition model described in claim 1 to perform entity recognition includes: Get the text to be recognized; Vectorizing the text to be recognized through the language representation model to obtain a sentence feature vector group of each paragraph, and obtaining a character feature vector group of each sentence in the paragraph; Using the trained TextCNN model to perform paragraph category recognition on the sentence feature vector group of the paragraph to obtain a paragraph category vector, and using the trained entity model to perform entity recognition on the character feature vector group of each sentence in the paragraph to obtain an entity category vector; and An input vector is constructed based on the paragraph category vector and the entity category vector, and the input vector is input into the trained Softmax to perform entity classification to obtain the entity category.

3. An entity recognition model building device, characterized in that: include: A sample acquisition module is used to acquire a marked text sample set, wherein the marked text sample set includes a plurality of preset paragraph categories and a plurality of preset entity categories; A sample processing module is used to split each text sample in the annotated text sample set into paragraphs, split each paragraph into sentences, and split each sentence into characters; A vectorization module, which uses a language representation model to perform vectorization processing on each sentence in each paragraph to obtain a sentence feature vector group of each paragraph, and uses a language representation model to perform vectorization processing on each character of each sentence to obtain a character feature vector group of each sentence; A TextCNN training module, which trains the TextCNN model based on at least the plurality of preset paragraph categories and the sentence feature vector groups of each paragraph to obtain a trained TextCNN model; An entity model training module, which trains the entity model based on at least the plurality of entity preset categories and the character feature vector groups of each sentence in each paragraph to obtain a trained entity model; a vector acquisition module, which uses the trained TextCNN model to perform paragraph category recognition on the vectorized text sample to obtain a paragraph category vector, and uses the trained entity model to perform entity recognition on the vectorized text sample to obtain an entity category vector; and A Softmax model training module, which constructs an input vector based on the paragraph category vector and the entity category vector, and inputs the input vector into the Softmax model for iterative training to obtain a trained Softmax model; Wherein, by calculating the loss function, it is determined whether the threshold is met, and if so, the iterative training is terminated to obtain the trained Softmax model; The loss function is calculated based on the loss function of the entity model training and the loss function of the Softmax training, and is expressed as CRF Loss+k*Softmax Loss, where k is a proportional coefficient and is a number greater than 0.

4. An entity recognition device, characterized in that: Using the entity recognition model established by the entity recognition model establishment method according to claim 1 to perform entity recognition, comprising: A module for obtaining text to be recognized is used to obtain text to be recognized; A vector group acquisition module is used to vectorize the text to be recognized through the language representation model to obtain the sentence feature vector group [CLS1, CLS2, ...] of each paragraph, and obtain the character feature vector group of each sentence in the paragraph [[token_embdedding1,token_embdedding2,…],…]; A category vector acquisition module, used to use the trained TextCNN model to perform paragraph category recognition on the sentence feature vector group of the paragraph to obtain a paragraph category vector, and use the trained entity model to perform entity recognition on the character feature vector group of each sentence in the paragraph to obtain an entity category vector; and The entity recognition module constructs an input vector based on the paragraph category vector and the entity category vector, and inputs the input vector into the trained Softmax to perform entity classification to obtain the entity category.

5. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory so that the processor executes the method according to claim 1 or 2.

6. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the method according to claim 1 or 2 when executed by a processor.

Images