Entity recognition model establishment method, system, electronic device and medium

Abstract

The present application discloses a method, system, electronic device and medium for establishing an entity recognition model. The method for establishing an entity recognition model includes: performing language masking on a first entity in an input representation vector, obtaining a first language masked text, selecting non-entity words from the word segmentation result of the input representation vector and performing language masking on the words, and obtaining a second language masked text; inputting the first language masked text and the second language masked text into the encoding layer of an initial Transformer model, and obtaining a first hidden layer vector and a third hidden layer vector from the output result; calculating a first training matrix and the first hidden layer vector, and obtaining a probability distribution vector, and then calculating a second training matrix and the third hidden layer vector, and obtaining a classification probability distribution vector; performing back propagation calculation on the initial Transformer model using the probability distribution vector and the classification probability distribution vector, and updating the encoding layer to obtain a final Transformer model.

Description

Technical Field

[0001] The present application relates to the field of knowledge graph technology, and in particular to a method, system, electronic device and medium for establishing an entity recognition model. Background Art

[0002] Named Entities Recognition (NER) is a basic task of Natural Language Processing (NLP), and its purpose is to identify named entities such as names of people, places, and organizations in a corpus. The main technologies for solving entity recognition include rule-based and dictionary-based methods, statistical-based methods, and deep learning-based methods. The main statistical-based methods are: Hidden Markov Model (HMM), Maximum Entropy (ME), Support Vector Machine (SVM), and Conditional Random Fields (CRF). The main deep learning-based methods are Bi-LSTM, Bi-LSTM+CRF, and methods based on pre-trained models. However, the current entity recognition methods all treat entity recognition as a sequence labeling task, without considering the linguistic characteristics of language entities in natural language. The introduction of relevant semantic information such as part of speech in the language sequence and the extraction of contextual information are of great significance to the discovery and recognition of entities. Therefore, how to deepen the mining of language texts and improve the accuracy of entity recognition has become a problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a method, system, electronic device and medium for establishing an entity recognition model, so as to at least solve the problem of lack of consideration for the linguistic characteristics of language entities in natural language, resulting in low accuracy in entity discovery and recognition, etc. through the present invention.

[0004] The present invention provides a method for establishing an entity recognition model, comprising:

[0005] The language masked text acquisition step: language masking is performed on the first entity in the input representation vector to obtain the first language masked text, and then non-entity words are selected from the word segmentation results of the input representation vector and language masked to obtain the second language masked text;

[0006] Hidden layer vector acquisition step: inputting the first language masked text and the second language masked text into the encoding layer of the initial Transformer model, and obtaining the first hidden layer vector and the third hidden layer vector from the output results;

[0007] Model pre-training step: after calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector;

[0008] Model acquisition step: using the probability distribution vector and the classification probability distribution vector to perform back propagation calculation on the initial Transformer model, updating the encoding layer, and obtaining the final Transformer model.

[0009] In the above-mentioned entity recognition model establishment method, the language masking text acquisition step includes:

[0010] Concatenate the part-of-speech representation vector, the word vector, and the position representation vector of the text to obtain the input representation vector;

[0011] After counting the number of entities included in the input representation vector through the entity information of the input representation vector, a preset number of first entities are selected from the entities, and language masking is performed on the first entities to obtain the first language masked text.

[0012] In the above-mentioned entity recognition model establishment method, the language masked text acquisition step further includes:

[0013] After performing word segmentation on the input representation vector to obtain the word segmentation result, a preset number of non-entity words are selected from the word segmentation result, and language masking is performed on the non-entity words to obtain the second language masked text.

[0014] In the above-mentioned entity recognition model establishment method, the hidden layer vector acquisition step includes:

[0015] Obtaining a plurality of second hidden layer vectors of masked texts having the same masking type in the first language masked text and the second language masked text through the output result of the initial Transformer model;

[0016] The third hidden layer vector is obtained by summing up a plurality of the second hidden layer vectors.

[0017] In the above-mentioned entity recognition model establishment method, the model training step includes:

[0018] After establishing the first training matrix in the initial Transformer model, calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector;

[0019] After the second training matrix is ​​established in the initial Transformer model, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector of the first language masked text and the second language masked text.

[0020] In the above-mentioned entity recognition model establishment method, the model acquisition step includes:

[0021] The probability distribution vector and the classification probability distribution vector are used to perform back propagation calculation on the initial Transformer model to obtain a loss function value.

[0022] In the above-mentioned entity recognition model establishment method, the model acquisition step also includes:

[0023] After updating the initial Transformer model parameters in the encoding layer according to the loss function value, the final Transformer model is obtained.

[0024] The present invention also provides an entity recognition model establishment system, wherein the entity recognition model establishment method described above is applicable, and the entity recognition model establishment system comprises:

[0025] A language masked text acquisition unit: performs language masking on the first entity in the input representation vector, obtains the first language masked text, selects non-entity words from the word segmentation result of the input representation vector and performs language masking on the non-entity words, and obtains the second language masked text;

[0026] A hidden layer vector acquisition unit: inputting the first language masked text and the second language masked text into the encoding layer of the initial Transformer model, and obtaining a first hidden layer vector and a third hidden layer vector from the output results;

[0027] Model pre-training unit: after calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector;

[0028] Model acquisition unit: Use the probability distribution vector and the classification probability distribution vector to perform back propagation calculation on the initial Transformer model, update the encoding layer, and obtain the final Transformer model.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned entity recognition model establishment methods when executing the computer program.

[0030] The present invention also provides an electronic device readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by the processor, any of the above-mentioned entity recognition model establishment methods is implemented.

[0031] Compared with the related art, the entity recognition model establishment method, system, electronic device and medium proposed in the present invention incorporate an input representation vector into the input representation stage. Since the input representation vector contains part-of-speech information, the integration of part-of-speech information increases the semantic richness of the model; the model's learning and capture of syntactic grammar are enhanced by masking entities and masking non-entity words; in the pre-training task, not only the language representation method is obtained, but also the relationship between the entity and the text context is learned by predicting entities and non-entity words, and the category of the masked part is predicted, which enhances the model's distinction between entities and ordinary words, and improves the accuracy of entity recognition and the ability to build graphs.

[0032] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0034] Figure 1 is a flow chart of a method for establishing an entity recognition model according to an embodiment of the present application;

[0035] Figure 2 is an input representation vector display diagram according to an embodiment of the present application;

[0036] Figure 3 is a pre-training task display diagram according to an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of the entity recognition model establishment system of the present invention;

[0038] Figure 5 is a framework diagram of an electronic device according to an embodiment of the present application.

[0039] Wherein, the accompanying drawings are marked as follows:

[0040] Language masked text acquisition unit: 51;

[0041] Hidden layer vector acquisition unit: 52;

[0042] Model pre-training units: 53;

[0043] Model acquisition unit: 54;

[0044] Bus:80;

[0045] Processor: 81;

[0046] Memory: 82;

[0047] Communication interface:83. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0049] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the disclosure of the present application, some changes in design, manufacturing or production based on the technical content disclosed in the present application are just conventional technical means, and should not be understood as the disclosure of the present application is insufficient.

[0050] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0051] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0052] The present invention incorporates part-of-speech information in the input representation stage and performs language masking on entity and non-entity words through language masking, and then inputs the masking results into the model to train the model, thereby deepening the mining and learning of language texts and improving the accuracy of entity recognition and discovery.

[0053] The present invention is described below in conjunction with specific embodiments.

[0054] Embodiment 1

[0055] This embodiment provides a method for establishing an entity recognition model. Figures 1 to 3 , Figure 1 is a flow chart of a method for establishing an entity recognition model according to an embodiment of the present application; Figure 2 is an input representation vector display diagram according to an embodiment of the present application; Figure 3 is a pre-training task display diagram according to an embodiment of the present application, such as Figures 1 to 3 As shown, the entity recognition model establishment method includes the following steps:

[0056] Language masked text acquisition step S1: language masking is performed on the first entity in the input representation vector to obtain the first language masked text, and then non-entity words are selected from the word segmentation result of the input representation vector and language masked to obtain the second language masked text;

[0057] Hidden layer vector acquisition step S2: inputting the first language masked text and the second language masked text into the encoding layer of the initial Transformer model, and obtaining the first hidden layer vector and the third hidden layer vector from the output results;

[0058] Model pre-training step S3: After calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector;

[0059] Model acquisition step S4: Use the probability distribution vector and the classification probability distribution vector to perform back propagation calculation on the initial Transformer model, update the encoding layer, and obtain the final Transformer model.

[0060] In an embodiment, the language masking text acquisition step S1 includes:

[0061] Concatenate the part-of-speech representation vector, word vector, and position representation vector of the text to obtain the input representation vector;

[0062] After counting the number of entities included in the input representation vector by inputting the entity information of the representation vector, a preset number of the first entities are selected from the entities, and language masking is performed on the first entities to obtain a first language masked text.

[0063] In the specific implementation, taking the position of a character as an example, first there is a vector matrix, and the position of the character is obtained, such as "2", then the second vector is taken out from the vector matrix as the position representation vector of the character. Similarly, for example, the character "I" has its serial number in the dictionary, and this dictionary is custom-defined, so the corresponding character vector is found according to the serial number. The part of speech of the character "I" is also found by establishing a part-of-speech dictionary to find the corresponding part-of-speech vector; after obtaining the part-of-speech representation vector, character vector and position representation vector of each character in the text, the above three vectors are concatenated to obtain the input representation vector of the text.

[0064] The number of entities contained in the input representation vector is counted by the entity information in the input representation vector. mask ) and randomly selects a preset number of entities from the entity mask =[50% Entity num ], round up to half of the number of entities) first entities, and perform language masking on the first entities to obtain the first language masked text, that is, all the words contained in the selected entity are replaced by the [mask] sign.

[0065] In an embodiment, the language masked text acquisition step S1 further includes:

[0066] After performing word segmentation on the input representation vector to obtain the word segmentation result, a preset number of non-entity words are selected from the word segmentation result, and language masking is performed on the non-entity words to obtain a second language masked text.

[0067] In a specific implementation, after the input representation vector is segmented to obtain the segmentation result, a preset number of words (Word num ) non-entity words, and perform language masking on the non-entity words to obtain the second language masked text. num =Random[0,Entity mask ](between 0 and Entity mask Random integers are picked from a closed interval).

[0068] In an embodiment, the hidden layer vector acquisition step S2 includes:

[0069] Obtain multiple second hidden layer vectors of masked texts with the same masking type in the first language masked text and the second language masked text through the output results of the initial Transformer model;

[0070] The multiple second hidden layer vectors are summed up to obtain the third hidden layer vector.

[0071] In the specific implementation, the so-called hidden layer vector (X8, ...X11) is the model output of the masked part. In addition, when the masked content is first input into the model, the model already knows which text content is masked, so in the output part the model naturally knows which masked content types are consistent; the first language masked text and the second language masked text are input into the encoding layer of the initial Transformer model, and the first hidden layer vector and multiple second hidden layer vectors of masked content with the same masking type are obtained from the output results, and then the multiple second hidden layer vectors are summed to calculate the third hidden layer vector The third latent vector is the sum of the latent vectors of the masked texts with the same masking type in the first language masked text and the second language masked text, where the same masking type refers to masked content with consistent masking content types. For example, "Minglue Technology" is masked as a unit when input into the model, then "Ming", "Lue", "Ke" and "Ji" belong to masked content with the same masking type.

[0072] In an embodiment, the model training step S3 includes:

[0073] After establishing the first training matrix in the initial Transformer model, the first training matrix and the first hidden layer vector are calculated to obtain a probability distribution vector;

[0074] After establishing the second training matrix in the initial Transformer model, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector of the first language masked text and the second language masked text.

[0075] In the specific implementation, the first training matrix is ​​established in the initial Transformer model After that, the first training matrix is ​​multiplied by the first hidden layer vector to obtain the non-entity vocabulary probability distribution vector (n-dimensional vector V); where m represents the dimension of the hidden layer vector X output by the encoder of the initial Transformer model, and n represents the dictionary size of the corpus.

[0076] Building the second training matrix in the initial Transformer model After that, the second training matrix and the third hidden layer vector Calculation is performed to obtain the classification probability distribution vector (U) of the first language masked text and the second language masked text.

[0077] In an embodiment, the model acquisition step S4 includes:

[0078] The probability distribution vector and the classification probability distribution vector are used to perform back propagation calculation on the initial Transformer model to obtain the loss function value.

[0079] In an embodiment, the model acquisition step S4 further includes:

[0080] After updating the initial Transformer model parameters in the encoding layer according to the loss function value, the final Transformer model is obtained.

[0081] Embodiment 2

[0082] Please refer to Figure 4 , Figure 4 The schematic diagram of the structure of the entity recognition model establishment system of the present invention is shown in FIG. Figure 4 As shown, the entity recognition model establishment system of the invention is applicable to the above-mentioned entity recognition model establishment method, and the entity recognition model establishment system includes:

[0083] The language masked text acquisition unit 51 performs language masking on the first entity in the input representation vector to obtain the first language masked text, then selects non-entity words from the word segmentation result of the input representation vector and performs language masking on the non-entity words to obtain the second language masked text;

[0084] Hidden layer vector acquisition unit 52: inputs the first language masked text and the second language masked text into the encoding layer of the initial Transformer model, and obtains a first hidden layer vector and a third hidden layer vector from the output result;

[0085] Model pre-training unit 53: after calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector, calculate the second training matrix and the third hidden layer vector to obtain the classification probability distribution vector;

[0086] Model acquisition unit 54: Use the probability distribution vector and the classification probability distribution vector to perform back propagation calculation on the initial Transformer model, update the encoding layer, and obtain a final Transformer model.

[0087] Embodiment 3

[0088] Combination Figure 5 As shown, this embodiment discloses a specific implementation of an electronic device. The electronic device may include a processor 81 and a memory 82 storing computer program instructions.

[0089] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0090] Among them, the memory 82 may include a large-capacity memory for data or instructions. For example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 82 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 82 may be inside or outside the abnormal data monitoring device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (ErasableProgrammable Read-Only Memory, FPROM for short), an electrically erasable PROM (Electrically ErasableProgrammable Read-Only Memory, EFPROM for short), an electrically alterable ROM (ElectricallyAlterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0091] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0092] The processor 81 implements any entity recognition model building method in the above embodiments by reading and executing computer program instructions stored in the memory 82.

[0093] In some of the embodiments, the electronic device may further include a communication interface 83 and a bus 80. Figure 5 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0094] The communication interface 83 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication port 83 can also implement data communication with other components such as: external devices, image / abnormal data monitoring equipment, databases, external storage, and image / abnormal data monitoring workstations.

[0095] The bus 80 includes hardware, software or both, and couples the components of the electronic device to each other. The bus 80 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0096] Electronic devices can be connected to the entity recognition model building system to achieve integration Figures 1 to 3 method.

[0097] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] In summary, part-of-speech information has positive significance for the discovery and recognition of named entities. Therefore, the present invention incorporates part-of-speech information in the input representation stage, and enhances the model's learning and capture of grammar through predictive masking. Then, the model's distinction between entities and common vocabulary is enhanced through model prediction tasks, thereby improving the accuracy of entity recognition.

[0099] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the protection scope of the attached claims.

Claims

1. A method for establishing an entity recognition model, characterized in that: Build an entity recognition prediction model to apply to Chinese entity recognition scenarios, including: The language masked text acquisition step: language masking is performed on the first entity in the input representation vector to obtain the first language masked text, and then non-entity words are selected from the word segmentation results of the input representation vector and language masked to obtain the second language masked text; Hidden layer vector acquisition step: inputting the first language masked text and the second language masked text into the encoding layer of the initial Transformer model, and obtaining the first hidden layer vector and the third hidden layer vector from the output results; Model pre-training step: after calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector; Model acquisition step: using the probability distribution vector and the classification probability distribution vector to perform back propagation calculation on the initial Transformer model, updating the encoding layer, and obtaining the final Transformer model.

2. The method for establishing an entity recognition model according to claim 1, characterized in that: The language masking text acquisition step comprises: Concatenate the part-of-speech representation vector, the word vector, and the position representation vector of the text to obtain the input representation vector; After counting the number of entities included in the input representation vector through the entity information of the input representation vector, a preset number of first entities are selected from the entities, and language masking is performed on the first entities to obtain the first language masked text.

3. The method for establishing an entity recognition model according to claim 1, characterized in that: The language masking text acquisition step also includes: After performing word segmentation on the input representation vector to obtain the word segmentation result, a preset number of non-entity words are selected from the word segmentation result, and language masking is performed on the non-entity words to obtain the second language masked text.

4. The method for establishing an entity recognition model according to claim 1, characterized in that: The hidden layer vector acquisition step comprises: Obtaining a plurality of second hidden layer vectors of masked texts having the same masking type in the first language masked text and the second language masked text through the output result of the initial Transformer model; The third hidden layer vector is obtained by summing up a plurality of the second hidden layer vectors.

5. The method for establishing an entity recognition model according to claim 1, characterized in that: The model pre-training step includes: After establishing the first training matrix in the initial Transformer model, calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector; After the second training matrix is ​​established in the initial Transformer model, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector of the first language masked text and the second language masked text.

6. The method for establishing an entity recognition model according to claim 1, characterized in that: The model acquisition step comprises: The probability distribution vector and the classification probability distribution vector are used to perform back propagation calculation on the initial Transformer model to obtain a loss function value.

7. The method for establishing an entity recognition model according to claim 6, characterized in that: The model acquisition step further includes: After updating the initial Transformer model parameters in the encoding layer according to the loss function value, the final Transformer model is obtained.

8. A system for establishing an entity recognition model, characterized in that: The entity recognition model establishment method according to any one of claims 1 to 7, wherein the entity recognition model establishment system comprises: A language masked text acquisition unit: performs language masking on the first entity in the input representation vector, obtains the first language masked text, selects non-entity words from the word segmentation result of the input representation vector and performs language masking on the non-entity words, and obtains the second language masked text; A hidden layer vector acquisition unit: inputting the first language masked text and the second language masked text into the encoding layer of the initial Transformer model, and obtaining a first hidden layer vector and a third hidden layer vector from the output results; Model pre-training unit: after calculating the first training matrix and the first hidden layer vector to obtain the probability distribution vector, the second training matrix and the third hidden layer vector are calculated to obtain the classification probability distribution vector; Model acquisition unit: Use the probability distribution vector and the classification probability distribution vector to perform back propagation calculation on the initial Transformer model, update the encoding layer, and obtain the final Transformer model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the entity recognition model establishment method according to any one of claims 1 to 7 is implemented.

10. An electronic device readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the entity recognition model establishment method as described in any one of claims 1 to 7.

Images