Natural language processing model construction method, system, electronic device and storage medium

Abstract

The present invention proposes a method, system, electronic device and storage medium for building a natural language processing model. The method and technical solution include using a joint extraction method to extract information, and mining information of different granularities at the same time, including word vectors and character vectors corresponding to part-of-speech information; in addition, the present invention performs negative sampling on the original training data to obtain a batch of negative samples to solve the low resource problem of the model and increase the recognition difficulty of the model. The present invention improves the implementation effect of unstructured text information extraction and improves the robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the field of data processing, and in particular relates to a natural language processing model construction method, system, electronic device and storage medium. Background Art

[0002] In the field of language processing, there are a large number of unstructured texts, especially news texts. There are many entities in these texts, and different entities have different relationships. Effective extraction of unstructured texts can assist us in the automatic understanding of texts and the construction of knowledge graphs.

[0003] Existing technologies mainly include template-based methods, pipeline-based information extraction methods and semi-supervised information extraction methods. However, the encoders of the above methods are not powerful enough, the encoded feature dimensions are not rich enough, it is impossible to train entity recognition and relationship classification at the same time, it is impossible to use the original data for joint training, it is impossible to solve the problem of error accumulation between models, and it is impossible to effectively enhance the data. When the data is relatively small, it has a relatively large impact on the effect of the model. Summary of the invention

[0004] The embodiments of the present application provide a natural language processing model construction method, system, electronic device and storage medium to at least solve the problem of poor extraction effect of current unstructured text information.

[0005] In a first aspect, an embodiment of the present application provides a method for constructing a natural language processing model, comprising:

[0006] S101. Annotate original training data in the text sample;

[0007] S102. Negatively sample the original training data to obtain negative example data;

[0008] S103. Combining the original training data and the negative example data into final training data;

[0009] S104. Using a natural language processing tool to obtain part-of-speech information in the text sample, and training a first word vector according to the final training data;

[0010] S105. Converting the words in the text sample into a word vector and a second word vector, and combining the word vector and the second word vector;

[0011] S106. Obtaining an entity classification loss function of the text sample according to the first word vector, the character vector and the second word vector;

[0012] S107. Obtaining a relation classification loss function of the text sample according to the relation information of the first word vector, the character vector, the second word vector and the text sample;

[0013] S108. Add the entity classification loss function and the relationship classification loss function to obtain a joint loss function, and perform back-propagation gradient operation on the joint loss function to obtain a natural language processing model.

[0014] Preferably, the original training data includes text, entities in the text, lengths corresponding to the text, sets of labels for the entities, positions of text pairs, and sets of labels for the text pairs.

[0015] Preferably, the step S102 includes specifying the number of entities and the number of relationships of the negative examples, acquiring different entities and forming corresponding relationships according to the different entities.

[0016] Preferably, the step S104 includes using Word2Vec when converting the first word vector; the first word vector is the word vector corresponding to the part-of-speech information.

[0017] Preferably, the step S105 includes using RoBERTa when converting the word vector, and using Word2Vec when converting the second word vector.

[0018] Preferably, step S106 includes inputting the CLS vector information, the vector information of the entity length and the vector information after the entity is subjected to maximum pooling into the entity classification model and concatenating them, and performing classification processing using the softmax function.

[0019] Preferably, the step S107 includes: inputting the head entity vector information, the tail entity vector information and the vector information after maximum pooling of the context information between the two entities into the relationship classification model and concatenating them, and using the softmax function for classification processing.

[0020] In a second aspect, an embodiment of the present application provides a natural language processing model construction system, which is applicable to the above-mentioned natural language processing model construction method, including:

[0021] Preprocessing unit: annotating original training data in the text sample, negatively sampling the original training data to obtain negative example data, and merging the original training data and the negative example data into final training data;

[0022] A vector conversion unit: using a natural language processing tool to obtain part-of-speech information in the text sample, training a first word vector according to the final training data, converting words in the text sample into a word vector and a second word vector, and combining the word vector and the second word vector;

[0023] An entity classification loss function acquisition unit: acquiring an entity classification loss function of the text sample according to the first word vector, the character vector and the second word vector;

[0024] A relation classification loss function acquisition unit: acquiring a relation classification loss function of the text sample according to the relation information of the first word vector, the character vector, the second word vector and the text sample;

[0025] Model construction unit: adding the entity classification loss function and the relationship classification loss function to obtain a joint loss function, and performing back-propagation gradient operation on the joint loss function to obtain a natural language processing model.

[0026] In some of the embodiments, the original training data includes text, entities in the text, lengths of the corresponding texts, sets of labels for the entities, positions of text pairs, and sets of labels for the text pairs.

[0027] In some embodiments, the preprocessing unit includes specifying the number of entities and the number of relationships of the negative examples, obtaining different entities and forming corresponding relationships according to the different entities.

[0028] In some embodiments, the vector conversion unit includes using Word2Vec when converting the first word vector; the first word vector is the word vector corresponding to the part-of-speech information.

[0029] In some embodiments, the vector conversion unit includes using RoBERTa when converting the word vector and using Word2Vec when converting the second word vector.

[0030] In some of the embodiments, the entity classification loss function acquisition unit includes inputting CLS vector information, vector information of entity length, and vector information after maximum pooling of the entity into the entity classification model and splicing them, and using the softmax function for classification processing.

[0031] In some of the embodiments, the relationship classification loss function acquisition unit includes: inputting the head entity vector information, the tail entity vector information, and the vector information after maximum pooling of the context information between the two entities into the relationship classification model and splicing them, and using the softmax function for classification processing.

[0032] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a natural language processing model construction method as described in the first aspect above.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a natural language processing model construction method as described in the first aspect above.

[0034] Compared with the related art, the embodiment of the present application provides a method for constructing a natural language processing model:

[0035] 1. Use the joint extraction method to extract information, which can better utilize the entity information and reduce the error accumulation problem caused by pipeline. In addition, a model is finally obtained, which is more convenient for model deployment.

[0036] 2. Using encoding information of different granularities, including pos-level information, character-level information based on RoBERTa, word-level information based on word2vec, and CLS information in RoBERTa, can more fully encode text information.

[0037] 3. Perform negative sampling on the original data to obtain a batch of negative samples to solve the low resource problem of the model and increase the recognition difficulty of the model to improve the robustness of the model.

[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 is a flow chart of a method for constructing a natural language processing model according to an embodiment of the present application;

[0041] Figure 2 It is a framework diagram of a system for building a natural language processing model according to an embodiment of the present application;

[0042] Figure 3 is a framework diagram of an electronic device according to an embodiment of the present application;

[0043] In the above picture:

[0044] 11. Preprocessing unit; 12. Vector conversion unit; 13. Entity classification loss function acquisition unit; 14. Relation classification loss function acquisition unit; 15. Model building unit; 20. Bus; 21. Processor; 22. Memory; 23. Communication interface. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0046] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0047] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0048] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a kind of", "the" and the like involved in this application do not indicate a quantitative limitation and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices.

[0049] In the field of language processing, there are a large number of unstructured texts. There are many entities in these texts, and different entities have different relationships. Effective extraction of unstructured texts can assist us in the automatic understanding of texts and the construction of knowledge graphs. The embodiments of the present invention propose a natural language processing model construction method, system, electronic device and storage medium, which can be applied to information extraction of unstructured texts. The embodiments of the present invention can be used for unstructured texts in the field of news.

[0050] The following is an explanation of some of the professional terms involved in the present invention:

[0051] Information extraction (IE) refers to extracting corresponding entities and the relationships between entities from a text. Specific technologies include named entity recognition (NER) and relationship classification (RE). Named entity recognition (NER) is a very basic task in the fields of NLP and knowledge graphs. It aims to locate and classify named entities in text into predefined categories, such as people, organizations, locations, time expressions, quantities, monetary values, percentages, etc. The effect of named entity recognition directly determines the effect of downstream tasks. Relation classification (RE) is a form of text classification. It classifies the extracted entity pairs and text information to obtain the relationships between entity pairs.

[0052] Word2vec is a group of related models used to generate word vectors. These models are shallow, two-layer neural networks that are trained to reconstruct linguistic word text. The network is represented by words and needs to guess the input words in adjacent positions. Under the word bag model assumption in word2vec, the order of words is not important. After training, the word2vec model can be used to map each word to a vector, which can be used to represent the relationship between words. The vector is the hidden layer of the neural network.

[0053] BERT is a pre-trained model released by Google, which uses a fixed mask mode and a relatively small batch size for training, so the power of BERT needs to be further developed. In 2019, Facebook released a deeper and larger pre-trained model, ROBERTA. ROBERTA uses a larger batch size, more data, longer training time, and longer sentences. At the same time, it uses a dynamic mask method to remove the NSP task in BERT, so that ROBERTA has achieved better performance and surpasses BERT in all indicators.

[0054] Softmax is a function used in the classification process to implement multi-classification. Simply put, it maps some output neurons to real numbers between (0-1) and normalizes them to ensure that the sum is 1, so that the sum of the probabilities of multiple classifications is exactly 1.

[0055] Max pooling is a commonly used pooling operation that reduces the amount of data by maximizing the value. Its usual operation is to divide the input image into several rectangular regions and output the maximum value for each subdomain. Currently, in addition to max pooling, average pooling is also commonly used. It reduces the complex calculations from the upper hidden layer, is not affected by the tilt or rotation of the target, and can effectively reduce the sampling method of data dimension.

[0056] See also Figure 1 , is a flow chart of a method for constructing a natural language processing model according to an embodiment of the present application, comprising the following steps:

[0057] S101. Annotate original training data in the text sample;

[0058] S102. Negatively sample the original training data to obtain negative example data;

[0059] S103. Combining the original training data and the negative example data into final training data;

[0060] S104. Using a natural language processing tool to obtain part-of-speech information in the text sample, and training a first word vector according to the final training data;

[0061] S105. Converting the words in the text sample into a word vector and a second word vector, and combining the word vector and the second word vector;

[0062] S106. Obtaining an entity classification loss function of the text sample according to the first word vector, the character vector and the second word vector;

[0063] S107. Obtaining a relation classification loss function of the text sample according to the relation information of the first word vector, the character vector, the second word vector and the text sample;

[0064] S108. Add the entity classification loss function and the relationship classification loss function to obtain a joint loss function, and perform back-propagation gradient operation on the joint loss function to obtain a natural language processing model.

[0065] In order to solve the error accumulation problem caused by the pipeline in the prior art, an embodiment of the present invention proposes a joint method to simultaneously learn the entities in the text and the relationship between entity pairs, that is, to consider the entity classification loss function and the relationship classification loss function at the same time.

[0066] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0067] The original training data includes text, entities in the text, length of the text, a label set of the entity, the position of the text pair, and a label set of the text pair.

[0068] Wherein, the step S102 includes specifying the number of entities and the number of relationships of the negative examples, acquiring different entities and forming corresponding relationships according to the different entities.

[0069] Wherein, the step S104 includes using Word2Vec when converting the first word vector; the first word vector is the word vector corresponding to the part-of-speech information.

[0070] ROBERTA is used as the encoder. Since ROBERTA uses a larger batch size, more data, longer training time, longer sentences, dynamic masks and other measures, its performance is better than BERT in each data set. At the same time, the length of each entity is considered and a width vector is assigned to each entity. Since the models of the BERT series are all based on byte segmentation, word-level information and POS-level information are not considered. Therefore, the embodiment of the present invention proposes to integrate word vector information and POS-level information into the encoding of features.

[0071] Wherein, the step S105 includes using RoBERTa when converting the word vector, and using Word2Vec when converting the second word vector.

[0072] In some of the embodiments, the pre-trained model used when converting the word vector may also be any one of xlnet, albert, and t5.

[0073] Wherein, the step S106 includes inputting the CLS vector information, the vector information of the entity length and the vector information after the entity is subjected to maximum pooling into the entity classification model and concatenating them, and performing classification processing using the softmax function.

[0074] Wherein, the step S107 includes: inputting the head entity vector information, the tail entity vector information and the vector information after the maximum pooling of the context information between the two entities into the relationship classification model and splicing them, and using the softmax function for classification processing.

[0075] The embodiment of the present application provides a natural language processing model construction system, which is applicable to the above-mentioned natural language processing model construction method. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0076] Figure 2 The framework diagram of the natural language processing model construction system according to the embodiment of the present application includes a preprocessing unit 11, a vector conversion unit 12, an entity classification loss function acquisition unit 13, a relationship classification loss function acquisition unit 14, and a model construction unit 15, wherein:

[0077] Preprocessing unit 11: annotating original training data in the text sample, negatively sampling the original training data to obtain negative example data, and merging the original training data and the negative example data into final training data;

[0078] Vector conversion unit 12: using a natural language processing tool to obtain part-of-speech information in the text sample, training a first word vector according to the final training data, converting words in the text sample into word vectors and second word vectors, and combining the word vectors and the second word vectors;

[0079] An entity classification loss function acquisition unit 13: acquires an entity classification loss function of the text sample according to the first word vector, the character vector and the second word vector;

[0080] A relation classification loss function acquisition unit 14: acquires a relation classification loss function of the text sample according to the relation information of the first word vector, the character vector, the second word vector and the text sample;

[0081] Model building unit 15: adding the entity classification loss function and the relationship classification loss function to obtain a joint loss function, and performing a back-propagation gradient operation on the joint loss function to obtain a natural language processing model.

[0082] In some of the embodiments, the original training data includes text, entities in the text, lengths of the corresponding texts, sets of labels for the entities, positions of text pairs, and sets of labels for the text pairs.

[0083] In some embodiments, the preprocessing unit 11 includes specifying the number of entities and the number of relationships of the negative examples, obtaining different entities and forming corresponding relationships according to the different entities.

[0084] In some embodiments, the vector conversion unit 12 includes using Word2Vec when converting the first word vector; the first word vector is the word vector corresponding to the part-of-speech information.

[0085] In some embodiments, the vector conversion unit 12 includes using RoBERTa when converting the word vector and using Word2Vec when converting the second word vector.

[0086] In some of the embodiments, the entity classification loss function acquisition unit 13 includes inputting CLS vector information, vector information of entity length, and vector information after maximum pooling of the entity into the entity classification model and concatenating them, and using the softmax function for classification processing.

[0087] In some of the embodiments, the relationship classification loss function acquisition unit 14 includes: inputting the head entity vector information, the tail entity vector information, and the vector information after maximum pooling of the context information between the two entities into the relationship classification model and splicing them, and using the softmax function for classification processing.

[0088] It should be noted that the above-mentioned units can be functional units or program units, and can be implemented by software or hardware. For units implemented by hardware, the above-mentioned units can be located in the same processor; or the above-mentioned units can be located in different processors in any combination.

[0089] In addition, combined Figure 1 The natural language processing model building method described in the embodiment of the present application can be implemented by an electronic device. Figure 3 Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application.

[0090] The computer device may include a processor 21 and a memory 22 storing computer program instructions.

[0091] Specifically, the processor 21 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0092] Among them, the memory 22 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 22 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 22 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 22 may be inside or outside the data processing device. In a specific embodiment, the memory 22 is a non-volatile memory. In a specific embodiment, the memory 22 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (ErasableProgrammable Read-Only Memory, EPROM for short), an electrically erasable PROM (ElectricallyErasableProgrammableRead-OnlyMemory, EEPROM for short), an electrically alterable ROM (ElectricallyAlterableRead-OnlyMemory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0093] The memory 22 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 21 .

[0094] The processor 21 implements any one of the natural language processing model building methods in the above embodiments by reading and executing computer program instructions stored in the memory 22.

[0095] In some embodiments, the computer device may further include a communication interface 23 and a bus 20. Figure 2 As shown, the processor 21, the memory 22, and the communication interface 23 are connected via a bus 20 and communicate with each other.

[0096] The communication interface 23 can realize data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0097] The bus 20 includes hardware, software or both, and couples the components of the electronic device to each other. The bus 20 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 20 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 20 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0098] The computer device can execute a natural language processing model construction method in an embodiment of the present application.

[0099] In addition, in combination with a natural language processing model construction method in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any natural language processing model construction method in the above embodiment is implemented.

[0100] The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes.

[0101] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a natural language processing model, characterized in that: include: S101. Annotate original training data in the text sample; S102. Negatively sample the original training data to obtain negative example data; S103. Combining the original training data and the negative example data into final training data; S104. Using a natural language processing tool to obtain part-of-speech information in the text sample, and training a first word vector according to the final training data; S105. Converting the words in the text sample into a word vector and a second word vector, and combining the word vector and the second word vector; S106. Obtaining an entity classification loss function of the text sample according to the first word vector, the character vector and the second word vector; S107. Obtaining a relation classification loss function of the text sample according to the relation information of the first word vector, the character vector, the second word vector and the text sample; S108. Adding the entity classification loss function and the relationship classification loss function to obtain a joint loss function, and performing a back-propagation gradient operation on the joint loss function to obtain a natural language processing model; Among them, the step S102 includes specifying the number of entities and the number of relationships of negative examples, obtaining different entities and forming corresponding relationships according to the different entities; the step S104 includes using Word2Vec when converting the first word vector; the first word vector is the word vector corresponding to the part-of-speech information; the step S105 includes using RoBERTa when converting the word vector, and using Word2Vec when converting the second word vector; the step S106 includes inputting the CLS vector information, the vector information of the entity length, and the vector information after the maximum pooling of the entity into the entity classification model and splicing them, and using the softmax function for classification processing; the step S107 includes: inputting the head entity vector information, the tail entity vector information, and the vector information after the maximum pooling of the context information between the two entities into the relationship classification model and splicing them, and using the softmax function for classification processing.

2. The method for constructing a natural language processing model according to claim 1, wherein: The original training data includes text, entities in the text, length of the text, a label set of the entity, the position of the text pair, and a label set of the text pair.

3. A natural language processing model building system, characterized in that: include: Preprocessing unit: annotating original training data in the text sample, negatively sampling the original training data to obtain negative example data, and merging the original training data and the negative example data into final training data; A vector conversion unit: using a natural language processing tool to obtain part-of-speech information in the text sample, training a first word vector according to the final training data, converting words in the text sample into a word vector and a second word vector, and combining the word vector and the second word vector; An entity classification loss function acquisition unit: acquiring an entity classification loss function of the text sample according to the first word vector, the character vector and the second word vector; A relation classification loss function acquisition unit: acquiring a relation classification loss function of the text sample according to the relation information of the first word vector, the character vector, the second word vector and the text sample; Model building unit: adding the entity classification loss function and the relationship classification loss function to obtain a joint loss function, and performing back propagation gradient operation on the joint loss function to obtain a natural language processing model; Among them, the preprocessing unit includes specifying the number of entities and the number of relationships of negative examples, obtaining different entities and forming corresponding relationships according to the different entities; the vector conversion unit includes using Word2Vec when converting the first word vector; the first word vector is the word vector corresponding to the part-of-speech information; the vector conversion unit includes using RoBERTa when converting the word vector, and using Word2Vec when converting the second word vector; the entity classification loss function acquisition unit includes inputting CLS vector information, vector information of entity length and vector information after maximum pooling of the entity into the entity classification model and splicing, and using the softmax function for classification processing; the relationship classification loss function acquisition unit includes: inputting the head entity vector information, the tail entity vector information and the vector information after maximum pooling of the context information between the two entities into the relationship classification model and splicing, and using the softmax function for classification processing.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the natural language processing model construction method as described in any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the natural language processing model construction method as claimed in claim 1 or 2 is implemented.

Images