Joint extraction method, device and storage medium for entities and entity relationships

Abstract

The present application discloses a joint extraction method, device and storage medium for entities and entity relationships, including: determining text information to be processed; based on the text information to be processed, determining first word vectors corresponding to each word in the text information to be processed; determining first label vectors corresponding to each first word vector, wherein the first label vector is used to indicate the entity type corresponding to each word; based on each first label vector, generating a corresponding second label vector, wherein the second label vector further includes the semantic association between the first label vectors on the basis of the first label vector; splicing the first word vector and the corresponding second label vector respectively, and generating multiple splicing vectors; and determining the entity relationship between the entities of the text information to be processed based on the multiple splicing vectors.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method, device and storage medium for joint extraction of entities and entity relationships. Background Art

[0002] In natural language processing tasks, entity recognition and entity relationship extraction are key technologies for building knowledge graphs. Knowledge graphs can be divided into two levels: the schema layer and the data layer. The schema layer defines the entity types and entity relationship types stored in the knowledge graph, while the data layer stores the actual knowledge information, usually in the form of triples.

[0003] Since the construction of the model layer must first clarify the entities and entity relationships, in the knowledge extraction stage, it is necessary to extract unstructured data and structured data from the corresponding text information to be processed according to the requirements of the model layer. For structured data, Python automation scripts can be used to extract entities and entity relationships, while for unstructured data, existing models are generally used to extract entities and entity relationships from unstructured data.

[0004] However, in the process of identifying entities in the text information to be processed and extracting the entity relationships between entities, since the label vectors generated based on the CRF layer (corresponding to the text information to be processed input into the model) are relatively independent, the semantic correlation between the label vectors is not considered. Therefore, the entity relationships obtained based on the model cannot well reflect the correlation between entities, and will affect the representation of context, semantics and contextual relationships, and thus affect the accuracy of the constructed knowledge graph.

[0005] With regard to the technical problem existing in the above-mentioned prior art that the label vectors (corresponding to the text information to be processed input into the model) are relatively independent and the semantic correlation between the label vectors is not considered, the entity relationship obtained based on the model cannot well reflect the correlation between the entities and will affect the representation of the context, semantics and contextual relationship, no effective solution has been proposed so far. Summary of the invention

[0006] The embodiments of the present disclosure provide a joint extraction method for entities and entity relationships, so as to at least solve the technical problem existing in the prior art that the label vectors (corresponding to the text information to be processed input into the model) are relatively independent and the semantic correlation between the label vectors is not considered. Therefore, the entity relationship obtained based on the model cannot well reflect the correlation between the entities and will affect the representation of context, semantics and contextual relationships.

[0007] According to one aspect of an embodiment of the present disclosure, a joint extraction method for entities and entity relationships is provided, including: determining text information to be processed; based on the text information to be processed, determining first word vectors corresponding to each word in the text information to be processed; determining first label vectors corresponding to each first word vector, wherein the first label vector is used to indicate the entity type corresponding to each word; based on each first label vector, generating a corresponding second label vector, wherein the second label vector further includes semantic associations between the first label vectors on the basis of the first label vector; splicing the first word vector and the corresponding second label vector respectively, and generating multiple splicing vectors; and determining entity relationships between entities of the text information to be processed based on the multiple splicing vectors.

[0008] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0009] According to another aspect of an embodiment of the present disclosure, a joint extraction device for entities and entity relationships is also provided, including: a first text information to be processed determination module, used to determine the text information to be processed; a first word vector determination module, used to determine, based on the text information to be processed, first word vectors corresponding to each word in the text information to be processed; a first label vector determination module, used to determine first label vectors corresponding to each first word vector, wherein the first label vector is used to indicate the entity type corresponding to each word; a second label vector generation module, used to generate a corresponding second label vector based on each first label vector, wherein the second label vector further includes the semantic association between the first label vectors on the basis of the first label vector; a splicing vector generation module, used to splice the first word vector and the corresponding second label vector respectively, and generate multiple splicing vectors; and an entity relationship determination module, used to determine the entity relationship between the entities of the text information to be processed based on multiple splicing vectors.

[0010] According to another aspect of the embodiments of the present disclosure, there is also provided a joint extraction device for entities and entity relationships, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: determining text information to be processed; based on the text information to be processed, determining first word vectors corresponding to each word in the text information to be processed; determining first label vectors corresponding to each first word vector, wherein the first label vector is used to indicate the entity type corresponding to each word; based on each first label vector, generating a corresponding second label vector, wherein the second label vector further includes the semantic association between the first label vectors on the basis of the first label vector; respectively splicing the first word vector and the corresponding second label vector to generate multiple splicing vectors; and determining the entity relationship between the entities of the text information to be processed based on the multiple splicing vectors.

[0011] In the technical solution disclosed in the present application, the first label vector is first determined based on the first word vector and the first CRF layer. Then, the corresponding second label vector is generated based on each first label vector, the second BILSTM layer and the second CRF layer. Therefore, it can be seen from the above that the second label vector further includes the semantic correlation between the first label vectors on the basis of the first label vector.

[0012] Therefore, unlike the existing method of directly determining the entity relationship between entities based only on relatively independent label vectors, the present application determines the entity relationship between entities based on a second label vector with higher semantic relevance. Therefore, the entity relationship determined in the technical solution of the present disclosure can better reflect the semantic relevance between the entities and accurately represent the context, semantics and contextual relationship corresponding to the text information to be processed.

[0013] This solves the technical problem in the prior art that the label vectors (corresponding to the text information to be processed input into the model) are relatively independent and the semantic correlation between the label vectors is not taken into consideration. As a result, the entity relationship obtained based on the model cannot well reflect the correlation between the entities and will affect the representation of the context, semantics and contextual relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0015] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present disclosure;

[0016] Figure 2is a schematic diagram of a joint extraction system for entities and entity relationships according to Embodiment 1 of the present application;

[0017] Figure 3 It is a schematic diagram of the hierarchical structure of the joint extraction model for entities and entity relationships according to Example 1 of the present application;

[0018] Figure 4 is a flowchart of the joint extraction method for entities and entity relationships according to Example 1 of the present application;

[0019] Figure 5 is a schematic diagram of a sequence annotation of a first label vector output by the first CRF layer according to Embodiment 1 of the present application;

[0020] Figure 6 This is a schematic diagram of a specific process of character embedding and word embedding according to Example 1 of the present application;

[0021] Figure 7 is a schematic diagram of the labeling result of a corpus sample according to an embodiment of the present application;

[0022] Figure 8 is a schematic diagram of a joint extraction device for entities and entity relationships according to Embodiment 2 of the present application; and

[0023] Fig. 9 It is a schematic diagram of a joint extraction device for entities and entity relationships according to Example 3 of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0026] Example 1

[0027] According to this embodiment, a method embodiment for the joint extraction of entities and entity relationships is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a joint extraction method based on entities and entity relationships. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0029] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As involved in the embodiments of the present disclosure, the data processing circuitry acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0030] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the joint extraction method for entities and entity relationships in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the joint extraction method for entities and entity relationships of the above-mentioned application is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0032] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0033] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0034] Figure 2is a schematic diagram of a system for joint extraction of entities and entity relationships according to this embodiment. Figure 2 As shown, the system includes: a terminal device 100 and a processor 200. The user sends the text information to be processed to the processor 200 through the terminal device 100. The processor 200 sends the received text information to be processed to a pre-trained joint extraction model, and uses the joint extraction model to output entities and entity relationships corresponding to the text information to be processed.

[0035] Figure 3 Schematic diagram of the hierarchical structure of the joint extraction model for entities and entity relationships according to the embodiment of the present application. Figure 3 As shown, the joint extraction model includes a word embedding layer, a NER recognition unit, a label embedding unit and a relation extraction unit. Among them, the NER recognition unit includes the first BILSTM layer and the first CRF layer. The label embedding unit includes the second BILSTM layer and the second CRF layer. The relation extraction unit includes a fully connected layer and a sigmoid layer.

[0036] The word embedding layer is used to generate a second word vector corresponding to each word in the text information to be processed based on the received text information to be processed through character embedding and word embedding.

[0037] The first BILSTM layer is used to generate a first word vector based on the second word vector. The first word vector further includes the semantic association between each word in the text information to be processed based on the second word vector. The first CRF layer is used to generate a first label vector based on the first word vector. The first label vector is used to indicate the entity type corresponding to each word in the text information to be processed.

[0038] The second BILSTM layer and the second CRF layer are used to further generate a second label vector containing semantic associations between the first label vectors based on the first label vectors.

[0039] The relationship extraction unit includes a fully connected layer and a sigmoid layer. The relationship extraction unit is used to concatenate the first word vector and the corresponding second label vector respectively, generate multiple concatenated vectors, and send the multiple concatenated vectors to the fully connected layer. The fully connected layer is used to determine the entity relationship value between entities based on the concatenated vectors corresponding to each word in the text information to be processed. The sigmoid layer is used to map the entity relationship value to between 0 and 1 based on the sigmoid activation function, thereby determining the entity relationship probability value between entities. The relationship extraction unit is used to determine the entity relationship between entities based on the entity relationship probability value between the entities in the text information to be processed and a preset threshold.

[0040] It should be noted that the terminal device 100 and the processor 200 in the system can both be applicable to the hardware structure described above.

[0041] In the above operating environment, according to the first aspect of this embodiment, a joint extraction method for entities and entity relationships is provided. The method is composed of Figure 2 The processor 200 shown in FIG. Figure 4 A schematic diagram showing the process of the method is shown in FIG. Figure 4 As shown, the method includes:

[0042] S402: Determine the text information to be processed;

[0043] S404: Based on the text information to be processed, determine first word vectors corresponding to each word in the text information to be processed;

[0044] S406: Determine first label vectors corresponding to the first word vectors, wherein the first label vectors are used to indicate the entity type corresponding to the first word.

[0045] S408: generating corresponding second label vectors based on the first label vectors, wherein the second label vectors further include semantic associations between the first label vectors based on the first label vectors;

[0046] S410: respectively concatenating the first word vector and the corresponding second label vector to generate a plurality of concatenated vectors; and

[0047] S412: Determine entity relationships between entities of the text information to be processed based on the multiple concatenation vectors.

[0048] Specifically, first, the terminal device 100 sends the received text information to be processed to the processor 200. Then, the processor 200 determines the text information to be processed and inputs the text information to be processed into the joint extraction model (S402). The joint extraction model is used to identify entities of the text information to be processed and determine entity relationships between entities. And the joint extraction model is a model generated by pre-training.

[0049] In addition, it is worth noting that the text information to be processed may be, for example, corpus information composed of multiple words. The text information to be processed may be Chinese corpus information or English corpus information. This embodiment takes English corpus information as an example, and the specific situation is not limited thereto.

[0050] Then, refer to Figure 3As shown, the word embedding layer in the joint extraction model is based on the text information to be processed, and generates a second word vector corresponding to each word in the text information to be processed through character embedding and word embedding. Among them, the second word vector is a concatenated vector containing character information and word information. After that, the word embedding layer sends multiple second word vectors to the first BILSTM layer to generate a first word vector (S404). Among them, the first word vector further includes the semantic relevance between each word on the basis of the second word vector. The above content will be described in detail later, so it will not be repeated here.

[0051] Further, the first BILSTM layer sends multiple first word vectors to the first CRF layer, so that the first CRF layer determines the first label vectors corresponding to each first word vector (S406). The first label vector is used to indicate the entity type corresponding to each word. Specifically, first, for the first word vector, the first CRF layer uses the conditional random field algorithm to decode and annotate the first word vector to generate a set of label sequences. Then, the first CRF layer determines the entity type with the highest probability in the label sequence, and uses the determined entity type with the highest probability as the entity type corresponding to the word. Thus, a first label vector is generated based on the label sequence and the determined entity type.

[0052] Figure 5 is a schematic diagram of a sequence annotation of the first label vector output by the first CRF layer according to Embodiment 1 of the present application. Figure 5 As shown in the figure, for example, the text information to be processed is "Atmospheric watervapor collection", so the first CRF layer calculates that in the label sequence corresponding to the word "Atmospheric", the probability that the entity type is B-function is 1.5, the probability that the entity type is I-function is 0.9, and the probability that the entity type is O is 0.1. The first CRF layer determines that the entity type with the highest probability is B-function, so the first CRF layer uses B-function as the entity type corresponding to the word "Atmospheric".

[0053] For another example, in the label sequence corresponding to the word "water", the probability that the entity type is B-function is 0.2, the probability that the entity type is I-function is 0.4, and the probability that the entity type is O is 0.1. The first CRF layer determines that the entity type with the highest probability is I-function, so the first CRF layer uses I-function as the entity type corresponding to the word "water". Thus, the first CRF layer can determine the entity type corresponding to each word in the text information to be processed.

[0054] Further, the first CRF layer sends multiple first label vectors to the second BILSTM layer and the second CRF layer, so that the second BILSTM layer and the second CRF layer generate corresponding second label vectors according to each first label vector (S408). Among them, the second label vector further includes the semantic relationship between the first label vectors on the basis of the first label vector. Specifically, when the first CRF layer sends the first label vector to the second BILSTM layer and the second CRF layer, since the first label vector first passes through the second BILSTM layer and then passes through the second CRF layer, and the second BILSTM layer can determine the association relationship between each first label vector, so that the vector generated by the second BILSTM layer can fully reflect the semantic association between each first label vector in the multiple first label vectors. Therefore, the second label vector generated by the second CRF layer further includes the semantic association between each first label vector on the basis of the first label vector.

[0055] Furthermore, since the second label vector includes the semantic correlation between the first label vectors based on the first label vector, the entity relationship finally obtained based on the joint extraction model in this embodiment can better reflect the correlation between entities and accurately represent the context, semantics and contextual relationships.

[0056] After that, the relation extraction unit concatenates the first word vector and the corresponding second label vector respectively, and generates a plurality of concatenated vectors (S410). Specifically, since the first BILSTM layer for generating the first word vector is used to capture the feature representation of the text information to be processed, and the second CRF layer is used to model the dependency relationship between the second label vectors, the concatenated vector generated by the relation extraction unit after concatenating the second label vector with the first word vector can capture the semantic relationship and dependency relationship between the second label vectors.

[0057] In addition, it is worth noting that the first word vector and the second label vector are both 64-dimensional vectors. Therefore, the first word vector and the second label vector can maintain a consistent representation space, which makes the first word vector and the second label vector have similar semantic spaces, so that the joint extraction model can better understand the semantic relevance between the first label vector contained in the second label vector, and can more accurately perform tasks such as entity relationship extraction and entity recognition.

[0058] Finally, the relationship extraction unit determines the entity relationship between the entities of the text information to be processed based on the multiple splicing vectors (S412). Specifically, the relationship extraction unit inputs the multiple splicing vectors into the fully connected layer, so that the fully connected layer determines the entity relationship between each entity and other entities based on the splicing vectors corresponding to each entity of the text information to be processed, and obtains the entity relationship value indicating the relationship score between the entities. Afterwards, the sigmoid activation function of the sigmoid layer maps the multiple entity relationship values ​​determined by the fully connected layer to between 0 and 1, thereby obtaining the entity relationship probability value corresponding to the entity relationship value. Among them, the entity relationship probability value is used to indicate the probability of the entity relationship between each entity.

[0059] Furthermore, the relationship extraction unit determines the entity relationship between the entities based on the preset entity relationship probability threshold and the calculated entity relationship probability value between the entities. That is, when the entity relationship probability value is greater than the preset threshold, the entity relationship corresponding to the entity relationship probability value is determined as the entity relationship between the entities; when the entity relationship probability value is less than the preset threshold, it indicates that there is no entity relationship corresponding to the entity relationship probability value.

[0060] For example, the text information to be processed is "The composite coating is suitable for capturing atmospheric water without energy consumption", so the fully connected layer determines that there are 6 entity relationships that need to be determined based on the splicing vectors corresponding to the three entities (i.e., composite coating, capturing atmospheric water, and without energy consumption). The following Table 1 shows the 6 entity relationships that need to be determined between the above three entities. It is worth noting that, considering that the entity relationship between two entities is directional, there are 6 entity relationships corresponding to the three entities.

[0061] Table 1

[0062] Entity 1 Entity 2 Composite coating capturingatmosphericwater capturingatmosphericwater Composite coating Composite coating withoutenergyconsumption withoutenergyconsumption Composite coating Composite coating withoutenergyconsumption withoutenergyconsumption Composite coating

[0063] Referring to Table 1, taking the entity relationship between entity 1 "composite coating" and entity 2 "capturing atmospheric water" as an example, the fully connected layer determines that there are 4 possible entity relationships between entity "composite coating" and entity "capturing atmospheric water", and outputs the entity relationship values ​​corresponding to the 4 entity relationships (2.5, 7.5, 2.5, 2.5). Afterwards, the sigmoid activation function of the sigmoid layer maps the entity relationship values ​​corresponding to the 4 entity relationships (2.5, 7.5, 2.5, 2.5) to between 0 and 1, thereby obtaining the entity relationship probability values ​​(0.25, 0.75, 0.25, 0.25). Furthermore, the relationship extraction unit determines that the Function relationship corresponding to the entity relationship probability value of 0.75 is the entity relationship between the entity "composite coating" and the entity "capturing atmospheric water" based on the preset entity relationship probability threshold of 0.5 and the calculated entity relationship probability values ​​(0.25, 0.75, 0.25, 0.25).

[0064] As described in the background technology, in the process of identifying entities in the text information to be processed and extracting the entity relationships between entities, since the label vectors generated based on the CRF layer (corresponding to the text information to be processed input into the model) are relatively independent, the semantic correlation between the label vectors is not considered. Therefore, the entity relationship obtained based on the model cannot well reflect the correlation between entities and will affect the representation of context, semantics and contextual relationships.

[0065] In view of this, the present application provides a joint extraction method for entities and entity relationships. And because in the technical solution disclosed in the present application, the first label vector is first determined based on the first word vector and the first CRF layer. Then, the corresponding second label vector is generated based on each first label vector, and the second BILSTM layer and the second CRF layer. Therefore, it can be seen from the above that the second label vector further includes the semantic correlation between the first label vectors on the basis of the first label vector.

[0066] Therefore, unlike the existing method of directly determining the entity relationship between entities based only on relatively independent label vectors, the present application determines the entity relationship between entities based on a second label vector with higher semantic relevance. Therefore, the entity relationship determined in the technical solution of the present disclosure can better reflect the semantic relevance between the entities and accurately represent the context, semantics and contextual relationship corresponding to the text information to be processed.

[0067] This solves the technical problem in the prior art that the label vectors (corresponding to the text information to be processed input into the model) are relatively independent and the semantic correlation between the label vectors is not taken into consideration. As a result, the entity relationship obtained based on the model cannot well reflect the correlation between the entities and will affect the representation of the context, semantics and contextual relationships.

[0068] Optionally, based on the text information to be processed, the operation of determining the first word vectors corresponding to each word in the text information to be processed includes: based on the text information to be processed, generating second word vectors corresponding to each word through character embedding and word embedding; and generating corresponding first word vectors based on each second word vector, wherein the first word vector further includes the semantic association between each word on the basis of the second word vector.

[0069] Specifically, first, when the processor 200 sends the text information to be processed to the joint extraction model, the word embedding layer in the joint extraction model will map the words in the text information to be processed into a high-dimensional vector representation (i.e., the second word vector), thereby capturing the semantic relationship between the words for subsequent processing of the words.

[0070] For example, in the technical solution disclosed in the present application, the word embedding layer first converts the text information to be processed into a one-hot code sequence, and then maps the one-hot code sequence to the corresponding word vector (i.e., the second word vector). In this step, the technical solution disclosed in the present application uses a pre-trained Skip-Gram Word2Vec model for word embedding. In addition, character embedding is also combined to further capture morphological features such as prefixes and suffixes.

[0071] For example, some words use the suffix "ium," such as "calcium" and "lithium." Figure 6 This is a specific process of character embedding and word embedding according to the embodiment of this application. Figure 6 As shown in Figure 1, the characters of the word "air" are converted into character vectors. These character vectors are then input into a bidirectional LSTM (i.e., BiLSTM) and the forward state and the reverse state are concatenated. a and B air Connect them together and put F ai and B ir Connect them together and put F air and B r Finally, the character embedding vector W generated by character embedding chars , and the word embedding vector W generated by word embedding word2vecThe two words are connected together to form a complete word vector (i.e., the second word vector). Thus, the second word vector generated based on the text information to be processed contains both character information and word information.

[0072] Therefore, since the technical solution disclosed in the present application is based on the text information to be processed, and generates second word vectors corresponding to each word through character embedding and word embedding, the second word vector contains both character information and word information. Therefore, compared with the word vector that only contains word information, the second word vector in the present application can capture the morphological characteristics of the word, thereby ensuring readability.

[0073] After the word embedding layer generates the second word vectors corresponding to each word in the text information to be processed, the multiple second word vectors are input into the first BILSTM layer. Among them, the first BILSTM layer is suitable for processing various types of sequential data, whether the text information to be processed is a short sentence or a long chapter. The first BILSTM layer can more comprehensively understand the contextual relationship between each word corresponding to each second word vector and provide valuable feature representation for subsequent applications.

[0074] Compared with the traditional unidirectional model, the first BILSTM layer can take into account the context of each word at the same time to better understand the position and semantics of the corresponding word in the text information to be processed. By utilizing the bidirectional encoding ability of the first BILSTM layer, the relationship between each word in the text information to be processed can be more comprehensively grasped, thereby extracting richer and more accurate feature representations.

[0075] In addition, the hidden state size of the first BILSTM layer in this embodiment is 64, and 3 layers of LSTM are used. The probability of dropping nodes inside the first BILSTM layer, that is, the probability of dropping nodes applied in the input and output of the first BILSTM layer is set to 0.9, that is, 10% of the nodes are dropped.

[0076] Therefore, since the technical solution disclosed in the present application generates the corresponding first word vector according to each second word vector and based on the first BILSTM layer, and the first BILSTM layer is used to determine the context and contextual relationship of the words corresponding to each second word vector, the generated first word vector further includes the semantic relevance between each word on the basis of the second word vector, which is of great help in determining the context and semantics.

[0077] Optionally, the operation of determining the entity relationship between entities of the text information to be processed based on multiple splicing vectors includes: determining the entity relationship probability value between the entities of the text information to be processed based on multiple splicing vectors, wherein the entity relationship probability value is used to indicate the entity relationship probability between each entity; and determining the entity relationship between the entities of the text information to be processed based on the entity relationship probability value and a preset threshold. Further optionally, the operation of determining the entity relationship probability value between entities of the text information to be processed based on multiple splicing vectors includes: determining the entity relationship value between entities of the text information to be processed based on multiple splicing vectors; and determining the entity relationship probability value between each entity based on a sigmoid activation function and multiple entity relationship values. Further optionally, the operation of determining the entity relationship between entities of the text information to be processed based on the entity relationship probability value and a preset threshold includes: when the entity relationship probability value is greater than the preset threshold, determining the entity relationship corresponding to the entity relationship probability value as the entity relationship between entities, otherwise excluding the entity relationship corresponding to the entity relationship probability value.

[0078] Specifically, when the relation extraction unit concatenates the plurality of second label vectors with the corresponding first word vectors to generate a plurality of concatenated vectors corresponding to the respective words in the text information to be processed, the relation extraction unit sends the plurality of concatenated vectors to the fully connected layer, and uses the relation extraction model in the fully connected layer to identify the entity relationship between each entity and other entities, and determines the entity relationship value indicating the relationship between the entities. The entity relationship value is used to represent the score value of the entity relationship between each two entities.

[0079] For example, the text information to be processed is "atmospheric water vapor collection device, the invention improves the automation of the atmospheric water vapor collection.", which contains three entities "atmospheric water vapor collection device", "improves the automation" and "atmospheric water vapor collection". Considering that the relationship between two entities is directional, the following table can be obtained. Table 2 shows the entity relationship between the entities corresponding to the above three entities.

[0080] Table 2

[0081]

[0082] Taking “atmospheric water vapor collect device” in entity 1 and “improves the automation” in entity 2 as examples, the fully connected layer outputs 4 entity relationship values ​​(2.5, 2.5, 7.5, 2.5).

[0083] Furthermore, based on the sigmoid activation function and multiple entity relationship values, the entity relationship probability values ​​between the entities are determined. For example, the four entity relationship values ​​output by the fully connected layer are mapped to between 0 and 1 through the sigmoid activation function, thereby obtaining the entity relationship probability values ​​(0.25, 0.25, 0.75, 0.25).

[0084] When the relationship extraction unit determines that the entity relationship probability threshold is 0.5, it can determine that the relationship between entity 1 "atmospheric water vapor collect device" and entity 2 "improves the automation" is a Value relationship represented by a probability of 0.75.

[0085] Similarly, the entity relationship value between every two entities is calculated, and it is determined that the entity relationship values ​​between the remaining groups of entities are all less than 0.5, so there is no relationship.

[0086] Therefore, it can be seen from the above that the relationship extraction unit in this application includes a fully connected layer and a sigmoid layer, and the fully connected layer can determine the entity relationship value between entities corresponding to the text information to be processed based on the splicing vector, and then the sigmoid activation function of the sigmoid layer can map the entity relationship value to the interval of 0 to 1, and perform binary classification on the output result to determine whether there is a specific relationship between the entities. Thereby, the technical effect of accurately determining the entity relationship between entities can be achieved.

[0087] In addition, before using the joint extraction model to extract entities and entity relationships from the processed text information, the joint extraction model needs to be trained first.

[0088] First, collect sample corpora, annotate the sample corpora, and construct a training sample set.

[0089] Specifically, to address the problem of small sample corpus, this application adopts a method that combines manual verification and Python automated scripts to complete the construction of the training sample set.

[0090] After obtaining the sample corpus, the sample corpus is annotated. In the process of annotating the samples, six types of entities are set: subject, component, substance, function, utility and action. Four types of entity relationships: composition relationship, action relationship, functional relationship and utility relationship. In common word recognition tasks, common annotation methods include BIO, BIOSE and Markup annotation methods. In this embodiment, according to actual needs, the BIO annotation method is selected. Among them, B represents the beginning of the entity, I represents the middle part of the entity, and O represents a non-entity word.

[0091] In order to establish the connection relationship between named entities, the entity head is used as the connection of entity relationship. For example, for entity entity1, its entity relationship is represented as {(relation1, entity2), (relation2, entity3)}. The meaning of the representation is entity1—relation1—entity2 and entity1—relation2—entity3. The representation of entity relationship can be a set, which means that each entity can establish entity relationship with multiple entities.

[0092] For example, for the corpus "The utility model relates to devices for separating water from gases, to the condensation of atmospheric watervapor and canbe used in agriculture, in particular for supplying moisture to the root system of plants. The technical result consists in obtaining atmospheric moisture without energy consumption."

[0093] Entity annotation: "devices" is annotated as an entity type entity, "separating water from gases" is annotated as a function type entity, and "without energy consumption" is annotated as a value type entity. Entity identification tags such as I-function indicate that the tag is inside a named entity of type function.

[0094] Entity relationship annotation: Entity entity has a relationship with entity function and entity value. Entity function and entity value are used as the head entity of entity. The element set of the relationship type of devices is {(separating water from gases, Function), (without energy consumption, Value)}, which means devices (entity)-Function (relationship)-separating water from gases (entity), devices (entity)-Value (relationship)-without energy consumption (entity). Simplify the relationship between entities. The entity relationship between two entities is connected only by the last word in the entity to reduce repetitive information.

[0095] For example, there is a Function relationship between the entity "devices" and the entity "separating water from gases", which only connects "devices" and "gases". In addition, for the case where there is no relationship, the label "N" is introduced in this embodiment, and the label itself is predicted as the head entity.

[0096] Figure 7 Schematic diagram of the labeling result of the corpus sample according to the embodiment of the present application. Figure 7 As shown in the figure, the first column is the sequence of words, the second column is the specific words, the third column is the BIO type label of the entity, the fourth column is the entity relationship label, and the fifth column is the position of the last word in the head entity of the entity relationship. For example, the last word gases in the head node of the Function relationship separating water from gases is at position 10.

[0097] After annotating the sample corpus and constructing the training sample set, the annotated sample set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The joint extraction model is trained using the training set. The joint extraction model is evaluated using the validation set, and the parameters of the joint extraction model are adjusted based on the evaluation results. The test set is used to measure the performance of the trained joint extraction model in actual application scenarios.

[0098] For example, "5device B-entity[Function]

[10] " means that the word at the 5th position is "device", the entity type label is "entity", the entity relationship label is "[Function]", and the word that has the Function entity relationship with device is at position 10. The position here refers to the position of the last word in the entity.

[0099] Specifically, the word embedding layer in the joint extraction model is used to map words into high-dimensional vector representations to capture the semantic relationship between words. The function of the word embedding layer is to convert words into dense vector representations so that the neural network can process the words.

[0100] The character embedding in the word embedding layer uses all the characters extracted from the training sample set to form a character set. Assume that the character set size is N. The dimension of the character embedding is 25, that is, each character will be embedded in a 25-dimensional vector space. For example, for the word "is", according to the character id [i, s], the corresponding character embedding vector will be found from the character embedding matrix. Therefore, the character embedding result of the word "is" is generated by concatenating the character embedding vectors corresponding to the characters "i" and "s". Its dimension is 25×2=50 dimensions.

[0101] Using character embedding, we generate character embedding vectors based on the corpus in the training set; using word embedding, we generate word embedding vectors based on the corpus in the training set. The word embedding layer then concatenates the character embedding vectors and word vectors in the corresponding dimensions to form a new word vector that contains both character and word information.

[0102] In terms of model design, this application chooses to combine word vectors and character vectors and use a pre-trained Word2Vec model. There are several reasons for this:

[0103] 1. Word vectors can better express the semantic relationship and similarity between words, which helps the joint extraction model to better understand and learn the relationship between words.

[0104] 2. Character vectors can capture the character-level features within words, and are particularly suitable for words with special structures such as many abbreviations and technical terms.

[0105] 3. The combination of word vectors and character vectors can make more effective use of information expressed at different levels, and improve the recognition accuracy and generalization ability of the joint extraction model for words and relationships.

[0106] The first BILSTM layer and the second BILSTM layer are used to model the input information (wherein the input information of the first BILSTM layer is the word vector corresponding to the sample corpus, and the input information of the second BILSTM layer is the label vector corresponding to the sample corpus) to capture contextual information, including forward and backward information, which helps to better understand the semantics of the sample.

[0107] The first CRF layer and the second CRF layer are the user linear chain conditional random field algorithms that decode and annotate the hidden state sequences output by the first BILSTM layer and the second BILSTM layer, respectively. The annotated results are expressed in the form of BIO, and the possible labels at the current position and the labels that have been determined at the previous position are considered, and the scores of each label at the current position are calculated. The optimal path, that is, the most likely label sequence, is found through a dynamic programming algorithm (such as the Viterbi algorithm). For example, after the sample corpus "Atmospheric water vapor collection" is calculated by the CRF model, the four words in the sample corpus have the highest probability in the linear chain as the starting word and the middle word of the function, so the sample corpus is divided into function-type entities.

[0108] The fully connected layer in the relation extraction unit is used to generate entity relationship values ​​between entities based on the concatenation vector corresponding to the sample corpus. The sigmoid layer is used to output the probability of the entity relationship between two entities. The function of the relation extraction unit is to classify the output results into two categories to determine whether there is a specific relationship in the concatenation vector.

[0109] For example, the sample corpus in the training set is "The composite coating is suitable for capturing atmospheric water.". This embodiment first vectorizes the sample corpus by jointly extracting the character embedding and word embedding of the word embedding layer in the model. Then, the obtained word vector corresponding to the sample corpus is input into the first BILSTM layer for sequence labeling. Furthermore, through the use of the first CRF layer, the named entities in the sample corpus can be accurately identified. That is, the entity type entity "Composite (B-entity) coating (I-entity)" and the function type entity "capturing (B-function) atmospheric (I-function) water (I-function)". Table 3 shows the entity label types in the embodiments of the present application.

[0110] Table 3

[0111]

[0112]

[0113] Each of the 13 entity labels included in Table 3 above is embedded into a 64-dimensional vector, and the 13 labels become a 13*64-dimensional label matrix.

[0114] Furthermore, the generated label vector is sent to the second BILSTM layer and the second CRF layer corresponding to the second BILSTM layer. The second CRF layer performs the same operation as above, thereby mapping the identified entity labels to the label matrix and finding the corresponding 64-dimensional label vector. The 64-dimensional label vector is concatenated with the 64-dimensional vector output by the initial first BILSTM layer to generate multiple concatenated vectors corresponding to each word in the sample.

[0115] For example, the word "composite" is labeled as B-entity after passing through the second CRF layer. B-entity is searched in the label matrix to find its corresponding 64-dimensional vector. The specific label vector representation is obtained through the BILSTM model and CRF model learning. The BILSTM model is used to capture the feature representation of the sequence, while the CRF model is used to model the dependency relationship between labels. Through the learning of these two models, the final label vector can capture the semantic association and dependency relationship between labels.

[0116] In addition, the label vector in this embodiment is 64-dimensional, and the word vector output by the first BILSTM layer is also 64-dimensional, so the dimensions of the two match and can maintain a consistent representation space, so that the vector representation of the entity label has a similar semantic space as the feature representation output by the first BILSTM layer. This helps the joint extraction model to better understand the relationship between the entity label and the context, and can more accurately perform tasks such as relationship extraction and named entity recognition. In addition, consistent dimensions can simplify the model design and adjustment process because all vector representations operate in the same space.

[0117] Finally, the concatenated vectors corresponding to each word in the sample corpus are transmitted to the fully connected layer, and the entity relationship value between every two entities is calculated through the fully connected layer. Finally, the entity relationship value is mapped to between 0 and 1 through the sigmoid activation function to obtain the entity relationship probability value between every two entities. Thus, the entity relationship between every two entities in the sample is obtained.

[0118] Therefore, the sample corpus in the training set can be used to perform the same operations as the above steps to train the joint extraction model.

[0119] Therefore, according to the first aspect of this embodiment, the determined entity relationship can better reflect the semantic relevance between the various entities and accurately represent the context, semantics and contextual relationship corresponding to the text information to be processed.

[0120] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0121] Therefore, according to this embodiment, the determined entity relationship can better reflect the semantic relevance between various entities, and ultimately ensure the accuracy of the constructed knowledge graph.

[0122] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0124] Example 2

[0125] Figure 8 FIG. 8 shows a joint extraction device 800 for entities and entity relationships according to the first aspect of this embodiment, and the device 800 corresponds to the method according to embodiment 1. Figure 8As shown, the device 800 includes: a first text information to be processed determination module 810, which is used to determine the text information to be processed; a first word vector determination module 820, which is used to determine the first word vectors corresponding to each word in the text information to be processed based on the text information to be processed; a first label vector determination module 830, which is used to determine the first label vectors corresponding to each first word vector, wherein the first label vector is used to indicate the entity type corresponding to each word; a second label vector generation module 840, which is used to generate a corresponding second label vector based on each first label vector, wherein the second label vector further includes the semantic association between the first label vectors on the basis of the first label vector; a splicing vector generation module 850, which is used to splice the first word vector and the corresponding second label vector respectively, and generate a plurality of splicing vectors; and an entity relationship determination module 860, which is used to determine the entity relationship between the entities of the text information to be processed based on the plurality of splicing vectors.

[0126] Optionally, the first word vector determination module 820 includes: a second word vector generation module, which is used to generate second word vectors corresponding to each word through character embedding and word embedding based on the text information to be processed; and a first word vector generation submodule, which is used to generate a corresponding first word vector based on each second word vector, wherein the first word vector further includes the semantic association between each word on the basis of the second word vector.

[0127] Optionally, the entity relationship determination module 860 includes: an entity relationship probability value determination module, which is used to determine the entity relationship probability value between entities of the text information to be processed based on multiple splicing vectors, wherein the entity relationship probability value is used to indicate the entity relationship probability between each entity; and an entity relationship determination submodule, which is used to determine the entity relationship between entities of the text information to be processed based on the entity relationship probability value and a preset threshold.

[0128] Optionally, the entity relationship probability value determination module includes: an entity relationship value determination module, which is used to determine the entity relationship values ​​between entities of the text information to be processed based on multiple splicing vectors; and an entity relationship probability value determination submodule, which is used to determine the entity relationship probability values ​​between each entity based on the sigmoid activation function and multiple entity relationship values.

[0129] Optionally, the entity relationship determination submodule includes: a judgment unit, which is used to determine that the entity relationship corresponding to the entity relationship probability value is an entity relationship between entities when the entity relationship probability value is greater than a preset threshold, otherwise exclude the entity relationship corresponding to the entity relationship probability value.

[0130] Therefore, according to this embodiment, the determined entity relationship can better reflect the semantic relevance between the various entities and accurately represent the context, semantics and contextual relationship corresponding to the text information to be processed.

[0131] Example 3

[0132] Fig. 9 FIG. 1 shows a joint extraction device 900 for entities and entity relationships according to the first aspect of this embodiment, and the device 900 corresponds to the method according to embodiment 1. Fig. 9 As shown, the device 900 includes: a processor 910; and a memory 920, which is connected to the processor 910 and is used to provide the processor 910 with instructions for processing the following processing steps: determining the text information to be processed; based on the text information to be processed, determining the first word vectors corresponding to each word in the text information to be processed; determining the first label vectors corresponding to each first word vector, wherein the first label vector is used to indicate the entity type corresponding to each word; based on each first label vector, generating a corresponding second label vector, wherein the second label vector further includes the semantic association between the first label vectors on the basis of the first label vector; splicing the first word vector and the corresponding second label vector respectively, and generating multiple splicing vectors; and determining the entity relationship between the entities of the text information to be processed based on the multiple splicing vectors.

[0133] Optionally, based on the text information to be processed, the operation of determining the first word vectors corresponding to each word in the text information to be processed includes: based on the text information to be processed, generating second word vectors corresponding to each word through character embedding and word embedding; and generating corresponding first word vectors based on each second word vector, wherein the first word vector further includes the semantic association between each word on the basis of the second word vector.

[0134] Optionally, the operation of determining entity relationships between entities of the text information to be processed based on multiple splicing vectors includes: determining entity relationship probability values ​​between entities of the text information to be processed based on multiple splicing vectors, wherein the entity relationship probability values ​​are used to indicate the entity relationship probabilities between respective entities; and determining entity relationships between entities of the text information to be processed based on the entity relationship probability values ​​and a preset threshold.

[0135] Optionally, the operation of determining the entity relationship probability values ​​between entities of the text information to be processed based on multiple splicing vectors includes: determining the entity relationship values ​​between entities of the text information to be processed based on multiple splicing vectors; and determining the entity relationship probability values ​​between each entity based on the sigmoid activation function and multiple entity relationship values.

[0136] Optionally, based on the entity relationship probability value and a preset threshold, the operation of determining the entity relationship between entities of the text information to be processed includes: when the entity relationship probability value is greater than the preset threshold, determining that the entity relationship corresponding to the entity relationship probability value is the entity relationship between entities, otherwise excluding the entity relationship corresponding to the entity relationship probability value.

[0137] Therefore, according to this embodiment, the determined entity relationship can better reflect the semantic relevance between the various entities and accurately represent the context, semantics and contextual relationship corresponding to the text information to be processed.

[0138] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0139] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0144] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A joint extraction method for entities and entity relationships, characterized in that: include: Determine text information to be processed, and input the text information to be processed into a joint extraction model, wherein the joint extraction model includes a word embedding layer, a NER recognition unit, and a label embedding unit, wherein the NER recognition unit includes a first BILSTM layer and a first CRF layer, and the label embedding unit includes a second BILSTM layer and a second CRF layer; The first BILSTM layer determines, based on the text information to be processed, first word vectors corresponding to each word in the text information to be processed; The first CFR layer determines first label vectors corresponding to the first word vectors, respectively, wherein the first label vectors are used to indicate the entity type corresponding to the first words; The second BILSTM layer and the second CRF layer generate corresponding second label vectors based on each first label vector, wherein the second label vector further includes semantic associations between the first label vectors on the basis of the first label vectors; respectively concatenating the first word vector and the corresponding second label vector to generate a plurality of concatenated vectors; and Based on the multiple concatenation vectors, determining the entity relationship between the entities of the text information to be processed, wherein The operation of determining, based on the text information to be processed, first word vectors corresponding to respective words in the text information to be processed includes: Based on the text information to be processed, generating second word vectors corresponding to each of the words respectively through character embedding and word embedding; as well as According to each second word vector, the corresponding first word vector is generated, wherein the first word vector further includes the semantic association between the respective words on the basis of the second word vector.

2. The method according to claim 1, characterized in that The operation of determining the entity relationship between entities of the text information to be processed based on the multiple splicing vectors includes: Determining entity relationship probability values ​​between entities of the text information to be processed based on the multiple concatenation vectors, wherein the entity relationship probability values ​​are used to indicate entity relationship probabilities between respective entities; and Based on the entity relationship probability value and a preset threshold, the entity relationship between the entities of the text information to be processed is determined.

3. The method according to claim 2, characterized in that The operation of determining the entity relationship probability values ​​between the entities of the text information to be processed based on the multiple splicing vectors includes: Determining entity relationship values ​​between entities of the text information to be processed based on the multiple splicing vectors; and Based on the sigmoid activation function and the multiple entity relationship values, the entity relationship probability values ​​between the entities are determined.

4. The method according to claim 3, characterized in that The operation of determining the entity relationship between entities in the text information to be processed based on the entity relationship probability value and a preset threshold value includes: When the entity relationship probability value is greater than the preset threshold, the entity relationship corresponding to the entity relationship probability value is determined to be the entity relationship between the entities; otherwise, the entity relationship corresponding to the entity relationship probability value is excluded.

5. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 4.

6. A joint extraction device for entities and entity relationships, characterized in that: include: A first module for determining text information to be processed, used for determining text information to be processed, and inputting the text information to be processed into a joint extraction model, wherein the joint extraction model includes a word embedding layer, a NER recognition unit, and a label embedding unit, wherein the NER recognition unit includes a first BILSTM layer and a first CRF layer, and the label embedding unit includes a second BILSTM layer and a second CRF layer; A first word vector determination module, configured for the first BILSTM layer to determine, based on the text information to be processed, first word vectors corresponding to respective words in the text information to be processed; A first label vector determination module, configured to determine, at the first CFR layer, first label vectors corresponding to respective first word vectors, wherein the first label vectors are used to indicate entity types corresponding to the respective words; A second label vector generating module, configured for the second BILSTM layer and the second CRF layer to generate corresponding second label vectors based on each first label vector, wherein the second label vector further includes semantic associations between the first label vectors on the basis of the first label vectors; a concatenated vector generating module, configured to concatenate the first word vector and the corresponding second label vector respectively, and generate a plurality of concatenated vectors; and An entity relationship determination module is used to determine the entity relationship between the entities of the text information to be processed based on the multiple splicing vectors, wherein The first word vector determination module includes: A second word vector generating module, configured to generate second word vectors corresponding to each of the words respectively through character embedding and word embedding based on the text information to be processed; as well as The first word vector generating submodule is used to generate the corresponding first word vector according to each second word vector, wherein the first word vector further includes the semantic association between the each word on the basis of the second word vector.

7. The device according to claim 6, characterized in that The entity relationship determination module includes: an entity relationship probability value determination module, configured to determine entity relationship probability values ​​between entities of the text information to be processed based on the multiple concatenation vectors, wherein the entity relationship probability values ​​are used to indicate entity relationship probabilities between respective entities; and The entity relationship determination submodule is used to determine the entity relationship between the entities of the text information to be processed based on the entity relationship probability value and a preset threshold.

8. A joint extraction device for entities and entity relationships, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Determine text information to be processed, and input the text information to be processed into a joint extraction model, wherein the joint extraction model includes a word embedding layer, a NER recognition unit, and a label embedding unit, wherein the NER recognition unit includes a first BILSTM layer and a first CRF layer, and the label embedding unit includes a second BILSTM layer and a second CRF layer; The first BILSTM layer determines, based on the text information to be processed, first word vectors corresponding to each word in the text information to be processed; The first CFR layer determines first label vectors corresponding to the first word vectors, respectively, wherein the first label vectors are used to indicate the entity type corresponding to the first words; The second BILSTM layer and the second CRF layer generate corresponding second label vectors based on each first label vector, wherein the second label vector further includes semantic associations between the first label vectors on the basis of the first label vectors; respectively concatenating the first word vector and the corresponding second label vector to generate a plurality of concatenated vectors; and Based on the multiple concatenation vectors, determining the entity relationship between the entities of the text information to be processed, wherein The operation of determining, based on the text information to be processed, first word vectors corresponding to respective words in the text information to be processed includes: Based on the text information to be processed, generating second word vectors corresponding to each of the words respectively through character embedding and word embedding; as well as According to each second word vector, the corresponding first word vector is generated, wherein the first word vector further includes the semantic association between the respective words on the basis of the second word vector.

Images