Information extraction method and device, computer storage medium and electronic equipment

Abstract

The application relates to the technical field of artificial intelligence, and provides an information extraction method and device, a computer storage medium and an electronic device. The method is applied to a distributed information extraction system; the method comprises the following steps: obtaining to-be-recognized text; inputting the to-be-recognized text into an information extraction model, performing entity recognition and relationship extraction on the to-be-recognized text by using the information extraction model, so as to obtain entity relationship information in the to-be-recognized text; wherein the information extraction model is generated by training a to-be-trained information extraction model according to an optimized loss function and global text labels, and the optimized loss function is optimized for sample imbalance and strengthens a relationship extraction task. The application can improve the accuracy of information extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an information extraction method, an information extraction device, a computer storage medium and an electronic device. BACKGROUND

[0002] Information extraction (IE) is to structure the information contained in the text and change it into a table-like organization form. The input information of the extraction system is the original text, and the output is the information point in a fixed format. Information points are extracted from various documents and integrated in a unified form, which is the main task of information extraction.

[0003] At present, there are two main categories of information extraction methods, one is two-stage extraction, and the other is joint extraction. The two-stage extraction is to build two independent models, extract entities in the text first, and then judge the relationship between entities. However, the two-stage extraction has the problem that when extracting entities, errors will be passed to the subsequent entity relationship judgment, affecting the judgment result. Joint extraction is widely used and has good effect, but the current joint extraction has the problems of sample imbalance in relationship extraction, relationship extraction loss function much smaller than entity recognition loss function, etc. And in information extraction, relationship extraction only considers the local relationship between entities and relationship pointing words, lacks overall recognition ability, and is prone to misjudgment. SUMMARY

[0004] Embodiments of the present application provide an information extraction method, an information extraction device, a computer storage medium and an electronic device, which can at least to some extent avoid the problems of sample imbalance in relationship extraction, relationship extraction loss function much smaller than entity recognition loss function, and lack of overall recognition, and improve the accuracy of information extraction results.

[0005] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0006] In a first aspect, an information extraction method is provided, the method comprising: obtaining a to-be-recognized text; inputting the to-be-recognized text into an information extraction model, performing entity recognition and relationship extraction on the to-be-recognized text by the information extraction model to obtain entity relationship information in the to-be-recognized text; wherein the information extraction model is generated by training a to-be-trained information extraction model according to an optimization loss function and a text global label, and the optimization loss function optimizes sample imbalance and strengthens the relationship extraction task.

[0007] In a second aspect, an information extraction apparatus is provided, which comprises: an acquisition module configured to acquire a text to be recognized; and a processing module configured to input the text to be recognized into an information extraction model, and perform entity recognition and relation extraction on the text to be recognized by using the information extraction model to obtain entity relation information in the text to be recognized; wherein the information extraction model is generated by training a to-be-trained information extraction model according to an optimized loss function and a text global label, and the optimized loss function is optimized for sample imbalance and the relation extraction task is strengthened.

[0008] In a third aspect, a computer-readable storage medium is provided, which comprises instructions that, when executed on a computer, cause the computer to perform the information extraction method according to any one of the above embodiments.

[0009] In a fourth aspect, an electronic device is provided, which comprises a processor, and a memory configured to store executable instructions of the processor; wherein the processor is configured to invoke the executable instructions to implement the information extraction method according to any one of the above embodiments.

[0010] In a fifth aspect, a computer program product is provided, which comprises instructions that, when executed on a computer, cause the computer to perform the information extraction method according to any one of the above embodiments.

[0011] The information extraction method provided by the embodiments of the present application can improve the fitting capability of the information extraction model in extracting the relation between entities, and improve the accuracy of entity recognition and relation recognition between entities, by using the information extraction model trained based on the optimized loss function and the text global label to perform information extraction on the text to be recognized.

[0012] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present application. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied is shown;

[0014] Figure 2 A flowchart of an information extraction method according to an embodiment of the present application is schematically shown;

[0015] Figure 3 A structural schematic diagram of an information extraction model according to one embodiment of the present application is illustratively shown;

[0016] Figure 4 A structural schematic diagram of a Bert sub-model 301 according to one embodiment of the present application is illustratively shown;

[0017] Figure 5 A calculation flow schematic diagram of a BILSTM-CRF according to one embodiment of the present application is illustratively shown;

[0018] Figure 6 Text encoding information output by a BILSTM sub-model to be trained according to one embodiment of the present application is illustratively shown;

[0019] Figure 7 A transition scoring matrix according to one embodiment of the present application is illustratively shown;

[0020] Figure 8 A flow schematic diagram of calculating a first loss function according to one embodiment of the present application is illustratively shown;

[0021] Figure 9 A model training process with added text global labels according to one embodiment of the present application is illustratively shown;

[0022] Figure 10 A block diagram of an information extraction apparatus according to one embodiment of the present application is illustratively shown;

[0023] Figure 11 A structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present application is shown. DETAILED DESCRIPTION

[0024] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art.

[0025] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, devices, implementations, and operations have not been shown or described in detail to avoid obscuring aspects of the application.

[0026] The block diagrams shown in the drawings are merely functional entities and do not necessarily have to correspond to physically independent entities. That is, the functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the drawings are merely exemplary illustrations and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to actual conditions.

[0028] In related technologies in the art, two-stage extraction and joint extraction are usually used for information extraction, where two-stage extraction is to build two independent models, first extract entities in the text, and then judge the relationship between the entities, and joint extraction simultaneously extracts text entities and the relationship between entities in one task.

[0029] However, if there is an error in the process of extracting entities in two-stage extraction, the error will be passed to the extraction of the relationship between entities, resulting in a large error and poor accuracy in the final output information. Joint extraction avoids the problems of two-stage extraction, but joint extraction also has corresponding disadvantages. Taking the commonly used joint extraction algorithm Multi-head Selection algorithm as an example, it has the following problems in text information extraction: 1) the problem of sample imbalance in relationship extraction, which makes it difficult to fit the relationship between entities; 2) the problem that the relationship extraction loss function is much smaller than the entity recognition loss function, which weakens the relationship extraction task; 3) when performing an information extraction task, relationship extraction only considers the local relationship between entities and relationship-oriented words, lacks the ability to identify the whole, and is prone to misjudgment.

[0030] To solve the problems in the related art, the embodiments of the present application provide a new information extraction method.

[0031] Before the technical solutions in the embodiments of the present application are described in detail, the technical terms that can be involved in the embodiments of the present application are first explained and described.

[0032] 1. Multi-head Selection algorithm: a multi-head selection algorithm, a model based on the multi-head selection algorithm belongs to a kind of joint extraction model, when carrying out joint extraction to text, first extract entity, then utilize entity boundary information to carry out relationship extraction.

[0033] 2、Bert model: full name is Bidirectional Encoder Representation from Transformers, the Bert model uses the Transformer Encoder block for connection, which is a typical bidirectional encoding model with strong language representation ability and feature extraction ability. And the Bert model is a pre-training model that can be used directly.

[0034] 3、BILSTM model: bidirectional long short-term memory model, full name is Bi-directional Long Short-Term Memory, which is composed of forward long short-term memory network (LSTM) and backward long short-term memory network (LSTM), which can better capture the bidirectional semantic dependency in the text.

[0035] 4、CRF model: conditional random field, full name is Conditional Random Field, which is a discriminative probabilistic model commonly used for labeling or analyzing sequence data.

[0036] 5、Entity recognition: also known as named entity recognition (Named Entities Recognition, NER), the main task is to identify the text range of named entities and classify them into predefined categories.

[0037] 6、Relationship extraction: is to extract the relationship between entities, specifically, it is to extract (subject, relationship, object) such triplets from a piece of text.

[0038] After introducing some technical terms used in the present application, the following will be combined Figure 1 with the technical scheme of the present application to explain the exemplary system architecture of the present application:

[0039] Figure 1 The schematic diagram of the exemplary system architecture to which the technical scheme of the present application can be applied is shown.

[0040] As Figure 1 shown, the system architecture 100 can include a terminal device 101, a network 102 and a server 103. The network 102 is used to provide a communication link medium between the terminal device 101 and the server 103. The network 102 can include various connection types, such as wired communication link, wireless communication link, etc.

[0041] It should be understood that Figure 1The number of terminal devices, networks, and servers in the figure is only illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers. For example, the server 103 can be a standalone physical server, or a server cluster or distributed system formed by multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device 101 can be a terminal device such as a notebook computer, a tablet computer, a desktop computer, a smart vehicle terminal, a smart phone, and the like, but is not limited thereto.

[0042] The information extraction method in the present application can be applied to various scenarios, such as relationship annotation, sentiment recognition, and the like, that is, any scenario involving relationship extraction can be implemented using the information extraction method in the embodiments of the present application.

[0043] In an embodiment of the present application, when the information extraction method in the present application is used for relationship annotation, the processing party can obtain the to-be-recognized text in the terminal device 101 through various channels, such as forums, micro blogs, video websites, news websites, etc. After the processing party determines the to-be-recognized text, a triggering operation can be performed to send the to-be-recognized text to the server 103 through the network 102. After receiving the to-be-recognized text, the server 103 can call the trained information extraction model and input the to-be-recognized text into the information extraction model to extract entities and relationships between entities in the to-be-recognized text through the information extraction model. The relationship between the entities extracted by the information extraction model depends on the training mode in the model training stage. When the information extraction model is trained to extract the first type of relationship between entities in the text sample, the trained information extraction model can be used to extract the first type of relationship between entities in the to-be-recognized text. When the information extraction model is trained to extract the second type of relationship between entities, the trained information extraction model can be used to extract the second type of relationship between entities in the to-be-recognized text, and so on. The first type of relationship and the second type of relationship are different, and can be opposite relationship categories. After completing the recognition and extraction of entities and entity relationships in the to-be-recognized text, the extraction result can be fed back to the terminal device 101 through the network 102 for reference by the processing party.

[0044] In an embodiment of the present application, the information extraction method in the present application can also be used for emotion recognition. The processor obtains the text corresponding to the dialogue or monologue through the terminal device 101, and sends the text as the to-be-recognized text to the server 103 through the network 102. After receiving the to-be-recognized text, the server 103 calls the trained information extraction model, and inputs the to-be-recognized text into the information extraction model. The information extraction model extracts the entities in the to-be-recognized text and the relationship between the entities according to the learned rules. Similarly, when the model training task is to enable the information extraction model to extract various emotions in the text, after the to-be-recognized text is input into the information extraction model, the to-be-recognized text is subjected to entity recognition and entity relationship recognition through the information extraction model, and the object emotion in the to-be-recognized text can be obtained. It should be noted that in the training stage, the model can perform multi-classification according to the types of emotions to train the information extraction model to recognize various types of emotions, or the emotions can be divided into two categories, i.e., the first type of emotion and the second type of emotion opposite to the first type of emotion, for example, happiness is the first type of emotion, and fear, anger and sadness are the second type of emotion. Then, the information extraction model only needs to perform two-classification according to the entity recognition result, and the emotion type in the to-be-recognized text can be obtained.

[0045] It should be noted that the information extraction method provided in the embodiments of the present application is generally executed by a server, and correspondingly, the information extraction device is generally arranged in the server. However, in other embodiments of the present application, the information extraction method provided in the embodiments of the present application can also be executed by a terminal device. Specifically, the information extraction model is built in the terminal device 101. When the processor determines the to-be-recognized text through the terminal device 101, the terminal device 101 can call the information extraction model to extract the entities and the entity relationship in the to-be-recognized text, and display the extracted information on the display screen of the terminal device 101 for reference by the processor.

[0046] In some embodiments of the present application, the information extraction model arranged in the terminal device 110 or the server 130 is a machine learning model for information extraction based on artificial intelligence technology.

[0047] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0048] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other major directions.

[0049] Natural language processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0050] Machine learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a discipline that studies how computers simulate or implement human learning methods to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and example-based learning.

[0051] The foregoing information extraction model is a model involving natural language processing and machine learning in artificial intelligence.

[0052] Next, the information extraction method in the embodiments of the present application will be described in detail.

[0053] Figure 2 The flowchart of the information extraction method according to an embodiment of the present application is schematically shown, which can be executed by a server, which can be specifically the server 103 shown in Figure 1 Referring to Figure 2 , the information extraction method at least includes S210 to S220, specifically:

[0054] In S210, the text to be recognized is obtained;

[0055] In S220, the text to be identified is input into the information extraction model, entity recognition and relation extraction are performed on the text to be identified by the information extraction model, and entity relation information in the text to be identified is obtained.

[0056] The information extraction model is generated by training a to-be-trained information extraction model according to an optimized loss function and a text global label, and the optimized loss function optimizes sample imbalance and strengthens the relation extraction task.

[0057] The information extraction method provided by the embodiments of the present application extracts information from the text to be identified by using the information extraction model trained based on the optimized loss function and the text global label. On the one hand, the imbalance between easy and difficult classification samples and the imbalance between the relation extraction loss function and the entity loss function are considered in the optimized loss function. Therefore, the to-be-trained information extraction model is trained by using the optimized loss function, which can improve the fitting capability of the information extraction model in extracting the relation between entities, and improve the accuracy of entity recognition and relation recognition between entities. On the other hand, the text global label is introduced when training the to-be-trained information extraction model, which improves the capability of the model in extracting the relation between entities from the whole text, and further improves the accuracy of the extracted relation between entities.

[0058] The specific implementation of each method step of the information extraction method in the present application is described in detail below.

[0059] In S210, the text to be identified is obtained.

[0060] In an embodiment of the present application, the type of the to-be-identified text is different according to different business scenarios. For example, when the business scenario is to identify information of a predetermined relationship category about an enterprise, the to-be-identified text can be news reports, forum comments, financial reports, etc. related to the enterprise; when the business scenario is to identify the emotion of an object, the to-be-identified text is a diary, a video, a conversation, etc. related to the object; when the business scenario is to identify information of a predetermined relationship category about an object with social influence, the to-be-identified text is news reports, comments, posts, audio and video recordings, etc. related to the object. Correspondingly, the way to obtain the to-be-identified text also includes multiple ways. Taking the example of identifying information of a predetermined relationship category about an enterprise and an object with social influence, the enterprise and the object can be taken as keywords to search for relevant news reports, comments, posts, etc. from the Internet, or to obtain information related to the enterprise and the object from some video websites or news websites, or to obtain information related to the enterprise and the object from paper files such as newspapers and magazines, etc. After obtaining various types of information, the non-text information can be converted into text information to obtain the required to-be-identified text. Taking the example of identifying the emotion of an object, the to-be-identified text can be obtained from the diary recorded by the object, the conversation with friends, the status shared by the object on the network, etc.

[0061] In an embodiment of the present application, after obtaining the to-be-identified text, it can be input into the trained information extraction model to perform entity recognition and relationship extraction on the to-be-identified text by the information extraction model to output the entity relationship information in the to-be-identified text.

[0062] In order to improve the accuracy of the information extraction result, the to-be-identified text can be preprocessed before being input into the information extraction model. Specifically, the preprocessing of the to-be-identified text mainly includes corpus-level granularity text processing and word-level granularity text processing. The corpus-level granularity text processing includes text standardization and text cleaning of the to-be-identified text. The text standardization includes converting full-width characters into half-width characters, converting uppercase letters into lowercase letters, and converting traditional Chinese into simplified Chinese, etc. The text cleaning is to remove non-text content such as html tags and emoticons, etc. The word-level granularity text processing includes word cleaning, word standardization, and spelling correction. The word cleaning includes deleting stop words and low-frequency words. The word standardization includes unifying the writing of certain words. The spelling correction includes single-word spelling correction and grammar usage correction.

[0063] In step S220, the text to be identified is input into an information extraction model, and entity recognition and relation extraction are performed on the text to be identified by the information extraction model to obtain entity relation information in the text to be identified; wherein the information extraction model is generated by training a to-be-trained information extraction model according to an optimized loss function and a text global label, and the sample imbalance is optimized and the relation extraction task is strengthened in the optimized loss function.

[0064] In an embodiment of the present application, after the text to be identified is preprocessed, the text to be identified can be input into an information extraction model, and entity recognition and relation extraction are performed on the text to be identified by the information extraction model to output entity relation information in the text to be identified. The information extraction model in the present application is an information extraction model based on the Multi-head Selection algorithm, but it is different from the information extraction model based on the Multi-head Selection algorithm commonly used. The information extraction model in the present application fully considers the problems of imbalance between easy classification samples and difficult classification samples and the relation extraction loss function being much smaller than the entity extraction loss function, and also introduces a text global label in the model training process, improves the overall recognition ability of the information extraction model to the text to be identified, and further improves the accuracy of information extraction.

[0065] Figure 3 The structure diagram of the information extraction model is shown as Figure 3As shown, the information extraction model 300 includes a Bert sub-model 301, a BILSTM sub-model 302, a CRF sub-model 303 and a relation extraction sub-model 304 connected in sequence, wherein the relation extraction sub-model 304 includes a label embedding layer 304-1 and a relation classification layer 304-2. After the to-be-recognized text is input into the information extraction model 300, the to-be-recognized text is first input into the Bert sub-model 301, and the characters in the to-be-recognized text are pre-encoded by the Bert sub-model 301 to obtain pre-encoding information corresponding to each character; then the pre-encoding information is input into the BILSTM sub-model 302, and the pre-encoding information is bidirectionally encoded by the BILSTM sub-model 302 to obtain text encoding information, and the elements contained in the text encoding information are probability values of each character corresponding to a certain entity label; the text encoding information enters the CRF sub-model 303, and the CRF sub-model 303 can recognize the entity in the to-be-recognized text based on the text encoding information according to the learned constraint condition, and then obtain the entity label information corresponding to the to-be-recognized text, that is, the entity information in the to-be-recognized text is labeled by using the entity labeling method; finally, the entity information is input into the relation extraction sub-model 304, and the relation between the entity information is extracted by the label embedding layer 304-1 and the relation classification layer 304-2 to output the entity relation information in the to-be-recognized text. Wherein, when the entity in the to-be-recognized text is labeled by using the entity labeling method, the BIO, BIOES and the like labeling method can be specifically used, and the present application does not make specific limitation thereto.

[0066] In an embodiment of the present application, the Bert sub-model 301 is a bidirectional encoder constructed based on a Transformer encoding block, Figure 4 The structure diagram of the Bert sub-model 301 is shown as follows, Figure 4As shown, the Bert sub-model 301 includes an input layer 401, a Transformer unit 402 composed of multiple Transformer layers, and an output layer 403. The characters in the text to be recognized are converted into one-dimensional vectors E1, E2, …, En in the input layer 401, specifically, the characters are converted into vectors based on a word vector table, and then the vectors corresponding to each character are input into the Transformer unit 402. Since the Transformer unit is constructed based on the multi-head self-attention mechanism to construct the features of each word, it can fully consider the importance of other words to the current word, and thus when processing the vectors corresponding to each character in the text to be recognized through the Transformer unit 402, it can output the pre-encoding information T1, T2, …, Tn corresponding to each character in the text to be recognized based on the context relationship in the text to be recognized. That is, the pre-encoding information is a vector representation corresponding to each character that integrates the context semantics. Finally, the pre-encoding information is output through the output layer 403.

[0067] In an embodiment of the present application, the BILSTM sub-model 302 is a set of bidirectional propagation word vector encoding model constructed on the basis of LSTM model. The LSTM model only considers the information from front to back when encoding the text and does not use the information from back to front, so it is easy to produce one-sided understanding of semantics in the encoding process. The BILSTM sub-model 302 is composed of forward LSTM and backward LSTM, which can splice the encoding results of characters in the text to be recognized from two directions, realizing the understanding of word context.

[0068] Before understanding how BILSTM performs bidirectional encoding, first understand the process of word encoding of LSTM model. The LSTM model is composed of input word Xt at time t, cell state Ct, hidden layer Ht, forgetting gate Ft, memory gate It, and output gate Ot. The forgetting gate Ft is used to calculate the forgetting (retention) degree of information, the memory gate It is used to calculate which information is saved to the cell state Ct, and the output gate Ot is used to calculate the degree of information output at the current time. The calculation process of the LSTM model is as follows: by forgetting the information in the cell state Ct and memorizing new information, useful information is transmitted to the subsequent time, and useless information is discarded, and the hidden layer state Ht is output at each time step. The forgetting, memory, and output are controlled by the forgetting gate Ft, the memory gate It, and the output gate Ot calculated and processed by the hidden layer state Ht-1 at this time and the current Xt.

[0069] The BILSTM is composed of a forward LSTM and a reverse LSTM. Regardless of the forward LSTM or the reverse LSTM, the specific calculation process is the same as that of the LSTM model described above, except that the order of character input is different. For the forward LSTM, the characters in the text to be recognized are sequentially input into the LSTM model. For the reverse LSTM, the characters in the text to be recognized are input into the LSTM model in reverse order. Finally, the encoding results of the same character by the forward LSTM and the reverse LSTM are spliced, and the encoding result corresponding to the character is obtained.

[0070] In an embodiment of the present application, Figure 5 A calculation flow diagram of the BILSTM-CRF is shown. As shown in Figure 5 In step S501, the pre-encoding information corresponding to each character in the text to be recognized output by the Bert sub-model 301 is input into the BILSTM sub-model 302. In step S502, the vectors corresponding to each character are encoded by the forward LSTM and the reverse LSTM according to the context information, and the final encoding result is obtained by splicing the forward encoding result and the reverse encoding result. The final encoding result is the probability of each entity label corresponding to each character in the text to be recognized. That is, the text encoding information output by the BILSTM sub-model 302 is composed of the probability of each character being marked as each entity label. The size of the text encoding information is MxN, where M is the total number of characters in the text to be recognized, and N is the number of entity labels. Both M and N are positive integers. In step S503, the text encoding information output by the BiLSTM sub-model 302 is input into the CRF sub-model 303. The CRF sub-model 303 performs entity recognition and labeling on the recognized entities based on the text encoding information according to the learned order-dependent information between labels, to obtain the final entity label information, which is the entity label sequence composed of the entity labels corresponding to each character.

[0071] Further, the entity label information is input into the relationship extraction sub-model 304, and the relationship between entities is extracted by the relationship extraction sub-model 304 to obtain the entity relationship information. In an embodiment of the present application, according to different application scenarios, the relationship between entities to be extracted is also different. For example, when the first type of relationship of an enterprise is identified, the information extracted by the relationship extraction sub-model 304 is the relationship between entities related to the first type of relationship. When the second type of relationship of an enterprise is identified, the information extracted by the relationship extraction sub-model 304 is the relationship between entities related to the second type of relationship. The first type of relationship and the second type of relationship are different relationship categories, and specifically, they can be opposite relationship categories.

[0072] In an embodiment of the present application, the relation extraction sub-model 304 comprises a label embedding layer 304-1 and a relation classification layer 304-2, wherein the label embedding layer 304-1 is capable of determining a plurality of entities in the text to be recognized according to the entity label information output by the CRF sub-model 303, and obtaining the character vector corresponding to each entity from the text encoding information output by the BILSTM sub-model 302, then taking any two entities in the plurality of entities as target entities, and splicing and compressing the character vector corresponding to the preset bit character of the target entity to form a one-dimensional vector; then the relation classification layer 304-2 is capable of calculating according to the one-dimensional vector to determine the relation between the two target entities. Wherein, the preset bit character can be a character at any position of the text corresponding to the entity, specifically, it can be the first character or the last character in the text, for example, the text corresponding to one entity is "A company", and the text corresponding to another entity is "loss", then "A" and "loss" can be taken as the preset bit character, or "manager" and "loss" can be taken as the preset bit character.

[0073] It is worth noting that the relation classification layer can classify the relation between entities, and the number of classifications varies according to different business scenarios, for example, the relation classification layer can be a sigmoid classification layer, which is used for binary classification, that is, by outputting 0 or 1 to indicate whether there is a certain relation between the two entities, for example, outputting 0 indicates that there is a first type of relation between the two entities, and outputting 1 indicates that there is a second type of relation between the two entities, the relation classification layer can also be a softmax classification layer, which is used for multi-classification.

[0074] In an embodiment of the present application, when the information extraction model performs entity recognition and entity relation extraction on the text to be recognized, it is based on the learned entity recognition rules and entity relation extraction rules to perform information extraction, therefore the performance of the information extraction model depends on the degree of model training, thus it is necessary to fully train the information extraction model to be trained before inputting the text to be recognized into the information extraction model for information extraction.

[0075] When training the information extraction model to be trained, first, a text sample is obtained, then the text sample is input into the information extraction model to be trained, and the information extraction model to be trained is used to perform entity recognition and relation extraction on the text sample to obtain predicted entity relation information; finally, the information extraction model to be trained is optimized according to the labeled entity label and labeled entity relation information corresponding to the text sample and the predicted entity relation information, to obtain the information extraction model.

[0076] Next, the training process of the information extraction model to be trained is described in detail.

[0077] In an embodiment of the present application, the information extraction model to be trained is the same as the composition of the information extraction model, including a Bert sub-model, a to-be-trained BILSTM sub-model, a to-be-trained CRF sub-model, and a to-be-trained relation extraction sub-model. The reason why the Bert sub-model is not the to-be-trained Bert sub-model is that the amount of parameters involved in the model training process is huge, and the Bert sub-model itself is a pre-trained language representation model. Therefore, in order to reduce the amount of data processing, a pre-trained Bert sub-model is used in the present application, and the parameters of the Bert sub-model do not need to be updated in the model training process.

[0078] After obtaining the text sample, the text sample can be input to the information extraction model to be trained. The process of information extraction of the text sample by the information extraction model to be trained is the same as the process of information extraction of the text to be recognized by the information extraction model in the above embodiment. After the characters in the text sample are pre-encoded by the Bert sub-model, pre-encoded information corresponding to each character which integrates the context relationship can be obtained. The pre-encoded information is input to the to-be-trained BILSTM sub-model, and the pre-encoded information is bidirectionally encoded by the to-be-trained BILSTM sub-model to obtain text encoding information corresponding to the text sample. The text encoding information is input to the to-be-trained CRF sub-model, and entity label information corresponding to the text sample is obtained by performing entity recognition based on the text encoding information by the to-be-trained CRF sub-model. Finally, the entity label information is input to the to-be-trained relation extraction sub-model, and the predicted entity relationship information is obtained by performing entity relationship extraction based on the entity label information by the to-be-trained relation extraction sub-model.

[0079] The process of pre-encoding the characters in the text sample by the Bert sub-model is the same as the process of pre-encoding the characters in the text to be recognized by the Bert sub-model in the above embodiment. First, the characters in the text sample are compared with the characters in the word vector table to obtain vectors corresponding to the characters in the text sample. Then, the self-attention encoding is performed based on the vectors corresponding to the characters by the multiple Transformer layers in the Transformer unit to obtain pre-encoded information corresponding to each character which integrates the context information.

[0080] Similarly, the process of bidirectionally encoding the pre-encoded information by the to-be-trained BILSTM sub-model to obtain the text encoding information and the process of performing entity recognition based on the text encoding result by the to-be-trained CRF sub-model and outputting the entity label information are the same as the process of bidirectionally encoding the pre-encoded information by the BILSTM sub-model 302 to obtain the text encoding information and the process of performing entity recognition based on the text encoding information by the CRF sub-model 303 and outputting the entity label information in the above embodiment, and will not be described here.

[0081] After the to-be-trained CRF sub-model outputs entity label information, the to-be-trained relation extraction sub-model can perform entity relation extraction according to the entity label information to obtain predicted entity relation information. Specifically, the to-be-trained relation extraction sub-model can determine multiple entities in the text sample according to the entity label information, and obtain character vectors corresponding to each entity according to the text encoding information. Meanwhile, any two entities in the multiple entities can be taken as target entities, and then character vectors corresponding to preset bit characters corresponding to the target entities are obtained. Next, the character vectors corresponding to the preset bit characters are spliced and compressed to obtain a one-dimensional vector. Finally, a logistic regression function is used to process the one-dimensional vector to obtain predicted entity relation information corresponding to the target entities.

[0082] In the training phase of the to-be-trained information extraction model, the text sample is input into the to-be-trained information extraction model for forward propagation, and then an optimization loss function is determined according to the information output by the to-be-trained information extraction model and the labeled entity label and labeled entity relation information corresponding to the text sample. Next, the to-be-trained information extraction model is parameter-optimized according to the optimization loss function to obtain the information extraction model.

[0083] In an embodiment of the present application, the information extraction model is a model based on the Multi-head Selection algorithm, which belongs to a joint extraction model. When calculating the loss function of the model, the loss functions of the to-be-trained BILSTM sub-model, the to-be-trained CRF sub-model and the to-be-trained relation extraction sub-model need to be considered. Since all outputs of the to-be-trained BILSTM sub-model will be taken as inputs of the to-be-trained CRF sub-model, that is, the loss function of the CRF sub-model needs to be calculated according to the output of the to-be-trained BILSTM sub-model, the loss function of the to-be-trained BILSTM sub-model will be passed to the to-be-trained CRF sub-model, and then the loss functions of the to-be-trained BILSTM sub-model and the to-be-trained CRF sub-model are a loss function. Correspondingly, the loss function of the to-be-trained information extraction model includes two parts, a first loss function corresponding to the to-be-trained BILSTM sub-model and the to-be-trained CRF sub-model, and a second loss function corresponding to the to-be-trained relation extraction sub-model.

[0084] The first loss function is calculated according to the labeled entity label corresponding to the text sample, and the second loss function is calculated according to the labeled entity relation information and the predicted entity relation information corresponding to the text sample. Next, the method for calculating the first loss function and the second loss function will be described.

[0085] In the process of training the to-be-trained CRF sub-model, two parameters are involved: emission score and transition score, wherein the emission score is the output from the to-be-trained BILSTM sub-model, and the transition score represents the score of adjacent word vectors changing from one entity label to another.

[0086] Figure 6 The text encoding information output by the to-be-trained BILSTM sub-model is shown as Figure 6 As shown in the figure, the text sample contains 5 characters, and the Bert sub-model outputs the pre-encoding information corresponding to each character, denoted as w0, w1, w2, w3 and w4 respectively. The types of entity labels include B-person, I-person, B-organization, I-organization and O, a total of 5 types. Then, by bidirectional encoding through the to-be-trained BILSTM sub-model, a 5x5 text encoding information can be obtained, wherein each element represents the probability of each character corresponding to each entity label.

[0087] Since each character corresponds to 5 entity labels, the label paths formed by the entity labels corresponding to the 5 characters have a total of 25. For each label path, the emission score corresponding thereto can be determined according to the text encoding information, and the calculation formula is shown as formula (1):

[0088] Emission score = X 0,B-person + X 1,I-person + X 2,O + X 3,B-organization + X 4,I-organization (1)

[0089] Wherein, Xi,label represents the probability value of the i-th character being marked with label.

[0090] For example, the label path is B-person, I-person, O, B-organization and I-organization. According to the text encoding information shown in Figure 6 , the corresponding probability values can be determined as 1.5, 0.4, 0.8, 1.0 and 0.9 respectively, and the emission score can be calculated as 4.6.

[0091] Figure 7 A transition score matrix is shown as Figure 7As shown, in the transition score matrix, the vertical axis represents the label type before transition, and the horizontal axis represents the label type after transition. Each element represents the score for changing from one label type to another, for example, T(B-person, I-person) = 0.9. For the text in the input model, multiple label paths can be formed based on the possible entity labels of each character. When calculating the transition score corresponding to each label path, the transition score between every two adjacent labels is first calculated, and then all transition scores are added together to obtain the transition score corresponding to that label path. Taking the label paths B-person, I-person, O, B-organization, and I-organization as an example, the corresponding transition score = t start→B-person +t B-person→I-person +t I-person→O +t O→B-organization +t B-organization→I-organization =0.8+0.9+0.85+0.7+0.8+0.2=4.25.

[0092] After obtaining the emission score and transfer score corresponding to the text, the path score S corresponding to the text can be calculated based on the emission score and transfer score, and the calculation formula is shown in formula (2):

[0093] S = O (launch score) + H (transfer score) (2)

[0094] Once you understand how to calculate the path score S, you can then calculate the first loss function. Figure 8 A flowchart illustrating the calculation of the first loss function is shown, as follows: Figure 8 As shown, in step S801, multiple combined paths are determined based on the character and entity tag set contained in the text sample, and the real path is determined based on the labeled entity tags; in step S802, a first emission score and a first transfer score corresponding to each of the combined paths are determined based on the text encoding information and the transfer matrix, and a second emission score and a second transfer score corresponding to the real path are determined; in step S803, a combined path score corresponding to each of the combined paths is determined based on the first emission score and the first transfer score, and a real path score corresponding to the real path is determined based on the second emission score and the second transfer score; in step S804, a total path score is obtained based on each of the combined path scores, and a real path score is obtained based on the real path score; in step S805, the real path score is compared with the total path score to obtain the first loss function.

[0095] In step S803, the formula for calculating the total path score based on the combined path score is shown in formula (3):

[0096] P = P1 + P2 +... + P n = e S1 + e S2 +... + e Sn (3)

[0097] wherein S1, S2,..., Sn are scores of each combined path.

[0098] Correspondingly, after obtaining the real path score, the real path score can also be obtained according to the above calculation method, and the first loss function can be obtained by dividing the real path score by the total path score.

[0099] The second loss function can be obtained by calculating the cross-entropy loss function. In different business scenarios, the types of cross-entropy loss functions are different. For a business scenario that only needs to be classified, only the binary classification cross-entropy loss function needs to be calculated. For a business scenario that needs to be classified, the multi-classification cross-entropy loss function needs to be calculated. Similarly, according to the type of the last layer of the information extraction model to be trained, the type of the loss function can also be determined. When the last layer is a sigmoid classification layer, a binary classification loss function is used. When the last layer is a softmax classification layer, a multi-classification loss function is used.

[0100] Taking the method of adopting binary classification to identify the first type of relationship as an example, finally only the judgment of whether the entities in the text are related and have the first type of relationship is needed. Therefore, when extracting the relationship between entities, either the first type of relationship is extracted or the second type of relationship is extracted. Correspondingly, the corresponding binary classification loss function is shown in formula (4):

[0101]

[0102] wherein y is the labeled entity relationship information corresponding to the text sample, and is 0 or 1, is the predicted entity relationship information, that is, the probability value of the related entities having the first type of relationship.

[0103] In an embodiment of the present application, it is necessary to determine whether there is a mutual relationship between any two entities in a text sample, so when performing entity relationship extraction, the relationship between each two entities can be extracted at the same time, and the scores of the corresponding characters of the two entities on each relationship are calculated through a loss function, and finally the label distribution between the entities is obtained. When extracting the relationship between each two entities, the preset position characters in the text corresponding to the entities can be identified, for example, the first character, the last character, etc. can be used as the preset position characters. For example, the text sample is "X film company's net loss value reaches 1 billion yuan", and the entities are the company name "X film company" and the relationship pointing word "loss", so only the character vectors corresponding to "X" and "loss" or "company" and "loss" are needed to calculate the relationship between the two entities. Since the character vector corresponding to each character is fused with the context semantics, the relationship between the two entities can be obtained even through one character in the text corresponding to the entities.

[0104] Since there are only a small number of mutual relationships between entities when extracting entity relationships, there is a significant sample imbalance problem, which makes it difficult to fit the relationship between entities, that is, there are fewer samples with mutual relationships that can be used to train the information extraction model to be trained, which leads to underfitting of the model obtained by training. Therefore, in order to improve the performance of the information extraction model, the sample ratio needs to be adjusted to overcome the sample imbalance problem. In an embodiment of the present application, the second loss function is mainly scaled as a whole, so that the easy-to-classify samples are reduced more than the difficult-to-classify samples, so that the weights of the difficult-to-classify samples are highlighted in the second loss function, so that the model focuses more on the difficult-to-classify samples during training. The easy-to-classify sample is a text sample in which there is no mutual relationship between entities, and the difficult-to-classify sample is a text sample in which there is a mutual relationship between entities.

[0105] In an embodiment of the present application, the pre-set sample weight and easy-to-classify sample penalty coefficient can be obtained, and then the predicted entity relationship information is optimized according to the sample weight and the easy-to-classify sample penalty coefficient to obtain optimized information, and finally the loss function is determined according to the labeled entity relationship information and the optimized information to obtain the second loss function. The specific expression of the optimized information is shown in formula (5):

[0106]

[0107] Wherein, α t is the sample weight, and γ is the easy-to-classify sample penalty coefficient.

[0108] From the analysis of formula (5), when a sample is misclassified, is very small, close to 0, so tends to 1, indicating that there is no big change compared to the original loss, when a sample is correctly classified, Close to 1, therefore A value close to 0 indicates a small contribution to the overall loss. This means that optimization information can increase the weight of inaccurately classified samples in the loss function, making the loss function favor hard-to-classify samples and overcoming the imbalance problem. Furthermore, γ is adjustable; as γ increases, This will also increase, further enhancing the weight of hard-to-classify samples in the loss function.

[0109] The second loss function, which adds weight to the hard-to-classify samples, can be obtained using formulas (4) and (5).

[0110] After determining the first loss function and the second loss function, the optimized loss function corresponding to the information extraction model to be trained can be determined based on the first loss function and the second loss function. Since a relation corresponds to at most two entities, and the number of samples containing relations in the text is very small, the relation extraction loss function (second loss function) is much smaller than the entity recognition loss function (first loss function). Therefore, in order to avoid the problem of the relation extraction task being weakened, when constructing the optimized loss function corresponding to the information extraction model to be trained, a first weight corresponding to the first loss function and a second weight corresponding to the second loss function are introduced, and the first weight is smaller than the second weight. This can increase the proportion of the relation extraction loss function in the overall loss function. The calculation formula of the optimized loss function corresponding to the information extraction model to be trained is shown in formula (6):

[0111] Loss = W n loss ner +W r loss rel (6)

[0112] Among them, W n As the first weight, loss ner For the first loss function, W r As the second weight, loss rel This is the second loss function.

[0113] As an example, W n It can be set to 1, W r It can be set to 10, or other values, as long as W is guaranteed. r Greater than W n That's all.

[0114] After the optimization loss function corresponding to the information extraction model to be trained, which balances the easy-to-classify samples and the difficult-to-classify samples and strengthens the relationship extraction loss function, is determined, the parameters of the information extraction model to be trained can be updated according to the optimization loss function. In an embodiment of the present application, the optimization parameters that make the optimization loss function reach a minimum value can be obtained by gradient descent of the optimization loss function, and the method of gradient descent of the optimization loss function can be stochastic gradient descent, batch gradient descent, mini-batch gradient descent, etc. By iteratively training the information extraction model to be trained for a preset number of times, the optimal parameters that make the model converge can be obtained, and then the information extraction model that can be used for information extraction tasks is obtained.

[0115] In an embodiment of the present application, when extracting the relationship between entities according to the identified entities, there may be misjudgment, for example, the text information is "Guangdong XX Technology Co., Ltd. The loss value of marketing in the second quarter is 700,000 yuan lower than that in the first quarter", which is a positive information from the overall text information that the enterprise loses less in the second quarter than in the first quarter. However, if only the entities "Guangdong XX Technology Co., Ltd.", "loss" and "decrease" are used to extract the relationship, the opposite recognition result will be obtained, so it is necessary to determine the relationship type from the overall text information. In an embodiment of the present application, when training the information extraction model to be trained, a text global label can be added to the text sample, which is used to indicate the relationship between entities determined according to the entire text. This can enable the trained information extraction model to extract the relationship between entities from the overall text, preventing misjudgment of the relationship between entities. The text global label can be set to different label values according to different business scenarios, for example, when the business scenario is a two-class business scenario, such as identifying the first type of relationship or the second type of relationship for an enterprise, the text global label can be set to 0 or 1, where 0 can represent that the entities are related and there is a first type of relationship, and 1 can represent that the entities are related and there is a second type of relationship, and vice versa; when the business scenario is a multi-class business scenario, such as sentiment recognition, different sentiment labels can be added to the text sample according to the sentiment classification, for example, if the sentiment is classified into four categories, a corresponding label can be set for each category of sentiment, and different text samples can be added with corresponding sentiment labels. When the text sample is input into the information extraction model to be trained, the text global label corresponding to the text sample is also input into the information extraction model to be trained, so that the information extraction model to be trained can be trained according to the text global label, and then the relationship between entities can be extracted from the overall text, improving the accuracy of the extracted information.

[0116] Figure 9 The model training process with the added text global label is shown as Figure 9As shown, the text sample and the corresponding text global label L are input to the information extraction model to be trained at the same time. First, the characters in the text sample are pre-encoded by the Bert sub-model 901 to obtain pre-encoded information corresponding to the characters in the text sample. Then, the pre-encoded information and the text global label are transmitted to the BILSTM sub-model 902 to be trained, and the pre-encoded information and the text global label are bidirectionally encoded by the BILSTM sub-model to obtain text encoding information. Then, the text encoding information is input to the CRF sub-model 903 to be trained, so as to identify the entity in the text sample based on the text encoding information and obtain entity label information, which includes label information corresponding to the text global label. For example, if the text global label is 0, the corresponding label is B-0, and if the text global label is 1, the corresponding label is B-1. Finally, the entity label information is transmitted to the relation extraction sub-model 904 to be trained. The label embedding layer 904-1 in the relation extraction sub-model 904 converts the entity label information into corresponding character vectors according to the text encoding information, and the relation classification layer calculates the input character vectors to obtain the probability that the vectors corresponding to each entity exist in a relationship, and then determines the entity relationship information according to the probability.

[0117] According to the training of the information extraction model in the above embodiment, since the optimization loss function used in the training process considers both the sample imbalance problem and the relationship extraction weakening problem, the information extraction model can better fit, and the accuracy of entity labeling and the accuracy of relationship extraction can be improved. At the same time, the text global label is introduced during training, so that the trained information extraction model can extract the relationship between entities from the whole text, avoiding the misjudgment caused by only extracting the local relationship between entity words.

[0118] The information extraction method in the embodiment of the application can improve the accuracy of the information extraction model in extracting entities and identifying relationships. The performance of the information extraction model mainly reflects in the accuracy P, recall rate R and F value. Table 1 shows the performance of the information extraction model trained by four different training methods, as follows:

[0119] Table 1 Performance of information extraction models trained by different training methods

[0120]

[0121]

[0122] The accuracy P, recall R, and F value of entity extraction and relationship identification of the information extraction model using only the Multi-head Selection (MHS) algorithm-based information extraction model, the information extraction model using the MHS algorithm-based information extraction model and optimizing the sample ratio, the information extraction model using the MHS algorithm-based information extraction model, optimizing the sample ratio, and strengthening the relationship extraction task, and the information extraction model using the MHS algorithm-based information extraction model, optimizing the sample ratio, strengthening the relationship extraction task, and introducing the text global label are shown in Table 1. By comparison, it can be seen that the problems of sample imbalance and weakening of the relationship extraction task are considered in the embodiments of the present application, and the performance of the information extraction model trained by introducing the text global label is optimal.

[0123] The information extraction method in the embodiments of the present application can be applied to multiple scenarios, such as relationship annotation, sentiment recognition, etc. For example, it can identify information of a predetermined relationship category for a certain enterprise or institution, perform sentiment recognition on a certain object, etc. Next, taking the identification of information of a second type of relationship for an enterprise as an example, the information extraction method in the embodiments of the present application is described.

[0124] In today's society, we are in a network era, and information spreads very quickly. If an enterprise cannot obtain information related to it and having a second type of relationship in time and process it in the first time, it may cause serious consequences, such as stock price drop, product oversupply, business volume drop, bankruptcy, etc. In order to obtain information related to the enterprise and having a second type of relationship in the first time, all information related to the enterprise name can be obtained on the network according to the enterprise name, which includes short videos, text reports, comments, and self-media articles, etc. Non-text information can be converted into text information, and then the collected text information related to the enterprise is input into the trained information extraction model to perform entity recognition and relationship recognition between entities on the input text information to determine whether the text information is information related to the enterprise and having a second type of relationship.

[0125] Before using the information extraction model to perform entity recognition and relationship recognition between entities on the text information related to the enterprise, the information extraction model to be trained also needs to be trained to enable the information extraction model to be trained to learn rules that can accurately perform entity recognition and relationship recognition between entities, and then the learned rules can be used to perform entity recognition and relationship recognition between entities on the input text information related to the enterprise to achieve information recognition of the enterprise.

[0126] In training the information extraction model to be trained, historical information related to the enterprise can be obtained, and the historical information is converted into text information, and the entity labels and the relationships between entities contained in the text information are labeled, and the text global label corresponding to the text information is labeled, so as to train the information extraction model to be trained according to the text information, the labeled entity label information and the entity relationship information, and the text global label. In the model training, the characters in the text sample are encoded and the entity recognition is performed by the Bert sub-model, the BILSTM sub-model to be trained and the CRF sub-model to be trained in the information extraction model to be trained to obtain the entity label information in the text sample. Then, the relationship extraction sub-model is used to predict the entity relationship information according to the recognized entity label information, and then the optimization loss function corresponding to the information extraction model to be trained is constructed according to the predicted entity relationship information output by the information extraction model to be trained, the labeled entity label and the labeled entity relationship information corresponding to the text sample, and the parameters of the information extraction model to be trained are optimized based on the optimization loss function, so as to obtain a converged and stable information extraction model. Since the optimization loss function balances the proportion of easy-to-classify samples and difficult-to-classify samples and strengthens the proportion of the relationship extraction task, the information extraction model obtained after the parameter optimization based on the optimization loss function can better recognize various entities and perform relationship extraction. In the model training process, the text global label corresponding to the text sample is also added, so that the trained information extraction model can extract the relationship between entities based on the whole text, rather than only extracting local relationships based on part of the entities, thereby avoiding misjudgment of the relationship between entities.

[0127] The information extraction method in the embodiment of the application extracts information from the text to be recognized by using the information extraction model trained based on the optimized loss function and the text global label. On the one hand, the imbalance between easy-to-classify samples and difficult-to-classify samples and between the relationship extraction loss function and the entity loss function is considered in the optimization loss function, so that the information extraction model to be trained is trained according to the optimized loss function and the text global label, which can improve the fitting ability of the information extraction model in extracting the relationship between entities and improve the accuracy of entity recognition and relationship recognition between entities. On the other hand, the text global label is introduced in training the information extraction model to be trained, so that the ability of the model to extract the relationship between entities from the whole text is improved, and the accuracy of the extracted relationship between entities is further improved.

[0128] It can be understood that in the specific embodiments of the present application, the text corresponding to the dialogue or monologue of the collection processing party is involved, the to-be-identified text is obtained from channels such as forums, micro blogs, video websites, news websites, and text samples and other related data are obtained. When the above embodiments of the present application are applied to specific products or technologies, the permission or consent of the processing party and the information publisher is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0129] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0130] The device embodiment of the present application is introduced below, which can be used to execute the information extraction method in the above embodiments of the present application. For details not disclosed in the device embodiment of the present application, please refer to the information extraction method described above.

[0131] Figure 10 A block diagram of an information extraction device according to one embodiment of the present application is schematically shown.

[0132] Referring to Figure 10 As shown in the figure, the information extraction device 1000 according to one embodiment of the present application comprises an acquisition module 1001 and a processing module 1002.

[0133] The acquisition module 1001 is configured to acquire to-be-identified text, and the processing module 1002 is configured to input the to-be-identified text into an information extraction model, perform entity recognition and relation extraction on the to-be-identified text through the information extraction model, and obtain entity relation information in the to-be-identified text. The information extraction model is generated by training a to-be-trained information extraction model according to an optimized loss function and text global label. The sample imbalance is optimized and the relation extraction task is strengthened in the optimized loss function.

[0134] In an embodiment of the present application, the information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence; the processing module 1002 is configured to: input the to-be-identified text into the Bert sub-model, pre-encode characters in the to-be-identified text through the Bert sub-model to obtain pre-encoding information corresponding to the characters; input the pre-encoding information into the BILSTM sub-model, bidirectionally encode the pre-encoding information through the BILSTM sub-model to obtain text encoding information; input the text encoding information into the CRF sub-model, perform entity identification based on the text encoding information through the CRF sub-model to obtain entity labels corresponding to the to-be-identified text; and input the entity labels into the relation extraction sub-model, perform entity relation extraction according to the entity labels through the relation extraction sub-model to obtain the entity relation information.

[0135] In an embodiment of the present application, the information extraction device 1000 further comprises: a sample acquisition module, configured to, before inputting the to-be-identified text into the information extraction model, acquire a text sample, input the text sample into the to-be-trained information extraction model, and perform entity identification and relation extraction on the text sample through the to-be-trained information extraction model to obtain predicted entity relation information; and a model training module, configured to optimize the to-be-trained information extraction model according to labeled entity labels and labeled entity relation information corresponding to the text sample and the predicted entity relation information, to obtain the information extraction model.

[0136] In an embodiment of the present application, the to-be-trained information extraction model comprises a Bert sub-model, a to-be-trained BILSTM sub-model, a to-be-trained CRF sub-model and a to-be-trained relation extraction sub-model; the sample acquisition module comprises: a first processing unit, configured to input the text sample into the Bert sub-model, pre-encode characters in the text sample through the Bert sub-model to obtain pre-encoding information corresponding to the characters; a second processing unit, configured to input the pre-encoding information into the to-be-trained BILSTM sub-model, bidirectionally encode the pre-encoding information through the to-be-trained BILSTM sub-model to obtain text encoding information; a third processing unit, configured to input the text encoding information into the to-be-trained CRF sub-model, perform entity identification based on the text encoding information through the to-be-trained CRF sub-model to obtain entity label information corresponding to the text sample; and a fourth processing unit, configured to input the entity label information into the to-be-trained relation extraction sub-model, perform entity relation extraction according to the entity label information through the to-be-trained relation extraction sub-model to obtain the predicted entity relation information.

[0137] In an embodiment of the present application, the Bert sub-model is a pre-trained sub-model; the first processing unit is configured to: compare the characters in the text sample with the characters in the character vector table to obtain vectors corresponding to the characters in the text sample; and perform self-attention coding on the vectors corresponding to the characters through a plurality of Transformer layers to obtain the pre-coding information corresponding to the characters.

[0138] In an embodiment of the present application, the fourth processing unit is configured to: determine a plurality of entities in the text sample according to the entity label information, and determine character vectors corresponding to each of the entities according to the entity label information and the text coding information; take any two entities in the plurality of entities as target entities, and obtain character vectors corresponding to preset bit characters corresponding to the target entities; splice and compress the character vectors corresponding to the preset bit characters to obtain a one-dimensional vector; and process the one-dimensional vector using a logistic regression function to obtain predicted entity relationship information corresponding to the target entities.

[0139] In an embodiment of the present application, the model training module comprises: a first loss function obtaining unit configured to determine a first loss function corresponding to the to-be-trained BILSTM sub-model and to-be-trained CRF sub-model according to the labeled entity label; a second loss function obtaining unit configured to determine a second loss function corresponding to the to-be-trained relationship extraction sub-model according to the labeled entity relationship information and the predicted entity relationship information; and a parameter optimization unit configured to determine the optimization loss function according to the first loss function and the second loss function, and to perform parameter optimization on the to-be-trained information extraction model according to the optimization loss function to obtain the information extraction model.

[0140] In an embodiment of the present application, the first loss function obtaining unit is configured to: determine a plurality of combination paths according to the characters and the entity label set contained in the text sample, and determine a true path according to the labeled entity label; determine first emission scores and first transition scores corresponding to each of the combination paths, and second emission scores and second transition scores corresponding to the true path according to the text coding information and the transition matrix; determine combination path scores corresponding to each of the combination paths according to the first emission scores and the first transition scores, and determine a true path score corresponding to the true path according to the second emission scores and the second transition scores; obtain a total path score according to each of the combination path scores, and obtain a true path score according to the true path score; and obtain the first loss function by comparing the true path score with the total path score.

[0141] In an embodiment of the present application, the second loss function obtaining unit is configured to: obtain a sample weight and an easy sample penalty coefficient, optimize the predicted entity relationship information according to the sample weight and the easy sample penalty coefficient to obtain optimization information, determine a classification loss function according to the labeled entity relationship information and the optimization information, and take the classification loss function as the second loss function.

[0142] In an embodiment of the present application, the parameter optimization unit is configured to: obtain a first weight and a second weight, the first weight being smaller than the second weight, multiply the first weight by the first loss function to obtain a first optimized loss function, multiply the second weight by the second loss function to obtain a second optimized loss function, and add the first optimized loss function and the second optimized loss function to obtain the optimization loss function.

[0143] In an embodiment of the present application, the information extraction apparatus 1000 is further configured to: when inputting the text sample into the to-be-trained information extraction model, input a text global label corresponding to the text sample into the to-be-trained information extraction model for training.

[0144] In an embodiment of the present application, the text global label is a relationship category label corresponding to the text sample, the relationship category label including 0 and 1, wherein 0 represents that the relationship between entities in the text sample is a first type of relationship, and 1 represents that the relationship between entities in the text sample is a second type of relationship, the first type of relationship being different from the second type of relationship.

[0145] Figure 11 A structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present application is shown.

[0146] It should be noted that, Figure 11 The computer system 1100 of the electronic device shown is only an example and should not impose any limitation on the functions and use range of embodiments of the present application.

[0147] As Figure 11As shown, the computer system 1100 includes a central processing unit (CPU) 1101 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1102 or loaded into a random access memory (RAM) 1103 from the storage section 1108, and realizes the image labeling method described in the above-described embodiments. In the RAM 1103, various programs and data required for the operation of the system are also stored. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0148] Connected to the I / O interface 1105 are an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as necessary. A removable recording medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1110 as necessary, so that a computer program read therefrom is installed in the storage section 1108 as necessary.

[0149] In particular, according to embodiments of the present application, the processes described below with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 1109, and / or installed from the removable recording medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, various functions defined in the system of the present application are performed.

[0150] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus or device. In this application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination thereof.

[0151] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0152] The units described in the embodiments of the present application can be implemented by software, or can be implemented by hardware, and the described units can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0153] As another aspect, the present application also provides a computer readable medium, which can be included in the information extraction device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements the method described in the above embodiments.

[0154] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0155] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to make a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) execute the method according to the embodiments of the present application.

[0156] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains.

[0157] It should be understood that the present application is not limited to the precise structures described and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An information extraction method characterized by, The method comprises: acquiring a text to be recognized; inputting the text to be recognized into an information extraction model, performing entity recognition and relation extraction on the text to be recognized by the information extraction model to acquire entity relation information in the text to be recognized; wherein the information extraction model is trained by the following method: acquiring a text sample, inputting the text sample and a text global label corresponding to the text sample into a to-be-trained information extraction model, performing entity recognition and relation extraction on the text sample according to the text global label by the to-be-trained information extraction model to acquire predicted entity relation information; determining a first loss function according to the labeled entity label corresponding to the text sample, determining a second loss function according to the labeled entity relation information corresponding to the text sample and the predicted entity relation information, and determining an optimization loss function according to the first loss function and the second loss function, and performing parameter optimization on the to-be-trained information extraction model according to the optimization loss function to acquire the information extraction model; wherein the second loss function is determined according to the labeled entity relation information and the optimized information, comprising: acquiring a sample weight and an easy sample penalty coefficient, optimizing the predicted entity relation information according to the sample weight and the easy sample penalty coefficient to acquire optimized information; determining the second loss function according to the labeled entity relation information and the optimized information; wherein the optimization loss function is determined according to the first loss function and the second loss function, comprising: acquiring a first weight and a second weight, the first weight being smaller than the second weight; multiplying the first weight by the first loss function to acquire a first optimization loss function, and multiplying the second weight by the second loss function to acquire a second optimization loss function; adding the first optimization loss function and the second optimization loss function to acquire the optimization loss function.

2. The method of claim 1, wherein, The information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence; the information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence; the information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence; the information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence; the information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence; the information extraction model comprises a Bert sub-model, a BILSTM sub-model, a CRF sub-model and a relation extraction sub-model connected in sequence.

3. The method of claim 1, wherein, The information extraction model to be trained comprises a Bert sub-model, a BILSTM sub-model to be trained, a CRF sub-model to be trained, and a relation extraction sub-model to be trained. The entity recognition and relation extraction of the text sample by the information extraction model to be trained according to the text global label comprises: inputting the text sample into the Bert sub-model, pre-encoding characters in the text sample by the Bert sub-model to obtain pre-encoding information corresponding to the characters; inputting the pre-encoding information and the text global label into the BILSTM sub-model to be trained, bidirectionally encoding the pre-encoding information and the text global label by the BILSTM sub-model to be trained to obtain text encoding information; inputting the text encoding information into the CRF sub-model to be trained, performing entity recognition based on the text encoding information by the CRF sub-model to be trained to obtain entity label information corresponding to the text sample, wherein the entity label information comprises label information corresponding to the text global label; inputting the entity label information into the relation extraction sub-model to be trained, performing entity relation extraction according to the entity label information by the relation extraction sub-model to be trained to obtain the predicted entity relation information.

4. The method of claim 3, wherein, The Bert sub-model is a pre-trained sub-model. The pre-encoding of the characters in the text sample by the Bert sub-model comprises: comparing the characters in the text sample with characters in a word vector table to obtain vectors corresponding to the characters in the text sample; self-attention encoding the vectors corresponding to the characters by multiple Transformer layers to obtain the pre-encoding information corresponding to the characters.

5. The method of claim 3, wherein, The entity relation extraction of the text sample by the relation extraction sub-model to be trained according to the entity label information comprises: determining multiple entities in the text sample according to the entity label information, and determining character vectors corresponding to the entities according to the entity label information and the text encoding information; taking any two entities in the multiple entities as target entities, obtaining character vectors corresponding to preset bit characters corresponding to the target entities; splicing and compressing the character vectors corresponding to the preset bit characters to obtain a one-dimensional vector; processing the one-dimensional vector by a logistic regression function to obtain predicted entity relation information corresponding to the target entities.

6. The method of claim 3, wherein, The determination of the first loss function according to the annotated entity label corresponding to the text sample comprises: determining the first loss function corresponding to the BILSTM sub-model to be trained and the CRF sub-model to be trained according to the annotated entity label. The determination of the second loss function according to the annotated entity relation information corresponding to the text sample and the predicted entity relation information comprises: Determine a second loss function corresponding to the to-be-trained relation extraction sub-model according to the annotated entity relation information and the predicted entity relation information.

7. The method of claim 6, wherein, The first loss function corresponding to the to-be-trained BILSTM sub-model and the to-be-trained CRF sub-model is determined according to the annotated entity label, and includes: Determine a plurality of combination paths according to the characters and the entity label set contained in the text sample, and determine a real path according to the annotated entity label; Determine a first emission score and a first transition score corresponding to each combination path according to the text encoding information and the transition matrix, and determine a second emission score and a second transition score corresponding to the real path; Determine a combination path score corresponding to each combination path according to the first emission score and the first transition score, and determine a real path score corresponding to the real path according to the second emission score and the second transition score; Obtain a total path score according to each combination path score, and obtain a real path score according to the real path score; Compare the real path score with the total path score to obtain the first loss function.

8. The method according to any one of claims 1 to 7, characterized in that, The text global label is a relation category label corresponding to the text sample, and the relation category label includes 0 and 1, wherein 0 represents that the relation between entities in the text sample is a first type of relation, and 1 represents that the relation between entities in the text sample is a second type of relation, and the first type of relation is different from the second type of relation.

9. An information extraction apparatus characterized by comprising: The device includes: An acquisition module is configured to acquire a to-be-identified text; A processing module is configured to input the to-be-identified text into an information extraction model, perform entity identification and relation extraction on the to-be-identified text through the information extraction model, and obtain entity relation information in the to-be-identified text. The information extraction model is trained in the following manner: Obtain a text sample, input the text sample and a text global label corresponding to the text sample into a to-be-trained information extraction model, perform entity identification and relation extraction on the text sample according to the text global label through the to-be-trained information extraction model, and obtain predicted entity relation information; Determine a first loss function according to annotated entity labels corresponding to the text sample, determine a second loss function according to annotated entity relation information and predicted entity relation information corresponding to the text sample, determine an optimization loss function according to the first loss function and the second loss function, and perform parameter optimization on the to-be-trained information extraction model according to the optimization loss function to obtain the information extraction model; The second loss function is determined according to the annotated entity relation information and the predicted entity relation information, and includes: Obtain a sample weight and an easy sample penalty coefficient, optimize the predicted entity relation information according to the sample weight and the easy sample penalty coefficient to obtain optimized information; Determine the second loss function according to the annotated entity relation information and the optimized information; The optimization loss function is determined according to the first loss function and the second loss function, and includes: obtaining a first weight and a second weight, the first weight being less than the second weight; multiplying the first weight with the first loss function to obtain a first optimized loss function, and multiplying the second weight with the second loss function to obtain a second optimized loss function; adding the first optimized loss function and the second optimized loss function to obtain the optimized loss function.

10. A computer readable medium characterized by comprising instructions which, when executed on a computer, cause the computer to perform the information extraction method of any one of claims 1 to 8.

11. An electronic device, comprising: comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to invoke the executable instructions to implement the information extraction method of any one of claims 1 to 8.

12. A computer program product comprising instructions, characterized in that, comprising instructions which, when executed on a computer, cause the computer to perform the information extraction method of any one of claims 1 to 8.

Images