Text processing, knowledge base construction method, device and storage medium
By conducting joint training and iterative training of coding networks and entity/relational classification networks on natural language processing methods, the low accuracy problem caused by independent training of entity recognition and relational classification models in traditional methods is solved, and higher recognition accuracy is achieved and the accuracy of knowledge base construction.
Patent Information
- Application Number
- CN202011403298.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-04
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2040-12-04
AI Technical Summary
When traditional natural language processing methods recognize entity relationships, the named entity recognition model and the relationship classification model are independently trained, ignoring the correlation between the two tasks, resulting in low model accuracy.
The training text is encoded through the encoding network, combined with the entity recognition network and the relationship classification network for joint training, updated the network parameters, and carried out multiple iterative training to fully consider the correlation between entity recognition and relationship classification.
The accuracy of entity recognition and relationship classification is improved, making the model more accurate when identifying entity relationships and can better build a knowledge base.
Smart Images

Figure CN112380867B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text processing method, apparatus, computer equipment and storage medium, and a knowledge base construction method, apparatus, computer equipment and storage medium. Background Art
[0002] With the rapid development of artificial intelligence technology, natural language processing has been widely used in many aspects. In natural language processing, entity relationship recognition is often required. Entity relationship recognition, also known as entity relationship extraction, aims to extract structured information from large-scale unstructured or semi-structured natural language sentences to determine the semantic relationship between entities in natural language sentences. It can solve the problem of entity classification in natural language sentences and is also an important basis for building complex knowledge base systems, such as text summarization, automatic question answering, machine translation, search engines, knowledge graphs, etc.
[0003] In traditional technology, when extracting entity relationships, a named entity recognition model is usually used to first perform entity recognition, and then a relationship classification model is used to classify the relationships of the recognized entity pairs. In this way, the named entity recognition model and the relationship classification model are trained through two independent training tasks, which often ignores the correlation between the two tasks, resulting in low accuracy of the trained model. Summary of the invention
[0004] Based on this, it is necessary to provide a text processing method, device, computer equipment and storage medium that can improve the accuracy of the model in response to the above technical problems.
[0005] A text processing method, comprising:
[0006] Obtaining training text and label data corresponding to the training text;
[0007] Encoding the training text through a coding network to obtain a training coding sequence corresponding to the training text;
[0008] Identify entities of the training text according to the training coding sequence and the entity recognition network to obtain entity recognition results;
[0009] Identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain a relationship recognition result;
[0010] Jointly training the encoding network, the entity recognition network, and the relationship classification network based on the entity recognition result, the relationship recognition result, and the label data to update the encoding network, the entity recognition network, and the relationship classification network;
[0011] According to the updated encoding network and the relationship recognition result, the training encoding sequence is updated, and the step of identifying the entities of the training text according to the training encoding sequence and the entity recognition network is returned to perform iterative training. When the iteration stop condition is met, the trained target encoding network, target entity recognition network and target relationship classification network are obtained; wherein the target encoding network, target entity recognition network and target relationship classification network are used to cooperate in identifying the entity relationships of the text to be processed.
[0012] In one embodiment, the relationship classification network includes a hidden layer, a convolution layer and a pooling layer; the inputting the training code segment into the relationship classification network for relationship recognition to obtain the relationship recognition result corresponding to the training entity pair includes: inputting the training code segment into the hidden layer, processing the training code segment through the hidden layer to obtain the corresponding first intermediate feature; inputting the first intermediate feature into the convolution layer, performing convolution processing on the first intermediate feature through the convolution layer, and obtaining the corresponding second intermediate feature; inputting the second intermediate feature into the pooling layer, and performing pooling processing on the second intermediate feature through the pooling layer; and performing entity relationship classification according to the pooling processing result to obtain a relationship recognition result.
[0013] A text processing device, comprising:
[0014] A data acquisition module, used to acquire training text and label data corresponding to the training text;
[0015] An encoding module, used for encoding the training text through an encoding network to obtain a training encoding sequence corresponding to the training text;
[0016] An entity recognition module, used to recognize entities of the training text according to the training coding sequence and the entity recognition network, and obtain entity recognition results;
[0017] A relationship recognition module, used to identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain a relationship recognition result;
[0018] A joint training module, used for jointly training the encoding network, the entity recognition network and the relationship classification network based on the entity recognition result, the relationship recognition result and the label data, so as to update the encoding network, the entity recognition network and the relationship classification network;
[0019] An iterative training module is used to update the training coding sequence according to the updated coding network and the relationship recognition result, and return to the step of identifying the entities of the training text according to the training coding sequence and the entity recognition network to perform iterative training. When the iteration stop condition is met, the trained target coding network, target entity recognition network and target relationship classification network are obtained; wherein the target coding network, target entity recognition network and target relationship classification network are used to cooperate in identifying the entity relationship of the text to be processed.
[0020] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned text processing method when executing the computer program.
[0021] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned text processing method when executed by a processor.
[0022] The above-mentioned text processing method, device, computer equipment and storage medium encode the training text through an encoding network to obtain a training encoding sequence corresponding to the training text, identify the entities of the training text according to the training encoding sequence and the entity recognition network to obtain an entity recognition result, identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain a relationship recognition result, and jointly train the encoding network, the entity recognition network and the relationship classification network based on the entity recognition result, the relationship recognition result and the label data to obtain an updated encoding network, entity recognition network and relationship classification network. Due to the joint training, the entity recognition network and the relationship classification network can share the network parameters of the encoding network, so the dependency between the entity recognition network and the relationship classification network can be achieved during the training process, so that the obtained entity recognition network and relationship classification network are more accurate. Furthermore, by performing multiple rounds of iterative training on the encoding network, the entity recognition network and the relationship classification network, and in each iteration, the training encoding sequence is updated according to the updated encoding network and the relationship recognition result, and when the iteration stop condition is met, the trained target encoding network is obtained. The target entity recognition network and the target relationship classification network can fully consider the correlation between the entity recognition network and the relationship classification network, and further improve the accuracy of the entity recognition network and the relationship classification network.
[0023] A method for constructing a knowledge base, characterized in that the method comprises:
[0024] Obtaining a text to be processed, encoding the text to be processed through a trained target encoding network, and obtaining an encoding sequence corresponding to the text to be processed;
[0025] Identify the entities of the text to be processed according to the encoding sequence and the trained target entity recognition network to obtain an entity recognition result;
[0026] Identify the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain a relationship recognition result;
[0027] The target encoding network, the target entity recognition network and the target relationship classification network are obtained by iterative joint training based on training texts; in each iterative training, the training encoding sequence input to the entity recognition network obtained by the previous training is updated according to the relationship recognition result output by the relationship classification network obtained by the previous training;
[0028] A knowledge base is constructed according to the entity recognition results and the relationship recognition results.
[0029] In one of the embodiments, the method for constructing the knowledge base also includes: obtaining a prescription to be reviewed, wherein the prescription to be reviewed includes a drug name; querying the corresponding target entity and target relationship from the drug knowledge base according to the drug name; and reviewing the prescription to be reviewed based on the queried target entity and target relationship to obtain a review result.
[0030] A device for constructing a knowledge base, characterized in that the device comprises:
[0031] A text acquisition module is used to acquire the text to be processed, encode the text to be processed through the trained target encoding network, and obtain the encoding sequence corresponding to the text to be processed;
[0032] An entity recognition module is used to recognize entities of the text to be processed according to the coding sequence and the trained target entity recognition network to obtain an entity recognition result;
[0033] A relationship recognition module is used to identify the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain a relationship recognition result; the target encoding network, the target entity recognition network and the target relationship classification network are obtained by iterative joint training based on the training text; in each iterative training, the training encoding sequence input to the entity recognition network obtained by the previous training is updated according to the relationship recognition result output by the relationship classification network obtained by the previous training;
[0034] The knowledge base construction module is used to construct a knowledge base according to the entity recognition results and the relationship recognition results.
[0035] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned knowledge base construction method when executing the computer program.
[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for constructing a knowledge base.
[0037] The above-mentioned knowledge base construction method, device, computer equipment and storage medium, after obtaining the text to be processed, encodes the text to be processed through the trained target encoding network to obtain the encoding sequence corresponding to the text to be processed, identifies the entity of the text to be processed according to the encoding sequence and the trained target entity recognition network to obtain the entity recognition result, identifies the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain the relationship recognition result, and constructs the knowledge base according to the entity recognition result and the relationship recognition result. Since the target encoding network, the target entity recognition network and the target relationship classification network are obtained by iterative joint training based on the training text, In each iterative training, the training coding sequence input into the entity recognition network obtained in the previous training is updated according to the relationship recognition result output by the relationship classification network obtained in the previous training. During the training process, the dependency and correlation between the entity recognition network and the relationship classification network are fully considered. The obtained entity recognition network and the target relationship classification network have good generalization performance and can accurately identify the entities and relationships in the processed text. The knowledge base constructed according to the entity recognition results and the relationship recognition results has significantly improved accuracy compared to the knowledge base established by manual annotation in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 An application environment diagram of a text processing method and a knowledge base construction method in one embodiment;
[0039] Figure 2 A flowchart of a text processing method in one embodiment;
[0040] Figure 3 A schematic diagram of a flow chart of steps before obtaining training text in one embodiment;
[0041] Figure 4 A flowchart of a text processing method in one embodiment;
[0042] Figure 5 A schematic diagram of the structure of a text processing network in one embodiment;
[0043] Figure 6is a schematic diagram of a BLSTM network in one embodiment;
[0044] Figure 7 A structural diagram of a CNN network in one embodiment;
[0045] Figure 8 An effect diagram of a text processing method in one embodiment;
[0046] Fig. 9 A schematic diagram of the steps of a method for constructing a knowledge base in one embodiment;
[0047] Fig. 10A It is a process diagram of a text processing method and a knowledge base construction method in an application scenario;
[0048] Fig. 10B A relationship diagram of the required working days and the magnitude of the knowledge base is shown in an embodiment;
[0049] Fig.11 is a structural block diagram of a text processing device in one embodiment;
[0050] Fig.12 is a structural block diagram of a knowledge base construction device in one embodiment;
[0051] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0053] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0054] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0055] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.
[0056] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
[0057] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.
[0058] The solutions provided in the embodiments of the present application involve technologies such as natural language processing and machine learning of artificial intelligence, which are specifically explained through the following embodiments.
[0059] The text processing method provided in this application can be applied to Figure 1 In the application environment shown, the server and the terminal can execute the text processing method independently, or the server and the terminal can execute the text processing method in collaboration.
[0060] In one embodiment, taking the server and the terminal collaboratively executing the text processing method as an example, the server can first obtain the training text and the label data corresponding to the training text, encode the training text through the encoding network to obtain the training coding sequence corresponding to the training text, identify the entities of the training text according to the training coding sequence and the entity recognition network to obtain the entity recognition result, identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain the relationship recognition result, and jointly train the encoding network, the entity recognition network and the relationship classification network based on the entity recognition result, the relationship recognition result and the label data to update the encoding network, The entity recognition network and the relationship classification network update the training coding sequence according to the updated coding network and the relationship recognition result, and return the step of identifying the entities of the training text according to the training coding sequence and the entity recognition network for iterative training. When the iteration stop condition is met, the trained target coding network, the target entity recognition network and the target relationship classification network are obtained. When performing entity relationship recognition, the server can send the obtained target coding network, the target entity recognition network and the target relationship classification network to the terminal, and the terminal can identify the entity relationship of the text to be processed according to the target coding network, the target entity recognition network and the target relationship classification network; or the terminal can send the text to be recognized to the server, and the server can identify the entity relationship of the text to be processed according to the target coding network, the target entity recognition network and the target relationship classification network, and return the recognition result to the terminal.
[0061] The text processing method provided in this application can be applied to Figure 1 In the application environment shown, the server and the terminal can execute the text processing method independently, or the server and the terminal can execute the text processing method in collaboration.
[0062] The terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.
[0063] In one embodiment, Figure 2 As shown, a text processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:
[0064] Step 202: Obtain training text and label data corresponding to the training text.
[0065] Among them, the training text refers to the sample text used to train the text processing network. The language of the training text can be determined according to actual needs. For example, the training text can be a Chinese sentence or an English sentence. The label data corresponding to the training text refers to the training label for supervised training of the text processing network. The text processing network here refers to a machine learning model for entity recognition and relationship recognition of the text to be processed. The text processing network includes an encoding network, an entity recognition network, and a relationship classification network. During the training process, the text processing network uses the training samples as input data and the label data as the expected output data, and continuously learns. At the end of the training, the obtained text processing network can perform entity recognition and relationship extraction on the input text to be processed. Then, during the supervised training process, the label data used may include the entity labels and relationship labels corresponding to the training samples.
[0066] Specifically, the server can obtain training samples and label data corresponding to the training samples from the training data set. The training data set includes multiple training samples and label data corresponding to each training sample. The training data set can be a manually annotated data set, that is, the label data corresponding to the training samples is obtained by manually annotating entities and relationships in the text; the training data set can also be a machine-automatically annotated data set, that is, the label data corresponding to the training samples is obtained by automatically annotating entities and relationships in the text by a computer.
[0067] In one embodiment, the training data set may be stored in a database, and the server may obtain the training text and the label data corresponding to the training text from the database. In another embodiment, the training data set may be stored in other computer devices, and the server may obtain the training text and the label data corresponding to the training text from the other computer devices through a network or other means.
[0068] Step 204: encode the training text through the encoding network to obtain a training encoding sequence corresponding to the training text.
[0069] The encoding network refers to the machine learning module used for encoding, and encoding refers to the process of converting information from one form or format to another form or format. The training encoding sequence refers to the encoding result obtained by the server encoding the text through the encoding network during the training process. The text element refers to the elements that make up the text, such as characters and words in the text.
[0070] Specifically, after the server obtains the training text, it inputs the training text into the encoding network, and encodes each text element in the training text through the encoding network. During the encoding process, each text element needs to be encoded one by one to obtain the code corresponding to each text element. The codes corresponding to each text element are sorted according to the position order of the text element in the text to form a training code sequence corresponding to the training text.
[0071] In one embodiment, when the encoding network is encoding, the server first performs feature mapping on each text element in the training text, maps each text element to the feature space to obtain a corresponding feature representation, and then encodes the obtained feature representation to obtain element encoding features, such as through a BLSTM (Bi-directional Long Short-Term Memory) network, an LSTM (Long Short-Term Memory) network, and a CNN (Convolutional Neural Networks) network to encode the feature representation of each text element to obtain element encoding features, and finally obtains a training encoding sequence corresponding to the training text based on these element encoding features.
[0072] In a specific embodiment, the server may map each character in the training text to a feature space through an embedding algorithm to obtain a corresponding feature representation. In another specific embodiment, the server may first perform word segmentation on the text to obtain a plurality of word segments corresponding to the text, and then map each word segmentation in the training text to a feature space through an embedding algorithm to obtain a corresponding feature representation. Word segmentation methods may include, for example, a forward maximum matching method, a reverse maximum matching method, a shortest path word segmentation method, a bidirectional maximum matching method, a word meaning word segmentation method, or a statistical word segmentation method, etc. For example, assuming that the training text is "Today is Sunday", the word segmentation obtained after word segmentation processing may be "Today / is / Sunday".
[0073] Step 206, identifying entities in the training text according to the training coding sequence and the entity recognition network to obtain entity recognition results.
[0074] The entity recognition network refers to a machine learning module for entity recognition (Named Entity Recognition, NER). The entity recognition network can use CRF (Conditional Random Field) network, LSTM (Long Short-Term Memory) network, HMM (Hidden Markov Model), etc.
[0075] Entities are text elements with characteristic meanings in training texts, usually words or phrases. For example, in the medical field, "The patient developed paroxysmal coughs and sputum after a rash more than 50 years ago. The cough was not severe. The sputum was white and sticky, easy to cough up, and the amount was small. He also felt chest tightness and shortness of breath after activities." Among them, "paroxysmal", "cough", "sputum", "not severe", "white sputum", "sticky", "easy to cough up", "small amount", "chest tightness", "shortness of breath", etc. are all entities.
[0076] Specifically, the server inputs the training coding sequence into the entity recognition network, performs entity recognition through the entity recognition network, and obtains entity recognition results. According to the entity recognition results, the server can determine multiple predicted entities corresponding to the training text.
[0077] In one embodiment, the entity recognition result may be a category identifier used to characterize the category to which each text element in the training text belongs. For example, assuming that the entity type includes two categories, namely, name and organizational structure, the actual number of categories includes five categories, namely, B-Person (the beginning part of the name), I-Person (the middle part of the name), B-Organization (the beginning part of the organization), I-Organization (the middle part of the organization), and O (non-entity information). The training text X is composed of text elements w1, w2, and w3. For the text element w1, the entity recognition result corresponding to the text element can be represented by a vector (1, 0, 0, 0, 0). According to the entity recognition result, it can be determined that w1 belongs to the B-Person category.
[0078] In another embodiment, the entity recognition result can be a probability or score for characterizing the entity category to which each text element in the training text belongs. The larger the probability or score corresponding to a category, the greater the possibility that the text element belongs to the category. For example, in the above example, for the text element w1, the entity recognition results can be 1.5 (B-Person), 0.9 (I-Person), 0.1 (B-Organization), 0.08 (I-Organization) and 0.05 (O). According to the entity recognition result, it can be determined that w1 belongs to the B-Person category.
[0079] Step 208, identifying entity relationships of the training text according to the entity recognition results and the relationship classification network to obtain relationship recognition results.
[0080] The relationship classification network refers to a machine learning module used to classify the relationships between entities to identify the relationships between entities. The relationship classification network can be, for example, CNN (Convolutional Neural Networks).
[0081] In one embodiment, entity relationships of a training text are identified based on entity recognition results and a relationship classification network, and the relationship recognition results are obtained, including: determining at least two training entities corresponding to the training text based on the entity recognition results; determining corresponding training entity pairs based on the at least two training entities; determining training coding segments corresponding to the training entity pairs from the training coding sequences; and inputting the training coding segments into the relationship classification network for relationship recognition to obtain relationship recognition results corresponding to the training entity pairs.
[0082] It can be understood that an entity relationship refers to the relationship between two entities. Then, when the server obtains the entity recognition result, it can determine at least two predicted entities corresponding to the training text based on the entity recognition result, pair these predicted entities, and then classify and identify the relationship between the entity pairs to obtain the relationship recognition result.
[0083] Since the relationship between entities is usually contained in the text description between the two entities, when classifying and identifying the relationship between entity pairs, the server can determine the encoding corresponding to the entity pair and the intermediate text of the entity pair from the training encoding sequence, obtain the encoding segment corresponding to the entity pair, and classify and identify the relationship between the entity pair through the encoding segment to obtain the relationship recognition result.
[0084] In one embodiment, the server may pair each two predicted entities of at least two predicted entities respectively to obtain multiple entity pairs. In another embodiment, since not all entities have relationships when performing relationship identification, the server may first determine that one or more main entities are needed, and then pair the main entities with other entities respectively, thereby obtaining multiple entity pairs. The main entity here refers to the entity that needs to be identified and has an entity relationship with it in the training text. According to different application scenarios, the server determines different main entities. For example, when the training text is a drug instruction manual, the entity relationship is the relationship between the drug and other entities, and the main entity is the drug name. For another example, when the training text is a disease diagnosis text, the entity relationship is the relationship between the disease and other entities, and the main entity can be the disease name.
[0085] Step 210 , jointly training the encoding network, the entity recognition network, and the relationship classification network based on the entity recognition results, the relationship recognition results, and the label data, so as to update the encoding network, the entity recognition network, and the relationship classification network.
[0086] Specifically, the label data includes entity labels and relationship labels. The server can determine the first loss function according to the difference between the entity recognition result and the entity label, determine the second loss function according to the difference between the relationship recognition result and the relationship label, and determine the first descent gradient generated by the first loss function and the second loss function in each iteration process according to the preset deep learning optimization algorithm, superimpose the descent gradient of the first loss function and the second descent gradient of the second loss function, and obtain the comprehensive descent gradient. The deep learning optimization algorithm can specifically be batch gradient descent (Batch Gradient Descent, BGD), stochastic gradient descent (Stochastic Gradient Descent, SGD), mini-batch gradient descent (Mini-Batch Gradient Descent, MBGD), AdaGrad (adaptive algorithm), RMSProp (Root Mean Square Prop) or Adam (Adaptive Moment Estimation), etc. The server back-propagates the comprehensive descent gradient to the encoding network, entity recognition network and relationship classification network to update the network parameters corresponding to the encoding network, entity recognition network and relationship classification network to realize joint training of the encoding network, entity recognition network and relationship classification network until the current training is terminated when the preset training stop condition is met, and the updated encoding network, entity recognition network and relationship classification network are obtained.
[0087] Taking the stochastic gradient descent method as an example, assuming that L1 and L2 are the first loss function and the second loss function respectively, f1(x,Θ adapt1 ) represents the actual output value of the entity recognition network when the input is x, Θ adapt1 is the network parameter of the entity recognition network, y1 is the entity label value of the entity recognition network, f2(x,Θ adapt2 ) represents the actual output value of the relational classification network when the input is x, Θ adapt2 is the network parameter of the relational classification network, y2 is the relational label value of the relational classification network, Θ adapt3 is the network parameter of the encoding network. The training data set contains n training samples {x (1) ,…,x (n)}, where x (i) The output target value of the corresponding entity recognition network is y1 (i) , where x (i) The output target value of the corresponding relational classification network is y2 (i) , then the comprehensive descent gradient corresponding to each iteration is Assuming that the learning rate of the stochastic gradient descent algorithm is η, the network parameters of the entity recognition network can be updated to Θ after back propagation.adapt1 -ηg, update the network parameters of the relational classification network to Θ adapt2 -ηg, update the network parameters of the encoding network to Θ adapt3 -ηg, and continue to iterate the changed network parameters as the current network parameters until the preset training stop condition is reached, ending the current training. The training stop condition may be that the comprehensive loss value reaches the preset minimum value, or the performance of the text processing network has not been significantly improved after a preset number of consecutive iterations.
[0088] Step 212, based on the updated encoding network and relationship recognition results, the training encoding sequence is updated, and the step of identifying entities of the training text based on the training encoding sequence and the entity recognition network is returned to perform iterative training. When the iteration stop condition is met, the trained target encoding network, target entity recognition network and target relationship classification network are obtained.
[0089] Specifically, the server inputs the training sample into the updated coding network, encodes the training sample through the updated coding network, obtains the coding result, and merges the relationship recognition result with the obtained coding result to update the training coding sequence, and returns to step 206 to start the second round of iterative training. When the iteration stop condition is met, the trained target coding network, target entity recognition network and target relationship classification network are obtained. Among them, the iteration stop condition can be that the number of iterations reaches a preset number of times, and the preset number of times can be set as needed, for example, it can be set to 3 times. The obtained target coding network, target entity recognition network and target relationship classification network can be used to cooperate in identifying the entity relationship of the text to be processed.
[0090] In one embodiment, the relationship recognition result and the obtained encoding result are merged by horizontally splicing the relationship recognition result and the encoding result. For example, assuming that the training text Y includes 5 text elements w1, w2, w3, w4, and w5, the encodings corresponding to the text elements w1 and w3 are k-dimensional vectors, respectively represented as (a1, a2...a k )、(b1,b2……b k ), the relationship recognition result representation vector corresponding to w1 and w3 is a j-dimensional vector represented by (c1, c2…cj), then (c1, c2…cj) is horizontally concatenated with (a1, a2…ak) and (b1, b2…bk) to obtain a k+j-dimensional vector represented by (a1, a2…a k , c1, c2...cj) and (b1, b2...b k , c1, c2…cj).
[0091] In the above text processing method, the training text is encoded by the encoding network to obtain the training encoding sequence corresponding to the training text, the entities of the training text are identified according to the training encoding sequence and the entity recognition network to obtain the entity recognition result, the entity relationship of the training text is identified according to the entity recognition result and the relationship classification network to obtain the relationship recognition result, and the encoding network, the entity recognition network and the relationship classification network are jointly trained based on the entity recognition result, the relationship recognition result and the label data to obtain the updated encoding network, the entity recognition network and the relationship classification network. Due to the joint training, the entity recognition network and the relationship classification network can share the network parameters of the encoding network, so that the dependency between the entity recognition network and the relationship classification network can be achieved during the training process, so that the obtained entity recognition network and the relationship classification network are more accurate. Further, by performing multiple rounds of iterative training on the encoding network, the entity recognition network and the relationship classification network, and in each iteration process, the training encoding sequence is updated according to the updated encoding network and the relationship recognition result, when the iteration stop condition is met, the trained target encoding network, the target entity recognition network and the target relationship classification network are obtained, so that the correlation between the entity recognition network and the relationship classification network can be fully considered, and the accuracy of the entity recognition network and the relationship classification network is further improved.
[0092] In one embodiment, before obtaining the training text and the label data corresponding to the training text, the method also includes: performing word segmentation processing on the training text to obtain a corresponding set of text elements; matching the set of text elements with an entity dictionary, and determining at least two first entity labels corresponding to the training text based on the matching results; the entity dictionary is obtained based on a pre-constructed knowledge graph; determining the entity relationship corresponding to at least two first entity labels based on the knowledge graph, and obtaining a first relationship label corresponding to the training text.
[0093] Specifically, the server first performs word segmentation processing on the training text, and composes the words obtained by the word segmentation processing into a text element set. The word segmentation method may be, for example, a forward maximum matching method, a reverse maximum matching method, a shortest path word segmentation method, a bidirectional maximum matching method, a word meaning word segmentation method, or a statistical word segmentation method, etc. For example, assuming that the training text is "Today is Sunday", the word segmentation obtained after the word segmentation processing may be "Today / is / Sunday".
[0094] The server obtains a preset entity dictionary, matches each text element in the text element set with each entity in the entity dictionary, and when any word in the text element successfully matches an entity in the entity dictionary, the text element is determined as the first entity label. The entity dictionary is obtained based on a pre-constructed knowledge graph, and the entities of each node in the knowledge graph are extracted to obtain the entity dictionary.
[0095] In one embodiment, during the matching process, the similarity between the words in the text element and the entities in the entity dictionary can be calculated. When the similarity is greater than a preset threshold, it is determined that the text element and the entity are successfully matched. The similarity calculation can use string similarity or cosine similarity. In another embodiment, the server can first normalize the text elements in the text element set, that is, unify the words in the original element set into standard word expressions, for example, normalize "diarrhea" and "diarrhea" to "diarrhea".
[0096] Furthermore, after obtaining the first entity label, it is necessary to determine the entity relationship corresponding to the first entity label. Since the first entity label is determined by matching with the entity dictionary, and the entity dictionary is obtained based on the knowledge graph, and the knowledge graph includes the corresponding entity relationships in the entity dictionary, the server can match the first entity label with the nodes in the knowledge graph, so as to determine the entity relationship corresponding to the first entity label based on the entity relationship in the knowledge graph, and obtain the first relationship label corresponding to the training text.
[0097] In the above embodiment, the first entity label is determined by matching the text element set with the entity dictionary. The pre-built knowledge graph can be used to extract entities and entity relationships in the training text, and the extracted entities and entity relationships are used as label data for the training text, thereby obtaining a small amount of high-quality annotated data and reducing the time and labor costs of manual annotation.
[0098] In one embodiment, Figure 3 As shown, before obtaining the training text, the above method also includes:
[0099] Step 302: perform word segmentation processing on the training text to obtain a corresponding text element set.
[0100] Step 304, matching the text element set with the entity dictionary, and determining at least two first entity tags corresponding to the training text according to the matching result; the entity dictionary is obtained according to a pre-constructed knowledge graph.
[0101] Step 306: determine entity relationships corresponding to at least two first entity tags based on the knowledge graph, and obtain first relationship tags corresponding to the training text.
[0102] Step 308: determine candidate text elements from the text element set, where the candidate text elements are text elements other than at least two first entity tags.
[0103] Step 310, calculating the word coagulation degree of the candidate text element.
[0104] The candidate text element is a text element other than at least two first entity tags. Since the entity dictionary is constructed based on a pre-constructed knowledge graph, the number of entities it contains is limited. For some new words, the entity dictionary may not contain them. In this case, a new word discovery algorithm can be used to extract entities based on word coagulation. For example, a candidate text element includes three characters A, B, and C. The formula for calculating word coagulation is as follows: Formula (1):
[0105]
[0106] Among them, P(ABC) is the probability of characters A, B, and C appearing together, P(A) is the probability of character A appearing alone, P(BC) is the probability of characters B and C appearing together, P(AB) is the probability of characters A and B appearing together, and P(C) is the probability of character C appearing alone. The larger the value, the less independent the three characters A, B, and C are. That is, the three characters A, B, and C are highly correlated and are likely to appear at the same time. In other words, the internal cohesion of the word is very high. The minimum value among is determined as the solidification degree of the candidate text element.
[0107] Step 312: When the calculated word coagulation degree exceeds a preset threshold, the candidate text element is determined as a second entity tag corresponding to the training text.
[0108] Specifically, the server determines whether the word coagulation degree of the candidate text element exceeds a preset threshold. If so, the server determines that the candidate text element is the second entity label corresponding to the training text, so that all entities in the training text can be extracted as comprehensively as possible.
[0109] Step 314, obtaining the entity relationship corresponding to the second entity tag, and obtaining the second relationship tag corresponding to the training text.
[0110] The entity relationships corresponding to the second entity tags include two types: entity relationships between second entity tags and entity relationships between the second entity tags and the first entity tags.
[0111] Specifically, the server can obtain a pre-trained relationship extraction model, determine an entity label pair based on the first entity label and the second entity label, input the entity label pair and the intermediate text corresponding to the entity label pair into the pre-trained relationship extraction model, and obtain the relationship corresponding to the entity label as the second relationship label corresponding to the training text.
[0112] In the above embodiment, entities in the training text can be extracted as comprehensively as possible by combining entity dictionary matching and word solidification calculation, so that the obtained training text can be better used in the training process and the accuracy of training is improved.
[0113] In one embodiment, Figure 4 As shown, the above method also includes:
[0114] Step 402, obtain the candidate text, encode the candidate text through the encoding network, and obtain the encoding sequence corresponding to the candidate text.
[0115] Step 404, identifying entities of the candidate text according to the encoding sequence corresponding to the candidate text and the entity recognition network, and obtaining entity recognition results corresponding to the candidate text.
[0116] Step 406, identifying the entity relationship of the candidate text according to the entity recognition result corresponding to the candidate text and the relationship classification network, and obtaining the relationship recognition result corresponding to the candidate text.
[0117] Among them, the candidate text is an unlabeled text. After obtaining the candidate text, the server encodes the candidate text through the encoding network to obtain the corresponding encoding sequence, inputs the encoding sequence into the entity recognition network for entity recognition to obtain the entity recognition result corresponding to the candidate text, and then determines the corresponding encoding segment from the encoding sequence according to the entity pair determined by the entity result and inputs it into the relationship classification network to perform relationship recognition on the entity pair to obtain the relationship recognition result.
[0118] Step 408, when the candidate text is determined to be an uncertain sample based on at least one of the entity recognition result corresponding to the candidate text and the relationship recognition result corresponding to the candidate text, the candidate text is sent to a preset terminal.
[0119] Uncertain samples refer to the most uncertain samples for classification. The most uncertain samples are selected as training samples because the data with less certainty may be more difficult to classify. These data are often some data near the boundary. In the embodiments of the present application, the active learning algorithm can learn more boundary information by observing these data.
[0120] In the process of active learning, when the server determines that the candidate text is an uncertain sample according to at least one of the entity recognition result corresponding to the candidate text and the relationship recognition result corresponding to the candidate text, the candidate text is sent to a preset terminal, where the preset terminal refers to a terminal that can provide manual annotation. Specifically, the server can use uncertainty measurement methods such as the smallest margin uncertainty (SMU), the least confidence uncertainty (LCU), and the largest margin uncertainty (LMU) as the basis for active learning to select annotation samples.
[0121] Taking the minimum margin uncertainty as an example, the minimum margin uncertainty reflects the best and second-best probabilities, that is, the probability of the most likely category minus the probability of the second most likely category. The meaning behind this value is: if the probability of the most likely category is significantly greater than the probability of the second most likely category, then the classification is very certain which category the data belongs to. Similarly, if the probability of the most likely category is not much greater than the probability of the second most likely category, then the classification is not so certain which category the data belongs to.
[0122] Specifically, the calculation formula of the minimum margin uncertainty is as follows:
[0123] φ SM (x) = P θ (y1|x)-P θ (y2|x) (2)
[0124] Among them, P θ (y1|x) is the probability of the most likely category, P θ (y2|x) is the probability of the second possible category. The server can refer to formula (2) to calculate the minimum margin uncertainty for the probability that each text element in the entity recognition result belongs to each entity category, and calculate the minimum margin uncertainty for the probability that the relationship corresponding to the entity pair in the relationship recognition result belongs to each relationship category. When any calculated minimum margin uncertainty is less than the preset threshold value, the candidate text is determined to be an uncertain sample.
[0125] Step 410: When the entity label and the relationship label corresponding to the candidate text returned by the preset terminal are received, the target encoding network, the target entity recognition network and the target relationship classification network are continuously trained according to the candidate text.
[0126] Specifically, the user corresponding to the preset terminal (for example, an expert or an authority) performs entity annotation and relationship annotation on the candidate text, and the preset terminal determines the annotation results corresponding to the entity annotation and the annotation results corresponding to the relationship annotation as the entity label and relationship label corresponding to the candidate text, and sends the entity label and relationship label corresponding to the candidate text to the server. At this time, since the candidate text has corresponding label data, it can be used for supervised training of the entity recognition network and the relationship classification network. Then the server can use the candidate text as a training sample to continue training the target encoding network, the target entity recognition network and the target relationship classification network to update the network parameters of the target encoding network, the target entity recognition network and the target relationship classification network. When the training stop condition is met, the training is terminated. It can be understood that as the number of training samples continues to increase, the accuracy of the text processing network gradually converges.
[0127] In the above embodiment, through active learning, only uncertain samples need to be selected for labeling, and all samples do not need to be labeled, which reduces the manpower cost of manual labeling. Moreover, since uncertain samples are usually data close to the classification boundary and have a large amount of information, manual labeling of these uncertain samples can obtain high-quality training samples. Model training through these high-quality training samples can improve the generalization performance of the model.
[0128] In one embodiment, the encoding network includes a feature mapping layer and an encoding layer; the training text is composed of a plurality of ordered text elements; encoding the training text through the encoding network to obtain a training encoding sequence corresponding to the training text includes: performing feature mapping on each text element through the feature mapping layer to obtain a feature representation corresponding to each text element; inputting each feature representation into the encoding layer to obtain an element encoding feature corresponding to each text element; and obtaining a training encoding sequence corresponding to the training text according to the element encoding feature corresponding to each text element.
[0129] Feature mapping refers to mapping text elements to a feature space of fixed dimensions to obtain a feature vector of fixed length, which is the feature representation corresponding to the text element. The text elements of the training text can be individual characters that make up the text elements or segmented words obtained by segmenting the text elements.
[0130] Specifically, the server inputs each text element into the encoding layer of the encoding network. The encoding layer uses a preset feature mapping algorithm to map the text elements into a feature space of fixed dimensions to obtain the feature representation corresponding to each text element. When performing feature mapping, the server can use the embedding method to perform feature mapping on the text elements. The main purpose of the embedding method is to reduce the dimension of (sparse) features. Its dimension reduction method can be compared to a fully connected layer (without an activation function). The dimension is reduced by calculating the weight matrix of the embedding layer, that is, large sparse vectors can be converted into a low-dimensional space that retains semantic relationships. Therefore, the server can map the text elements to a feature space of fixed dimensions to obtain feature vectors, thereby realizing feature extraction of text elements.
[0131] After obtaining the feature representation corresponding to each text element, the service inputs the feature representation corresponding to each text element into the encoding layer, and encodes each feature representation through the encoding layer to obtain the element encoding features corresponding to each text element. The encoding layer can specifically adopt network structures such as BLSTM (Bi-directional Long Short-Term Memory) network, LSTM (Long Short-Term Memory) network, CNN (Convolutional Neural Networks, Convolutional Neural Networks) network, etc.
[0132] After obtaining the element coding features corresponding to each text element, the element coding features are combined into a coding sequence according to the order of the corresponding text elements in the training text to obtain a training coding sequence corresponding to the training text.
[0133] In one embodiment, each feature representation is input into a coding layer to obtain an element coding feature corresponding to each text element, including: encoding each feature representation in a forward direction according to the sequence of text elements corresponding to each feature representation to obtain a forward coding feature corresponding to each text element; encoding each feature representation in a backward direction according to the sequence of text elements corresponding to each feature representation to obtain a backward coding feature corresponding to each text element; and fusing the forward coding feature and the backward coding feature corresponding to each text element to obtain an element coding feature corresponding to each text element.
[0134] Specifically, when the server encodes at the encoding layer, it can encode each feature representation in the forward direction according to the order of the text elements corresponding to each feature representation, and obtain the front encoding features corresponding to each text element, and the obtained front encoding features contain the forward information of the training text. The server further encodes each feature representation in the backward direction according to the order of the text elements corresponding to each feature representation, and obtains the backward encoding features corresponding to each text element, and the obtained backward encoding features contain the backward information of the training text. The server further fuses the forward encoding features and backward encoding features corresponding to each text element, and obtains the element encoding features corresponding to each text element. At this time, the obtained element encoding features contain the forward information and the backward information, which can better express the features of the text elements. Fusion refers to the process of expressing multiple features with one feature, and fusion can specifically be splicing, combination, etc.
[0135] In one embodiment, the entity recognition network includes a decoding layer and a classification layer; the entities of the training text are identified according to the training coding sequence and the entity recognition network, and the entity recognition result is obtained, including: at the decoding layer, the element coding features are decoded according to the sequence of text elements corresponding to each element coding feature to obtain the element decoding features corresponding to each text element; at the classification layer, each element decoding feature is subjected to entity classification processing to obtain the entity recognition result.
[0136] Specifically, the server inputs each element encoding feature into the decoding layer, and in the decoding layer, the element encoding feature is decoded according to the order of the text elements corresponding to each element encoding feature to obtain the element decoding feature corresponding to each text element, wherein decoding is the inverse process of encoding. In a specific embodiment, the decoding layer can use an LSTM network.
[0137] After obtaining the element decoding features, the server further inputs the element decoding features into the classification layer. In the classification layer, each element decoding feature is subjected to entity classification processing to obtain an entity recognition result. According to the entity recognition result, at least two predicted entities corresponding to the training text can be determined. In a specific embodiment, the classification layer can use a softmax algorithm for classification.
[0138] In one embodiment, a relationship classification network includes a hidden layer, a convolution layer and a pooling layer; a training coding segment is input into the relationship classification network for relationship recognition, and the relationship recognition result corresponding to the training entity pair is obtained, including: inputting the training coding segment into the hidden layer, processing the training coding segment through the hidden layer, and obtaining the corresponding first intermediate feature; inputting the first intermediate feature into the convolution layer, and convolution processing is performed on the first intermediate feature through the convolution layer to obtain the corresponding second intermediate feature; inputting the second intermediate feature into the pooling layer, and pooling processing is performed on the second intermediate feature through the pooling layer; entity relationship classification is performed according to the pooling processing result to obtain a relationship recognition result.
[0139] Specifically, the server inputs the training code segment into the hidden layer, and abstracts the training code segment into another dimensional space through the hidden layer to obtain the first intermediate feature. Compared with the original training code segment, the first intermediate feature can show more abstract features, thereby better performing linear division and facilitating subsequent classification processing.
[0140] The server then inputs the first intermediate feature into the convolution layer, and performs convolution processing on the first intermediate feature through the convolution layer. The convolution processing process can be understood as a feature extraction process, and the second intermediate feature is obtained after the convolution processing.
[0141] The server then inputs the second intermediate feature into the pooling layer, where the second intermediate feature is pooled. The purpose of pooling is to reduce the dimension of the feature, thereby reducing the amount of calculation during classification. Maximum pooling (MaxPooling) and average pooling (Average Pooling) use pooling functions to further process the feature mapping results obtained by the convolution operation. Pooling will statistically summarize the feature values of an unknown and adjacent position in the plane. And use the summarized result as the value of this position in the plane. Maximum pooling calculates the maximum value of the position and its adjacent matrix area, and uses this maximum value as the value of the position. Average pooling calculates the average value of the position and its adjacent matrix area, and uses this value as the value of the position. Using pooling will not change the depth of the data matrix, but will only reduce it in height and bandwidth to achieve the purpose of dimensionality reduction.
[0142] Finally, the server can classify entity relationships according to the pooling results to obtain entity recognition results. In one embodiment, the server can input the pooling results into a fully connected layer, and classify entity relationships through the fully connected layer to obtain relationship recognition results.
[0143] In a specific embodiment, reference Figure 5 , shows a schematic diagram of the structure of a text processing network in a specific embodiment. Figure 5, the text processing method provided in the present application is specifically described. The text processing method provided in the embodiment of the present application includes three rounds of iterative training.
[0144] 1) First round of training:
[0145] refer to Figure 5 , the training text includes five text elements w1, w2, w3, w4, w5. The server first inputs the training text into the encoding network, which includes an input layer, an embedding layer, and an encoding layer. In the input layer, the server inputs w1, w2, w3, w4, w5 into the embedding layer. The embedding layer performs feature mapping on w1, w2, w3, w4, w5 through the embedding method, and obtains the feature representations of w1, w2, w3, w4, w5 as e1, e2, e3, e4, e5 respectively. The server then inputs the feature representations e1, e2, e3, e4, e5 into the encoding layer for encoding to obtain the element encoding features h1, h2, h3, h4, h5 corresponding to each text element. h1, h2, h3, h4, h5 constitute an element encoding sequence, in which the encoding layer uses a BLSTM network for bidirectional encoding. The schematic diagram of the BLSTM network is shown in the figure. Figure 6 shown.
[0146] like Figure 6 As shown, the BLSTM network includes a forward encoding layer and a backward encoding layer, and the input is e i-1 ,e i ,e i+1 , the forward encoding layer forward encodes the input to obtain The backward coding layer performs backward coding to obtain Will Splicing to get e i-1 The corresponding encoding feature is h t-1 ,Will Splicing to get e i The corresponding encoding feature is h t ,Will Splicing to get e i+1 The corresponding encoding feature is h t+1 .
[0147] The forward coding layer and the backward coding layer of the BLSTM network both use LSTM networks. For the LSTM network, its specific implementation is as follows (3)-(6):
[0148] f t =σ(W f x t +U f h t-1 +b f ) (3)
[0149] i t =σ(W i x t +U i h t-1 +b i ) (4)
[0150] o t =σ(W o x t +U o h t-1 +b o ) (5)
[0151] c t =f t ·c t-1 +i t ·σ(W c x t +U c h t-1 +b c ) (6)
[0152] Among them, [W f ,U f ,b f ,W i ,U i ,b i ,W o ,U o ,b o ] are the corresponding weights and are the network model parameters of LSTM; f t is the output of the forget gate at time t in the LSTM network, σ is the sigmoid function, x t Input for the current time, h t-1 is the output of the LSTM network at time t-1; i t is the output of the input gate in the LSTM network; o t is the output of the output gate in the LSTM network; c t is the cell memory at time t, c t-1 is the cell memory at time t-1.
[0153] Continue to refer Figure 5, the server then uses the element encoding sequence as the input of the entity recognition network, i.e. x1, x2, x3, x4, x5. The entity recognition network includes a decoding layer and a classification layer. The server inputs x1, x2, x3, x4, x5 into the decoding layer. The decoding layer uses an LSTM network. The decoding layer decodes x1, x2, x3, x4, x5 to obtain element decoding features y1, y2, y3, y4, y5. The element decoding features are further input into the softmax layer for entity classification to obtain the entity recognition result, where FB, FI, FE, O, and LU are the entity categories represented by the entity recognition result.
[0154] Continue to refer Figure 5 The server determines the training coding segment corresponding to the entity pair from the element coding sequence output by the coding network according to the entity recognition result, and classifies the relationship through the CNN network to obtain the relationship recognition result, where r1, r2, and r3 are the scores of the entity relationship corresponding to the entity pair belonging to each relationship category. The server jointly trains the coding network, entity recognition network, and relationship classification network based on the difference between the entity recognition result and the entity label, and the difference between the relationship recognition result and the relationship label until the network converges, ending this round of training. The specific structure of the CNN network is as follows: Figure 7 shown.
[0155] refer to Figure 7 The CNN network includes a hidden layer, a convolution layer, and a pooling layer. The training coding segment is input into the hidden layer, and the training coding segment is processed through the hidden layer to obtain the corresponding first intermediate feature. The first intermediate feature is input into the convolution layer, and the first intermediate feature is convoluted through the convolution layer to obtain the corresponding second intermediate feature. The second intermediate feature is input into the pooling layer, and the second intermediate feature is pooled through the pooling layer. The entity relationship is classified according to the pooling processing result to obtain the relationship recognition result.
[0156] 2) Second round of training:
[0157] The server uses the encoding network, entity recognition network and relationship classification network obtained in the first round of training as the current encoding network, current entity recognition network and current relationship classification network, inputs the training text into the current encoding network, repeats the steps of the first round of training to obtain the training encoding sequence, and further inputs r1, r2, r3 into the activation function layer for normalization, and concatenates the normalized result with the entities corresponding to r1, r2, r3 in the training encoding sequence to obtain the current input x1, x2, x3, x4, x5 of the entity recognition network. The current input decoding layer is decoded to obtain element decoding features y1, y2, y3, y4, y5, and the element decoding features are further input into the softmax layer for entity classification to obtain entity recognition results. The server determines the training coding segment corresponding to the entity pair from the element coding sequence output by the encoding network according to the entity recognition results, and classifies the relationship through the CNN network to obtain the relationship recognition results, where r1, r2, and r3 are the scores of the entity relationships corresponding to the entity pairs belonging to each relationship category. The server jointly trains the encoding network, entity recognition network, and relationship classification network based on the difference between the entity recognition result and the entity label, and the difference between the relationship recognition result and the relationship label until the network converges, ending this round of training.
[0158] (3) The third round of training:
[0159] The server uses the encoding network, entity recognition network and relationship classification network obtained in the second round of training as the current encoding network, current entity recognition network and current relationship classification network, and repeats the steps of the second round of training until the network converges, ending the entire training process and obtaining the target encoding network, target entity recognition network and target relationship classification network.
[0160] like Figure 8 As shown, it is a rendering of the text processing method of the present application, which specifically shows the accuracy of the trained target entity recognition network and target relationship classification network, and the accuracy of the separately trained entity recognition model and separately trained relationship classification model in the related art when performing entity recognition and relationship extraction. Figure 8 , showing that the target entity recognition network and target relationship classification network trained by the text processing method of the present application have significantly improved accuracy in identifying disease, surgery, indication, drug and symptom entities and entity relationships related to these types of entities.
[0161] In one embodiment, the above-mentioned text processing method also includes: obtaining the text to be processed, encoding the text to be processed through a target encoding network, and obtaining a coding sequence corresponding to the text to be processed; identifying the entities of the text to be processed according to the coding sequence corresponding to the text to be processed and the target entity recognition network, and obtaining the entity recognition result corresponding to the text to be processed; identifying the entity relationship of the text to be processed according to the entity recognition result and the target relationship classification network, and obtaining the relationship recognition result corresponding to the text to be processed; and constructing a knowledge base according to the entity recognition result and the relationship recognition result corresponding to the text to be processed.
[0162] Among them, the text to be processed refers to the text to be subjected to entity recognition and relationship extraction. After the server obtains the text to be processed, it encodes the text to be processed through the target encoding network to obtain the encoding sequence corresponding to the text to be processed, identifies the entity of the text to be processed according to the encoding sequence corresponding to the text to be processed and the target entity recognition network, obtains the entity recognition result corresponding to the text to be processed, identifies the entity relationship of the text to be processed according to the entity recognition result and the target relationship classification network, obtains the relationship recognition result corresponding to the text to be processed, and the server can determine the entity in the text to be processed according to the entity recognition result, and can determine the entity relationship of the text to be processed according to the relationship recognition result, thereby realizing entity recognition and relationship extraction of the text to be processed. The server further stores the identified entities and entity relationships in the knowledge base to construct the knowledge base.
[0163] In one embodiment, Fig. 9 As shown in the figure, a knowledge base construction method is provided, which can be applied to Figure 1 In the application environment shown. The server and the terminal can each execute the knowledge base construction method independently, or the server and the terminal can execute the knowledge base construction method collaboratively. The knowledge base construction method specifically includes the following steps:
[0164] Step 902, obtaining the text to be processed, encoding the text to be processed through the trained target encoding network, and obtaining the encoding sequence corresponding to the text to be processed.
[0165] Step 904, identifying entities of the text to be processed according to the encoding sequence and the trained target entity recognition network to obtain entity recognition results.
[0166] Step 906, identifying the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain the relationship recognition result.
[0167] Among them, the target encoding network, target entity recognition network and target relationship classification network are obtained by iterative joint training based on the training text; in each iterative training, the training encoding sequence input into the entity recognition network obtained by the previous training is updated according to the relationship recognition result output by the relationship classification network obtained by the previous training.
[0168] Step 908: construct a knowledge base based on the entity recognition results and the relationship recognition results.
[0169] It can be understood that the implementation details of this embodiment can be referred to the description in the above embodiment, and this application will not go into details here.
[0170] In the above embodiment, after obtaining the text to be processed, the text to be processed is encoded by the trained target encoding network to obtain the encoding sequence corresponding to the text to be processed, the entities of the text to be processed are identified according to the encoding sequence and the trained target entity recognition network to obtain the entity recognition result, the entity relationship of the text to be processed is identified according to the entity recognition result and the trained target relationship classification network to obtain the relationship recognition result, and a knowledge base is constructed according to the entity recognition result and the relationship recognition result. Since the target encoding network, the target entity recognition network and the target relationship classification network are obtained by iterative joint training based on the training text, in each iterative training, the training encoding sequence input to the entity recognition network obtained by the previous training is updated according to the relationship recognition result output by the relationship classification network obtained by the previous training. During the training process, the dependency and correlation of the entity recognition network and the relationship classification network are fully considered. The obtained entity recognition network and the target relationship classification network have good generalization performance and can accurately identify the entities and relationships in the text to be processed, so that the knowledge base constructed according to the entity recognition results and the relationship recognition results is significantly improved in accuracy compared with the knowledge base established by manual annotation in the related technology.
[0171] It is understandable that in the knowledge base construction method provided in the embodiment of the present application, the target encoding network, the target entity recognition network and the target relationship classification network can be trained using the text processing method provided in any of the above embodiments, and the present application will not elaborate on them here.
[0172] In one embodiment, the text to be processed is a drug instruction manual text; the knowledge base is a drug knowledge base; the entity relationship of the text to be processed is identified according to the entity recognition result and the trained target relationship classification network, and the relationship recognition result is obtained, including: determining at least two predicted entities corresponding to the text to be processed according to the entity recognition result; determining the target drug name from the at least two predicted entities, and forming a predicted entity pair with each other predicted entity; determining the coding segment corresponding to each predicted entity pair from the coding sequence; and inputting each coding segment into the target relationship classification network for relationship recognition, to obtain the relationship recognition result corresponding to each predicted entity pair.
[0173] The target drug name refers to the name of the drug used in the drug instructions. For example, if a drug instruction is the instruction for Amoxicillin Capsules, then the target drug name is Amoxicillin Capsules.
[0174] In this embodiment, the text to be processed is a drug instruction sheet, and the entity types obtained by entity recognition of the text to be processed include drugs, diseases, symptoms, etc. Relationship recognition mainly identifies the relationship between the drug used in the drug instruction sheet and other entities. Therefore, the target drug name in at least two predicted entities determined according to the entity recognition result can be determined as the main entity, and the main entity is respectively formed into an entity pair with other predicted entities, and then the coding segments corresponding to each predicted entity pair are determined from the coding sequence, and each coding segment is respectively input into the target relationship classification network for relationship recognition, and the relationship recognition results corresponding to each predicted entity pair are obtained. According to the relationship recognition results, the relationship type corresponding to the entity pair can be determined, and the relationship types include "suitable for", "contraindicated for", "suitable population", etc.
[0175] As shown in Table 1 below, this is an example of the knowledge of the drug instructions for a certain drug. Among them, based on the knowledge corresponding to the indications, the entities "bacillary dysentery", "chronic enteritis", "intestinal infection" and "diarrhea" can be extracted; based on the knowledge corresponding to the adverse reactions, the entities "nausea", "vomiting", "dizziness", "headache", "blurred vision" and "chapped skin" can be extracted; based on the knowledge corresponding to the contraindicated drugs, the entity "strong muscles and bones tablets" can be extracted; based on the knowledge corresponding to the complications, the entities "receptor agonists" and "hypokalemia" can be extracted; based on the knowledge corresponding to the drug ingredients, the entities "protein" and "hydrolyzed protein" can be extracted.
[0176] Table 1
[0177]
[0178] In one embodiment, the method for constructing the above-mentioned knowledge base also includes: obtaining the prescription to be reviewed, which includes the name of the drug; querying the corresponding target entity and target relationship from the drug knowledge base according to the drug name; reviewing the prescription to be reviewed according to the queried target entity and target relationship to obtain the review result.
[0179] Among them, the prescription to be reviewed refers to the medical prescription to be reviewed for accuracy. The prescription to be reviewed includes information such as the name of the drug, the dosage of the drug, and the diagnosis result of the disease. The diagnosis result of the disease specifically includes the name of the disease, symptoms, etc. After obtaining the prescription to be reviewed, the corresponding target entity and target relationship can be queried from the drug knowledge base according to the name of the drug in the prescription to be reviewed. Since the target entity and target relationship corresponding to the drug are obtained according to the drug instructions, the target entity and target relationship corresponding to the drug name cover the content of the drug instructions. Therefore, the prescription to be reviewed can be reviewed according to the queried target entity and target relationship to obtain the review result. For example, the disease in the prescription to be reviewed can be compared with the queried indication entity to review whether the drug in the prescription is applicable to the treatment of the current disease; the drug dosage in the prescription to be reviewed can also be compared with the queried drug dosage entity to review whether the drug dosage in the prescription is applicable to the treatment of the current disease.
[0180] The present application also provides an application scenario, which applies the above-mentioned text processing method and knowledge base construction method. In this application scenario, it is necessary to build a drug knowledge base based on the hospital's drug list and conduct prescription review based on the drug knowledge base. In this application scenario, the steps of the text processing method and the knowledge base construction method can be Fig. 10A To illustrate, the following is combined Fig. 10A Explain the application of text processing methods and knowledge base construction methods in this application scenario.
[0181] 1. Obtain unlabeled medical short sentence texts, automatically label them through new word discovery and medical dictionaries, and generate training data sets.
[0182] Specifically, the medical sentence text is segmented to obtain a corresponding text element set, the text element set is matched with a medical dictionary, and at least two first entity labels corresponding to the medical sentence text are determined based on the matching result, wherein the medical dictionary is obtained based on a pre-constructed knowledge graph, and therefore the entity relationship corresponding to the at least two first entity labels can be determined based on the knowledge graph to obtain the first relationship label corresponding to the medical sentence text.
[0183] Further, a candidate text element is determined from the text element set, the candidate text element being a text element other than at least two first entity tags, and the word coagulation degree of the candidate text element is calculated. When the calculated word coagulation degree exceeds a preset threshold, the candidate text element is determined as the second entity tag corresponding to the medical phrase text, the entity relationship corresponding to the second entity tag is obtained, and the second relationship tag corresponding to the medical phrase text is obtained.
[0184] 2. The encoding network, entity recognition network and relationship classification network are jointly trained through the training data set. The encoding network, entity recognition network and relationship classification network constitute a text processing network.
[0185] Among them, the encoding network includes the embedding layer and the BLSTM network layer; the entity recognition network includes the LSTM network layer and the softmax classification layer; the relationship classification network includes the hidden layer, the convolution layer and the pooling layer.
[0186] Medical short sentence texts and entity labels and relationship labels corresponding to the medical short sentence texts are obtained from the training data set. In the embedding layer, the medical short sentence texts are feature mapped to each text element to obtain the feature representation corresponding to each text element. In the BLSTM network layer, each feature representation is encoded in the forward direction according to the sequence of text elements corresponding to each feature representation to obtain the forward encoding feature corresponding to each text element. In the backward direction according to the sequence of text elements corresponding to each feature representation to obtain the backward encoding feature corresponding to each text element, the forward encoding feature and the backward encoding feature corresponding to each text element are fused respectively to obtain the element encoding feature corresponding to each text element. According to the element encoding feature corresponding to each text element, the training encoding sequence corresponding to the medical short sentence text is obtained. In the LSTM network layer, the element encoding feature is decoded according to the sequence of text elements corresponding to each element encoding feature to obtain to the element decoding features corresponding to each text element; in the softmax classification layer, entity classification processing is performed on the decoding features of each element to obtain an entity recognition result, at least two training entities corresponding to the medical short sentence text are determined according to the entity recognition result, and the corresponding training entity pair is determined according to the at least two training entities, and the training coding segment corresponding to the training entity pair is determined from the training coding sequence, and the training coding segment is input into the relationship classification network for relationship recognition to obtain the relationship recognition result corresponding to the training entity pair, and the encoding network, entity recognition network and relationship classification network are jointly trained based on the difference between the entity recognition result and the entity label, and the difference between the relationship recognition result and the relationship label to update the encoding network, entity recognition network and relationship classification network, and the training coding sequence is updated according to the updated encoding network and relationship recognition result, and the processing step of the embedding layer is returned to perform iterative training. When the iteration stop condition is met, the trained target encoding network, target entity recognition network and target relationship classification network are obtained.
[0187] 3. Obtain the drug instructions of each drug in the drug list, preprocess the text of each drug instruction, input the preprocessed drug instruction text into the text processing network for entity recognition and relationship recognition, obtain the entities and relationships corresponding to each drug instruction, and build a drug knowledge base based on these entities and relationships.
[0188] Preprocessing the drug instructions text may specifically include subject replacement, sentence segmentation, word segmentation, etc. of the drug instructions text. Among them, subject replacement refers to replacing other expressions referring to the drug in the drug instructions with the drug name, for example, replacing "the drug" in the drug instructions with "drug name". Sentence segmentation may specifically be segmenting the drug instructions text according to punctuation marks in the instructions text. After the preprocessing is completed, the obtained drug instructions text is input into the target text processing network, and the drug instructions text is encoded by the target encoding network of the text processing network to obtain the encoding sequence corresponding to the drug instructions text. The entities of the drug instructions text are identified according to the encoding sequence and the target entity recognition network to obtain the entity recognition result, and the entity relationship of the drug instructions text is identified according to the entity recognition result and the target relationship classification network to obtain the relationship recognition result. The entities and relationships corresponding to the drug instructions are determined according to the entity recognition result and the relationship recognition result, and a drug knowledge base is constructed according to the entities and relationships. The constructed drug knowledge base can be used for prescription review.
[0189] 4. Select uncertain samples through active learning algorithms for manual labeling, and continue to train the target encoding network, target entity recognition network and target relationship classification network through the labeled samples.
[0190] According to the entity recognition results and relationship recognition results corresponding to the drug instructions text, it is judged whether the drug instructions text is an uncertain sample. If so, the drug instructions text is sent to the terminal corresponding to the pharmacist for entity and relationship annotation. After the pharmacist returns the annotation results, the entity and relationship data corresponding to the drug instructions in the drug knowledge base are updated according to the annotation results. At the same time, the drug instructions text is used as a training sample to continue training the target encoding network, target entity recognition network and target relationship classification network to update the network parameters of the target encoding network, target entity recognition network and target relationship classification network. When the convergence conditions are met, the training ends. The generalization performance of the updated target encoding network, target entity recognition network and target relationship classification network has been further improved.
[0191] Compared with the related art of manually annotating the entity and relationship of the drug instructions to build a drug knowledge base, the combination of joint learning and active learning in this application scenario can greatly improve the accuracy of knowledge base construction with a small amount of manual annotation cost.
[0192] Fig. 10B This is a graph showing the relationship between the number of working days required for manual annotation of drug instructions and the level of the knowledge base in an embodiment. As can be seen from FIG10 , with the continuous iteration of the level of the drug knowledge base, the accuracy of the text processing network gradually converges, and the time required for annotation gradually decreases.
[0193] It should be understood that although Figure 1-1 The steps in the flowchart of FIG. 0 are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Moreover, Figure 1-1 At least part of the steps in 0 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0194] In one embodiment, Fig.11 As shown, a text processing device 1100 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes:
[0195] The data acquisition module 1102 is used to acquire the training text and the label data corresponding to the training text;
[0196] The encoding module 1104 is used to encode the training text through an encoding network to obtain a training encoding sequence corresponding to the training text;
[0197] An entity recognition module 1106 is used to recognize entities of the training text according to the training coding sequence and the entity recognition network to obtain an entity recognition result;
[0198] A relationship identification module 1108 is used to identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain a relationship recognition result;
[0199] A joint training module 1110, configured to jointly train the encoding network, the entity recognition network and the relationship classification network based on the entity recognition result, the relationship recognition result and the label data, so as to update the encoding network, the entity recognition network and the relationship classification network;
[0200] The iterative training module 1112 is used to update the training coding sequence according to the updated coding network and the relationship recognition result, and return to the step of identifying the entities of the training text according to the training coding sequence and the entity recognition network for iterative training. When the iteration stop condition is met, the trained target coding network, target entity recognition network and target relationship classification network are obtained; wherein the target coding network, target entity recognition network and target relationship classification network are used to cooperate in identifying the entity relationship of the text to be processed.
[0201] In one embodiment, the above-mentioned device also includes: a labeling module, which is used to perform word segmentation on the training text to obtain a corresponding set of text elements; match the text element set with the entity dictionary, and determine at least two first entity tags corresponding to the training text according to the matching results; the entity dictionary is obtained according to a pre-constructed knowledge graph; determine the entity relationship corresponding to at least two first entity tags according to the knowledge graph, and obtain the first relationship tag corresponding to the training text.
[0202] In one embodiment, the annotation module is also used to determine candidate text elements from the text element set, the candidate text elements being text elements other than at least two first entity tags; calculating the word coagulation degree of the candidate text elements; when the calculated word coagulation degree exceeds a preset threshold, determining the candidate text element as the second entity tag corresponding to the training text; obtaining the entity relationship corresponding to the second entity tag, and obtaining the second relationship tag corresponding to the training text.
[0203] In one embodiment, the above-mentioned device also includes: an active learning module, which is used to obtain candidate text, encode the candidate text through an encoding network, and obtain a coding sequence corresponding to the candidate text; identify the entity of the candidate text according to the coding sequence corresponding to the candidate text and the entity recognition network, and obtain the entity recognition result corresponding to the candidate text; identify the entity relationship of the candidate text according to the entity recognition result corresponding to the candidate text and the relationship classification network, and obtain the relationship recognition result corresponding to the candidate text; when the candidate text is determined to be an uncertain sample according to at least one of the entity recognition result corresponding to the candidate text and the relationship recognition result corresponding to the candidate text, the candidate text is sent to a preset terminal; when the entity label and relationship label corresponding to the candidate text returned by the preset terminal are received, the training data set is updated according to the candidate text.
[0204] In one embodiment, the encoding network includes a feature mapping layer and an encoding layer; the encoding module is also used to perform feature mapping on each text element through the feature mapping layer to obtain a feature representation corresponding to each text element; each feature representation is input into the encoding layer to obtain an element encoding feature corresponding to each text element; and a training encoding sequence corresponding to the training text is obtained according to the element encoding features corresponding to each text element.
[0205] In one embodiment, the encoding module is also used to encode each feature representation in a forward direction according to the sequence of text elements corresponding to each feature representation, so as to obtain forward encoding features corresponding to each text element; encode each feature representation in a backward direction according to the sequence of text elements corresponding to each feature representation, so as to obtain backward encoding features corresponding to each text element; and fuse the forward encoding features and backward encoding features corresponding to each text element respectively, so as to obtain element encoding features corresponding to each text element.
[0206] In one embodiment, the entity recognition network includes a decoding layer and a classification layer; the entity recognition module is also used to decode the element encoding features at the decoding layer according to the order of text elements corresponding to each element encoding feature to obtain element decoding features corresponding to each text element; at the classification layer, perform entity classification processing on each element decoding feature to obtain an entity recognition result.
[0207] In one embodiment, the entity recognition module is also used to determine at least two training entities corresponding to the training text based on the entity recognition results; determine the corresponding training entity pair based on the at least two training entities; determine the training coding segment corresponding to the training entity pair from the training coding sequence; input the training coding segment into the relationship classification network for relationship recognition, and obtain the relationship recognition result corresponding to the training entity pair.
[0208] In one embodiment, the relationship classification network includes a hidden layer, a convolution layer and a pooling layer; the relationship recognition module is also used to input the training code segment into the hidden layer, process the training code segment through the hidden layer, and obtain the corresponding first intermediate feature; input the first intermediate feature into the convolution layer, perform convolution processing on the first intermediate feature through the convolution layer, and obtain the corresponding second intermediate feature; input the second intermediate feature into the pooling layer, and perform pooling processing on the second intermediate feature through the pooling layer; perform entity relationship classification according to the pooling processing result to obtain a relationship recognition result.
[0209] In one embodiment, the above-mentioned device also includes: a knowledge base construction module, which is used to obtain the text to be processed, encode the text to be processed through a target encoding network, and obtain a coding sequence corresponding to the text to be processed; identify the entities of the text to be processed according to the coding sequence corresponding to the text to be processed and the target entity recognition network, and obtain the entity recognition result corresponding to the text to be processed; identify the entity relationship of the text to be processed according to the entity recognition result and the target relationship classification network, and obtain the relationship recognition result corresponding to the text to be processed; and construct a knowledge base according to the entity recognition result and the relationship recognition result corresponding to the text to be processed.
[0210] The above-mentioned text processing device encodes the training text through the encoding network to obtain the training encoding sequence corresponding to the training text, identifies the entities of the training text according to the training encoding sequence and the entity recognition network to obtain the entity recognition result, identifies the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain the relationship recognition result, and jointly trains the encoding network, the entity recognition network and the relationship classification network based on the entity recognition result, the relationship recognition result and the label data to obtain the updated encoding network, the entity recognition network and the relationship classification network. Due to the joint training, the entity recognition network and the relationship classification network can share the network parameters of the encoding network, so that the dependence between the entity recognition network and the relationship classification network can be realized during the training process, so that the obtained entity recognition network and the relationship classification network are more accurate. Further, by performing multiple rounds of iterative training on the encoding network, the entity recognition network and the relationship classification network, and in each iteration process, the training encoding sequence is updated according to the updated encoding network and the relationship recognition result, when the iteration stop condition is met, the trained target encoding network, the target entity recognition network and the target relationship classification network are obtained, so that the correlation between the entity recognition network and the relationship classification network can be fully considered, and the accuracy of the entity recognition network and the relationship classification network is further improved.
[0211] In one embodiment, Fig.12 As shown, a knowledge base construction device 1200 is provided. The device can adopt a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes:
[0212] The text acquisition module 1202 is used to acquire the text to be processed, encode the text to be processed through the trained target encoding network, and obtain the encoding sequence corresponding to the text to be processed;
[0213] An entity recognition module 1204 is used to recognize entities in the text to be processed according to the encoding sequence and the trained target entity recognition network to obtain entity recognition results;
[0214] The relationship identification module 1206 is used to identify the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain the relationship recognition result; the target encoding network, the target entity recognition network and the target relationship classification network are obtained by iterative joint training based on the training text; in each iterative training, the training encoding sequence input to the entity recognition network obtained by the previous training is updated according to the relationship recognition result output by the relationship classification network obtained by the previous training;
[0215] The knowledge base construction module 1208 is used to construct a knowledge base according to the entity recognition results and the relationship recognition results.
[0216] In one embodiment, the text to be processed is a drug instruction manual text; the knowledge base is a drug knowledge base; the relationship identification module is also used to determine at least two predicted entities corresponding to the text to be processed according to the entity recognition result; determine the name of the target drug from the at least two predicted entities, and form a predicted entity pair with each other predicted entity; determine the coding segment corresponding to each predicted entity pair from the coding sequence; input each coding segment into the target relationship classification network for relationship identification, and obtain the relationship identification result corresponding to each predicted entity pair.
[0217] In one embodiment, the above-mentioned device also includes: a prescription review module, which is used to obtain a prescription to be reviewed, which includes a drug name; query the corresponding target entity and target relationship from the drug knowledge base according to the drug name; review the prescription to be reviewed based on the queried target entity and target relationship to obtain a review result.
[0218] The above-mentioned knowledge base construction device, after obtaining the text to be processed, encodes the text to be processed through the trained target encoding network to obtain the encoding sequence corresponding to the text to be processed, identifies the entities of the text to be processed according to the encoding sequence and the trained target entity recognition network to obtain the entity recognition result, identifies the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain the relationship recognition result, and constructs the knowledge base according to the entity recognition result and the relationship recognition result. Since the target encoding network, the target entity recognition network and the target relationship classification network are obtained by iterative joint training based on the training text, in each iterative training, the training encoding sequence input to the entity recognition network obtained by the previous training is updated according to the relationship recognition result output by the relationship classification network obtained by the previous training. During the training process, the dependency and correlation of the entity recognition network and the relationship classification network are fully considered. The obtained entity recognition network and the target relationship classification network have good generalization performance and can accurately identify the entities and relationships in the text to be processed, so that the knowledge base constructed according to the entity recognition results and the relationship recognition results is significantly improved in accuracy compared with the knowledge base established by the manual annotation method in the related technology.
[0219] For the specific definition of the text processing device and the knowledge base construction device, please refer to the definition of the text processing method and the knowledge base construction method in the above text, which will not be repeated here. Each module in the above-mentioned text processing device and the knowledge base construction device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0220] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a text processing method or a knowledge base construction method is implemented.
[0221] Those skilled in the art will understand that Fig.13The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0222] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0223] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0224] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0225] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0226] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A text processing method, characterized in that: The method comprises: Acquire a training text and label data corresponding to the training text; the label data includes an entity label corresponding to the training text and a relationship label corresponding to the training text; Encoding the training text through a coding network to obtain a training coding sequence corresponding to the training text; Identify entities of the training text according to the training coding sequence and the entity recognition network to obtain entity recognition results; Identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain a relationship recognition result; Determine a first loss function value according to the difference between the entity recognition result and the entity label, determine a second loss function value according to the difference between the relationship recognition result and the relationship label, determine a first descent gradient according to the first loss function value, determine a second descent gradient according to the second loss function value, superimpose the first descent gradient and the second descent gradient to obtain a comprehensive descent gradient, and jointly train the encoding network, the entity recognition network, and the relationship classification network according to the comprehensive descent gradient to update the encoding network, the entity recognition network, and the relationship classification network; Inputting the training text into the updated encoding network, encoding the training text through the updated encoding network to obtain an encoding result, and fusing the relationship recognition result with the encoding result to update the training encoding sequence; Return to the step of identifying entities of the training text according to the training coding sequence and the entity recognition network to perform iterative training. When the iteration stop condition is met, the trained target coding network, target entity recognition network and target relationship classification network are obtained; wherein the target coding network, target entity recognition network and target relationship classification network are used to cooperate in identifying entity relationships of the text to be processed.
2. The method according to claim 1, characterized in that Before obtaining the training text and the label data corresponding to the training text, the method further includes: Performing word segmentation processing on the training text to obtain a corresponding text element set; Matching the text element set with an entity dictionary, and determining at least two first entity tags corresponding to the training text according to the matching result; the entity dictionary is obtained according to a pre-constructed knowledge graph; Determine the entity relationship corresponding to the at least two first entity tags according to the knowledge graph, and obtain the first relationship tag corresponding to the training text.
3. The method according to claim 2, characterized in that Before acquiring the training text, the method further includes: Determine a candidate text element from the text element set, the candidate text element being a text element other than the at least two first entity tags; Calculating the word coagulation degree of the candidate text element; When the calculated word coagulation degree exceeds a preset threshold, determining the candidate text element as a second entity tag corresponding to the training text; The entity relationship corresponding to the second entity tag is obtained, and the second relationship tag corresponding to the training text is obtained.
4. The method according to claim 1, characterized in that: The method further comprises: Obtaining candidate texts, encoding the candidate texts through an encoding network, and obtaining encoding sequences corresponding to the candidate texts; Identify the entity of the candidate text according to the encoding sequence corresponding to the candidate text and the entity recognition network, and obtain the entity recognition result corresponding to the candidate text; Identify the entity relationship of the candidate text according to the entity recognition result corresponding to the candidate text and the relationship classification network, and obtain the relationship recognition result corresponding to the candidate text; When the candidate text is determined to be an uncertain sample according to at least one of the entity recognition result corresponding to the candidate text and the relationship recognition result corresponding to the candidate text, the candidate text is sent to a preset terminal; When the entity label and the relationship label corresponding to the candidate text returned by the preset terminal are received, the target encoding network, the target entity recognition network and the target relationship classification network are continuously trained according to the candidate text.
5. The method according to claim 1, characterized in that The encoding network includes a feature mapping layer and an encoding layer; the training text is composed of a plurality of ordered text elements; encoding the training text through the encoding network to obtain a training encoding sequence corresponding to the training text includes: Perform feature mapping on each of the text elements through a feature mapping layer to obtain feature representations corresponding to each of the text elements; Input each of the feature representations into the encoding layer to obtain the element encoding features corresponding to each of the text elements; A training coding sequence corresponding to the training text is obtained according to the element coding features corresponding to each of the text elements.
6. The method according to claim 5, characterized in that The step of inputting each of the feature representations into the encoding layer to obtain the element encoding features corresponding to each of the text elements comprises: According to the order of the text elements corresponding to the feature representations, encoding the feature representations in a forward direction to obtain the forward encoding features corresponding to the text elements; According to the order of the text elements corresponding to the feature representations, encoding the feature representations in a backward direction to obtain the backward encoding features corresponding to the text elements; The forward coding features and the backward coding features corresponding to each of the text elements are respectively fused to obtain the element coding features corresponding to each of the text elements.
7. The method according to claim 5, characterized in that The entity recognition network includes a decoding layer and a classification layer; the entity recognition result obtained by identifying the entity of the training text according to the training coding sequence and the entity recognition network includes: At the decoding layer, according to the order of the text elements corresponding to the element encoding features, the element encoding features are decoded to obtain element decoding features corresponding to the text elements; At the classification layer, entity classification processing is performed on each element decoding feature to obtain an entity recognition result.
8. The method according to claim 1, characterized in that The identifying the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain the relationship recognition result includes: Determine at least two training entities corresponding to the training text according to the entity recognition result; Determining a corresponding training entity pair according to the at least two training entities; Determine the training coding segment corresponding to the training entity pair from the training coding sequence; The training coding segment is input into a relation classification network for relation recognition, and a relation recognition result corresponding to the training entity pair is obtained.
9. The method according to claim 8, characterized in that The relationship classification network includes a hidden layer, a convolutional layer and a pooling layer; the inputting the training coding segment into the relationship classification network for relationship recognition, and obtaining the relationship recognition result corresponding to the training entity pair includes: Inputting the training code segment into the hidden layer, and processing the training code segment through the hidden layer to obtain a corresponding first intermediate feature; Inputting the first intermediate feature into the convolution layer, and performing convolution processing on the first intermediate feature through the convolution layer to obtain a corresponding second intermediate feature; Inputting the second intermediate feature into the pooling layer, and performing pooling processing on the second intermediate feature through the pooling layer; Entity relationships are classified according to the pooling processing results to obtain relationship recognition results.
10. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Acquire a text to be processed, encode the text to be processed through the target encoding network, and obtain a coding sequence corresponding to the text to be processed; Identify the entity of the text to be processed according to the encoding sequence corresponding to the text to be processed and the target entity recognition network, and obtain the entity recognition result corresponding to the text to be processed; Identify the entity relationship of the text to be processed according to the entity recognition result and the target relationship classification network, and obtain the relationship recognition result corresponding to the text to be processed; A knowledge base is constructed according to the entity recognition results and relationship recognition results corresponding to the text to be processed.
11. A method for constructing a knowledge base, characterized in that: The method comprises: Obtaining a text to be processed, encoding the text to be processed through a trained target encoding network to obtain an encoding sequence corresponding to the text to be processed; Identify the entities of the text to be processed according to the encoding sequence and the trained target entity recognition network to obtain an entity recognition result; Identify the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain a relationship recognition result; The target encoding network, the target entity recognition network and the target relationship classification network are obtained by using the text processing method according to any one of claims 1 to 10; A knowledge base is constructed according to the entity recognition results and the relationship recognition results.
12. The method according to claim 11, characterized in that The text to be processed is a drug instruction manual text; the knowledge base is a drug knowledge base; the entity relationship of the text to be processed is identified according to the entity recognition result and the trained target relationship classification network, and the relationship recognition result obtained includes: Determine at least two predicted entities corresponding to the text to be processed according to the entity recognition result; Determine a target drug name from the at least two prediction entities, and combine the target drug name with each other prediction entity to form a prediction entity pair; Determine, from the coding sequence, coding segments corresponding to each of the predicted entity pairs; Each of the coding segments is input into the target relationship classification network for relationship recognition, and the relationship recognition results corresponding to each of the predicted entities are obtained.
13. The method according to claim 12, characterized in that The method further comprises: Obtaining a prescription to be reviewed, wherein the prescription to be reviewed includes the name of the drug; According to the drug name, query the drug knowledge base for the corresponding target entity and target relationship; The prescription to be reviewed is reviewed according to the queried target entity and target relationship to obtain a review result.
14. A text processing device, characterized in that: The device comprises: A data acquisition module, used to acquire training text and label data corresponding to the training text; the label data includes entity labels corresponding to the training text and relationship labels corresponding to the training text; An encoding module, used for encoding the training text through an encoding network to obtain a training encoding sequence corresponding to the training text; An entity recognition module, used to recognize entities of the training text according to the training coding sequence and the entity recognition network, and obtain entity recognition results; A relationship recognition module, used to identify the entity relationship of the training text according to the entity recognition result and the relationship classification network to obtain a relationship recognition result; A joint training module, used to determine a first loss function value according to the difference between the entity recognition result and the entity label, determine a second loss function value according to the difference between the relationship recognition result and the relationship label, determine a first descent gradient according to the first loss function value, determine a second descent gradient according to the second loss function value, superimpose the first descent gradient and the second descent gradient to obtain a comprehensive descent gradient, and jointly train the encoding network, the entity recognition network and the relationship classification network according to the comprehensive descent gradient to update the encoding network, the entity recognition network and the relationship classification network; An iterative training module is used to input the training text into an updated encoding network, encode the training text through the updated encoding network to obtain an encoding result, and merge the relationship recognition result with the encoding result to update the training encoding sequence, and return to the step of identifying the entities of the training text according to the training encoding sequence and the entity recognition network to perform iterative training. When the iteration stop condition is met, the trained target encoding network, target entity recognition network and target relationship classification network are obtained; wherein the target encoding network, target entity recognition network and target relationship classification network are used to cooperate in identifying the entity relationship of the text to be processed.
15. The device according to claim 14, characterized in that The device also includes: a labeling module, which is used to: perform word segmentation processing on the training text to obtain a corresponding text element set; match the text element set with an entity dictionary, and determine at least two first entity tags corresponding to the training text according to the matching result; the entity dictionary is obtained according to a pre-constructed knowledge graph; determine the entity relationship corresponding to the at least two first entity tags according to the knowledge graph, and obtain the first relationship tag corresponding to the training text.
16. The device according to claim 15, characterized in that The annotation module is also used to: determine a candidate text element from the text element set, the candidate text element being a text element other than the at least two first entity tags; calculate the word coagulation degree of the candidate text element; when the calculated word coagulation degree exceeds a preset threshold, determine the candidate text element as the second entity tag corresponding to the training text; obtain the entity relationship corresponding to the second entity tag, and obtain the second relationship tag corresponding to the training text.
17. The device according to claim 14, characterized in that The device also includes: an active learning module, which is used to: obtain candidate text, encode the candidate text through an encoding network, and obtain a coding sequence corresponding to the candidate text; identify the entity of the candidate text according to the coding sequence corresponding to the candidate text and an entity recognition network, and obtain an entity recognition result corresponding to the candidate text; identify the entity relationship of the candidate text according to the entity recognition result corresponding to the candidate text and a relationship classification network, and obtain a relationship recognition result corresponding to the candidate text; when the candidate text is determined to be an uncertain sample according to at least one of the entity recognition result corresponding to the candidate text and the relationship recognition result corresponding to the candidate text, the candidate text is sent to a preset terminal; when the entity label and relationship label corresponding to the candidate text returned by the preset terminal are received, the target encoding network, the target entity recognition network, and the target relationship classification network are continued to be trained according to the candidate text.
18. The device according to claim 14, characterized in that The encoding network includes a feature mapping layer and an encoding layer; the training text is composed of multiple ordered text elements; the encoding module is also used to: perform feature mapping on each of the text elements through the feature mapping layer to obtain the feature representation corresponding to each of the text elements; input each of the feature representations into the encoding layer to obtain the element encoding features corresponding to each of the text elements; and obtain the training encoding sequence corresponding to the training text according to the element encoding features corresponding to each of the text elements.
19. The device according to claim 18, characterized in that The encoding module is also used to: encode each feature representation in a forward direction according to the sequence of text elements corresponding to each feature representation, so as to obtain forward encoding features corresponding to each text element; encode each feature representation in a backward direction according to the sequence of text elements corresponding to each feature representation, so as to obtain backward encoding features corresponding to each text element; and fuse the forward encoding features and backward encoding features corresponding to each text element respectively, so as to obtain element encoding features corresponding to each text element.
20. The device according to claim 18, characterized in that The entity recognition network includes a decoding layer and a classification layer; the entity recognition module is also used to: at the decoding layer, decode the element encoding features according to the order of the text elements corresponding to each element encoding feature to obtain the element decoding features corresponding to each of the text elements; at the classification layer, perform entity classification processing on each of the element decoding features to obtain the entity recognition result.
21. The device according to claim 14, characterized in that The entity recognition module is further used to: determine at least two training entities corresponding to the training text according to the entity recognition result; determine a corresponding training entity pair according to the at least two training entities; and determine a training coding segment corresponding to the training entity pair from the training coding sequence; The relationship identification module is also used to input the training coding segment into the relationship classification network to perform relationship identification, and obtain the relationship identification results corresponding to the training entity pairs.
22. The device according to claim 21, characterized in that The relationship classification network includes a hidden layer, a convolution layer and a pooling layer; the relationship identification module is further used to: input the training code segment into the hidden layer, process the training code segment through the hidden layer, and obtain a corresponding first intermediate feature; input the first intermediate feature into the convolution layer, and perform convolution processing on the first intermediate feature through the convolution layer to obtain a corresponding second intermediate feature; The second intermediate feature is input into the pooling layer, and the second intermediate feature is pooled by the pooling layer; entity relationship classification is performed according to the pooling processing result to obtain a relationship recognition result.
23. The device according to any one of claims 14 to 21, characterized in that The device also includes: a knowledge base construction module, which is used to obtain a text to be processed, encode the text to be processed through the target encoding network, and obtain a coding sequence corresponding to the text to be processed; identify the entity of the text to be processed according to the coding sequence corresponding to the text to be processed and the target entity recognition network, and obtain an entity recognition result corresponding to the text to be processed; identify the entity relationship of the text to be processed according to the entity recognition result and the target relationship classification network, and obtain a relationship recognition result corresponding to the text to be processed; and construct a knowledge base according to the entity recognition result and the relationship recognition result corresponding to the text to be processed.
24. A device for constructing a knowledge base, characterized in that: The device comprises: A text acquisition module is used to acquire the text to be processed, encode the text to be processed through the trained target encoding network, and obtain the encoding sequence corresponding to the text to be processed; An entity recognition module is used to recognize entities of the text to be processed according to the coding sequence and the trained target entity recognition network to obtain an entity recognition result; A relationship recognition module, used for identifying the entity relationship of the text to be processed according to the entity recognition result and the trained target relationship classification network to obtain a relationship recognition result; the target encoding network, the target entity recognition network and the target relationship classification network are obtained by using the text processing method according to any one of claims 1 to 10; The knowledge base construction module is used to construct a knowledge base according to the entity recognition results and the relationship recognition results.
25. The device according to claim 24, characterized in that The text to be processed is a drug instruction manual text; the knowledge base is a drug knowledge base; the relationship identification module is also used to: determine at least two predicted entities corresponding to the text to be processed according to the entity recognition result; determine the target drug name from the at least two predicted entities, and form a predicted entity pair with the target drug name and other predicted entities respectively; determine the coding segment corresponding to each of the predicted entity pairs from the coding sequence; input each of the coding segments into the target relationship classification network for relationship identification, and obtain the relationship identification result corresponding to each of the predicted entity pairs.
26. The device according to claim 25, characterized in that The device also includes: a prescription review module, which is used to obtain a prescription to be reviewed, wherein the prescription to be reviewed includes a drug name; query the corresponding target entity and target relationship from the drug knowledge base according to the drug name; and review the prescription to be reviewed based on the queried target entity and target relationship to obtain a review result.
27. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.
28. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.
