A structured information extraction method for medical instruction manual text

Abstract

The present invention discloses a structured information extraction method for medical instruction text, and belongs to the technical field of medical natural language processing. First, an entity recognition model based on adversarial transfer learning for unlabeled data is used to perform preliminary entity recognition of the medical instruction text. Then, based on entity recognition, a new task of extracting usage and dosage tables is proposed, that is, the complex knowledge structure in the usage and dosage text is extracted in the form of a table. And for this task, a table extraction algorithm based on feature fusion is designed to extract the usage and dosage table of the instruction manual. Finally, based on entity recognition, a relationship extraction method based on the medical entity character attention mechanism is adopted to extract the relationship between entities. The structured information extraction method proposed in the present invention can solve the problem of complex information extraction of medical instruction text.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a structured information extraction method for Chinese medical texts. Background Art

[0002] Medical instructions serve as a guide for users to select and use drugs. They contain a wealth of information such as entities and relationships. Extracting this structured information is of great significance to medical research and clinical diagnosis. Entity recognition and relationship extraction technology can extract structured information from unstructured text and is the basis of medical text processing tasks.

[0003] The first step in extracting structured information for medical instructions is to perform entity recognition. The current mainstream entity relationship recognition method based on deep learning requires a large amount of annotated corpus for training. However, in the Chinese medical field, especially in medical instructions, the annotated data is far less than the unlabeled data. In addition, due to the particularity of medical data, the annotation of medical data requires the guidance of experts in specific fields, and a large number of people are needed to annotate, which requires a lot of manpower, material resources and financial resources. Obviously, the cost is expensive. Therefore, the information extraction task of medical texts is subject to certain limitations, and relevant methods are needed to solve the problem of insufficient annotated data.

[0004] The usage and dosage text of medical instructions has a complex knowledge structure. The same drug has different usages for different diseases. Even for the same disease, there are different dosages at different times. Traditional entities and relationships cannot accurately define this structure. How to extract the usage and dosage from the instructions in a structured form based on entity recognition so that the extraction results can vividly describe this knowledge structure, that is, different conditions correspond to different usage and dosage, is a new task and challenge.

[0005] After the entities are extracted from the instruction manual, there may be rich relationships between the entities, such as "applicable" and "treatment", as well as "contraindications" and "not applicable". The latter may be more important for the user's medication guidance. However, because these relationships are relatively few, they are often not accurately identified, that is, there is a "long-tail distribution" problem. How to accurately identify these relationships is also a problem that needs to be solved. Summary of the invention

[0006] The purpose of the present invention is to provide a method for extracting structured information from medical instruction manuals in view of the deficiencies of the prior art.

[0007] The object of the present invention is achieved through the following technical solution: a method for extracting structured information from a medical instruction manual, comprising the following steps:

[0008] (1) Named entity recognition: For unlabeled instruction manual text, adversarial transfer learning technology is used to perform preliminary named entity recognition;

[0009] (2) Extraction of usage and dosage table from the instruction manual: Using the features contained in the instruction manual text, a usage and dosage table extraction algorithm based on feature fusion is used to obtain structured usage and dosage table information;

[0010] (3) Medical relationship extraction: We use the fact that the characters that make up medical entities can express rich semantic information and combine it with the attention mechanism to extract the relationships between entities, ultimately obtaining structured medical entity relationship triplet information.

[0011] Further: the step (1) comprises the following sub-steps:

[0012] (1.1) The input is the text of the source domain and the target domain. Character embedding is first performed, and the BERT pre-trained model is used to obtain context-based word vector representation;

[0013] (1.2) The extracted feature vector x is used as the input of the next feature extraction layer, which uses BiLSTM to extract features from the input vector, where x S ∈S and x T ∈T represent the input text from the source domain and the target domain respectively. Based on LSTM, the BiLSTM network adopts a bidirectional structure to extract sequence features from front to back and from back to front respectively, so as to better extract the features of this method from the perspective of temporal features. Therefore, it is more suitable for text processing tasks. The state h of the hidden layer i It can be expressed as follows:

[0014]

[0015] in, and They represent the forward and backward hidden states of the i-th unit in the LSTM network, respectively, and + represents a connection operation.

[0016] (1.3) The feature extraction results of BiLSTM are input into the adversarial training layer. This layer first uses the source domain data to train an entity classifier. The classifier is a feedforward neural network followed by a conditional random field (CRF) classification model. The extracted data and features are used for entity recognition. The loss function is defined as follows:

[0017]

[0018] Next, we train a domain discriminator, which is a feedforward neural network followed by a Softmax model. Its loss function is defined as follows:

[0019]

[0020] Through training, the model is ultimately unable to distinguish between the source domain and the target domain, making the feature extractor domain-invariant, and the model trained in the source domain can be applied to the target domain.

[0021] (1.4) Through the above steps, we finally get a model with good entity recognition effect in the field of medical instructions.

[0022] Further: step (2) comprises the following sub-steps:

[0023] (2.1) Usage table extraction is a new task proposed by this invention. The formal definition of this task is as follows:

[0024]

[0025] The input of table extraction is the text T and the entity set E contained in the text, and the output is table = {t1, t2, ...t k …,t n}, where t k Represents a row of content in a table, and t k ={condition, dosage} indicates the usage and dosage of the drug under a certain condition (condition).

[0026] (2.2) Based on the new table extraction task, a table extraction method based on feature fusion is adopted, which includes three parts: data preprocessing, feature extraction and group matching.

[0027] (2.3) Before table extraction, the text needs to be preprocessed, mainly including location labeling, data cleaning, medical entity labeling and reference resolution.

[0028] (2.4) After completing data preprocessing, feature extraction of the instruction manual text begins. Three main features are extracted, namely entity features, clue word features, and context features.

[0029] (2.5) The entities are grouped according to the extracted text features, and each group is output as a row in a table.

[0030] Further: The relation extraction algorithm used in step (3) mainly includes the following steps:

[0031] (3.1) The input is the sentence of the relation to be extracted and the corresponding head entity and tail entity. The first step is to map the input into a low-dimensional vector representation. Assume that the given input sentence consists of n characters, that is, S = {s1, s2, …, s n}, the mapped vector is represented by R = {r1, r2, …, r n}. Assume that the entity contains m characters, that is, E = {c1, c2, ..., c m}, where each symbol c i It is mapped to the eigenvector e through the mapping matrix M. i , the entity is finally represented as a vector E = {e1,e2,…,e m}.

[0032] (3.2) After completing the vector mapping, the next step is to encode the features of the input. When encoding a sentence, the input sentence is R = {r1, r2, ..., r n}, then the hidden state H of the forward LSTM f It is expressed as:

[0033]

[0034] The encoding result H of the reverse LSTM b It is expressed as:

[0035]

[0036] The final encoded sentence hidden state is H s :

[0037] H s ={h1,h2,…,h n}

[0038] in,

[0039] Given an entity input E = {e1, e2, ..., e m}, after BiLSTM is used to encode the entity, the hidden state of the final entity is expressed as:

[0040] H e =BLSTM{e1,e2,…,e m}

[0041] Then the average pooling result of the hidden layer state is used as the final output form of the entity after encoding. The final encoding vectors of the head and tail entities are and

[0042] (3.3) After encoding the features of sentences and entities, the attention mechanism is then used to fully utilize the semantic information provided by the characters that make up the entity. The entity encoding vector and the sentence encoding vector are calculated with attention, so as to select the part that is most relevant to the given input query as the input of the next classification layer. Finally, the relationship between entities is extracted to obtain entity relationship triples.

[0043] The beneficial effects of the present invention are that the structured information extraction method proposed in the present invention can firstly effectively solve the problem of insufficient annotated data in the field of medical instructions, and make full use of the existing relevant annotated data to realize entity extraction in the field of instructions; and the newly proposed table extraction, as a new information extraction task, can solve the problem of knowledge extraction with complex hierarchical structure of the usage and dosage text of the instructions; finally, the proposed relationship extraction algorithm based on the attention mechanism makes full use of the characteristic that the characters constituting the medical entity can express rich semantic information, and even for a small number of relationships, it can be accurately identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a structured information extraction method for medical instruction manual text proposed by the present invention;

[0045] Figure 2 This is an entity recognition model diagram based on adversarial transfer learning;

[0046] Figure 3 This is a flowchart of table extraction based on feature fusion;

[0047] Figure 4 It is a flowchart of relation extraction based on character attention mechanism. DETAILED DESCRIPTION

[0048] The present invention proposes a structured information extraction method for medical instructions text. The extraction process framework is as shown in the attached figure. Figure 1 As shown, this method can extract structured table information and entity relationship triple information from unstructured text, and specifically includes the following steps:

[0049] 1. Named Entity Recognition: To address the problem of insufficient annotated data in the field of medical instructions, an entity recognition model based on adversarial transfer learning is proposed. This model makes full use of the existing annotated data in related fields and obtains a model with good entity recognition effect in the field of medical instructions through transfer learning based on adversarial training. It includes the following sub-steps:

[0050] 1.1 As attached Figure 2As shown in the figure, the first step of named entity recognition is to embed the input characters to obtain a word vector representation based on the context. The BERT pre-training task is a multi-task model, including the Masked Language Model (MLM) task and the Next Sentence Prediction (NSP) task. The former refers to randomly masking some words during the training process and then predicting the word through the context in order to better train the bidirectional representation of the text. During the training of the latter, for two real sentences, a sentence is randomly selected with a probability of 50% to replace the second sentence, and then it is determined whether the second sentence is the context of the first sentence, so as to better learn the semantic connection between the two sentences. BERT can solve the problem that the same word expresses different semantics in different contexts. Therefore, this method converts the input sentence into a feature vector through BERT, and then uses it as the input of the next layer feature extractor BiLSTM.

[0051] 1.2 Then use the feature vector x extracted by BERT as the input of the next bidirectional LSTM, where x S ∈S and x T ∈T represent the input text from the source domain and the target domain respectively. The unidirectional LSTM network only uses past information in transmission and ignores future information. In text processing tasks, both contextual information is important, so this method uses bidirectional LSTM for feature extraction. Based on LSTM, the BiLSTM network adopts a bidirectional structure to extract sequence features from front to back and from back to front, respectively, so as to better extract features from the perspective of temporal features, and is therefore more suitable for text processing tasks. Due to the excellent feature extraction capability of BiLSTM, this method uses bidirectional LSTM to extract sequence features and contextual representation information in sentences after vectorizing the input text, and inputs the results into the entity classifier, domain discriminator, and target domain autoencoder of the adversarial training layer.

[0052] 1.3 The adversarial training layer mainly includes entity classifier, domain discriminator and target domain autoencoder. The input of entity classifier is the output of feature extractor. The classifier is a feedforward neural network followed by conditional random field (CRF) classification model. It performs entity recognition on the extracted data and features. The use of CRF for entity recognition takes semantic context into consideration, making the network more suitable for named entity recognition tasks. The entity classifier acquires entity classification ability by learning to predict entity labels during training. The domain discriminator is a feedforward neural network followed by a Softmax model. It performs domain identification on the extracted features and data, that is, it identifies whether the input data is from the source domain or the target domain. After training, the domain discriminator cannot distinguish between the source domain and the target domain, making the feature extractor domain invariant. The target domain autoencoder enables the feature extractor to extract domain-specific features that are helpful for entity recognition. Because only extracting common features of the two domains will ignore the unique features of the target domain, thereby limiting the classification ability of the target domain entity classifier, the target domain characteristic encoder is used to reconstruct the target domain data so that the extracted features are domain-unique.

[0053] 1.4 Through the above steps, we finally get a model with good entity recognition effect in the field of medical instructions.

[0054] 2. Extraction of usage and dosage table in instructions: A table extraction task based on the usage and dosage text of medical instructions is proposed, that is, the complex knowledge structure in the usage and dosage text is extracted in the form of a table. Each row in the table is expressed in the form of Condition-Dosage, where Condition represents the condition and Dosage represents the usage and dosage under the condition. For this task, a table extraction method based on feature fusion is designed to make full use of the rich clue words and other features contained in the text, so as to realize the extraction of the usage and dosage table in the instructions. It includes the following sub-steps:

[0055] 2.1 As attached Figure 3 As shown, the table extraction process includes three parts: data preprocessing, feature extraction, and group matching;

[0056] 2.2 Data preprocessing mainly includes location labeling, data cleaning, medical entity labeling and reference resolution.

[0057] Position annotation: The entity recognition result includes entity id, entity type, entity name and entity position. The entity position indicates the distance from the start position of the text to the first character of the entity. According to the entity position and the number of characters contained in the entity, the start and end positions of the entity in the text are determined, and then the entity id is used to mark the entity at the start and end positions. The position-annotated instruction text has marked all the entities that have been initially identified in the text, which serves as the basis for the medical entity annotation. Because the instruction text crawled from the website contains many irrelevant characters and HTML tags, which increases the difficulty of the subsequent extraction task, data cleaning is performed next.

[0058] Data cleaning: The manual text directly crawled from the web contains many irrelevant characters and tags. Data cleaning needs to remove these HTML tags. However, some tags can express rich semantic information. This method uses special symbols to replace these tags, such as In Hypertext Markup Language, it indicates a line break, generally indicating the end of a usage and quantity description, so sentence separators are used instead. , <strong>< / strong> Indicates emphasis. In the text of the instructions, the main purpose is to emphasize the diseases for which the drug is suitable, so the disease type label is used instead of <strong>Etc. For these tags that can represent special semantic information, this method first traverses the text to replace special symbols during data cleaning, and then uses regular matching to delete HTML tags and irrelevant characters in pairs.

[0059] Medical entity labeling: After location labeling and data cleaning, the next step is to label the entities at each location, that is, to give each medical entity a semantic label according to its type, which serves as the basis for subsequent feature extraction.

[0060] Reference resolution: Nouns such as "this product" and "as above" are common in the instructions. For example, "this product is suitable for respiratory infections caused by sensitive bacteria" represents the main drug in the instructions. "For the treatment of unstable angina, the usage and dosage are as above, refer to acute myocardial infarction" does not give the specific usage and dosage for the treatment of angina pectoris, but refers to the treatment method of the previous disease. These references can be well understood by users, but cannot be understood by computers, so reference resolution is required before extraction. Reference resolution refers to replacing the pronouns appearing in the text with their corresponding words with specific meanings. This method replaces these demonstrative pronouns with their corresponding entities or labels, replacing "this product" with the drug entity and "this disease" with the symptom entity. For pronouns that appear in the text and whose reference objects are not entities, such as "as above", this method uses feature labels to mark them. When encountering such labels during the extraction process, the previous group of extracted content is used as the usage and dosage of this group.

[0061] 2.3 After completing data preprocessing, feature extraction of the instruction manual text begins. This method mainly extracts three types of features: entity features, clue word features, and context features.

[0062] Entity features: various types of entity type labels. This method has been labeled in data preprocessing. Given that each entity type, including the method of use, dosage, frequency of use, and interval of use, can only appear once in each row of the table, the number of occurrences of each entity is calculated as the entity quantity feature.

[0063] Clue word features: The instruction manual contains many clue words such as <start>and <end>Respectively represent the start and end partition numbers of table extraction <se>Indicates the start or end of the extraction of a row of content. During the extraction process, this method mainly uses these clue words as the primary basis to divide different rows of the table.

[0064] Contextual features: According to the language structure characteristics of the instruction manual text, each usage and dosage content has similar contextual expressions. When the {MEDGFRIN} statement appears, that is, the four entity types in dosage appear simultaneously and continuously, it can be considered that the dosage has been extracted and a row of the table has been extracted. When the three entity types {DNSTPA} in condition appear completely, it can also be considered that the extraction of the condition is completed. Therefore, the labels appearing in the record statement are used as contextual features as the basis for the subsequent grouping.

[0065] (2.4) According to the extracted text features, the entities are grouped and each group is output as a row of the table. The group matching algorithm is shown in Algorithm 1 below.

[0066] Algorithm 1. Usage table extraction group matching detailed algorithm

[0067]

[0068] 3. Medical relationship extraction: A relationship extraction method based on the medical entity character attention mechanism is proposed. This method makes full use of the fact that the characters that make up the medical entity can express rich semantic information, and combines the attention mechanism to extract the relationship between entities. The specific contents are as follows:

[0069] The relationship extraction flow chart is as follows Figure 4 As shown, where r i represents the i-th character vector of each sentence after the sentence is vectorized. and Denote the embedding vectors corresponding to the i-th character input in the head entity and the tail entity, q and r respectively. e The feature encoding vectors of the head and tail entities are and Calculated in different ways, the weight parameter α i It is calculated by the attention function from q and the hidden state h.

[0070] 3.1 The embedding layer vectorizes the input sentences and entities and maps the input into a low-dimensional vector representation. When encoding a sentence, the characters that make up the sentence must first be converted into their corresponding vectors. The distance of each character relative to the entity is also an important feature, so the distance is also converted into a corresponding vector through a pre-trained mapping matrix. When encoding an entity, the characters that make up the entity are mapped to their corresponding vectors. Assume that a given input sentence consists of n characters, that is, S = {s1, s2, …, s n}, the input sentence is finally mapped to a vector representation of R = {r1, r2, …, r n The vector mapping of each entity is similar to that of a sentence. Each character that makes up the entity is mapped to a corresponding vector representation through a vector matrix. Assuming that the entity contains m characters, that is, E = {c1, c2, …, c m }, where each symbol c i It is mapped to the eigenvector e through the mapping matrix M. i , the entity is finally represented as a vector E = {e1,e2,…,e m }.

[0071] 3.2 After comparison and analysis, BiLSTM is also used as the encoder to process the input information, because BiLSTM shows better performance in sentence feature extraction when given the character composition information of the entity. Secondly, compared with the CNN-based model, the LSTM-based model has clearer semantics when combined with the next step of the attention mechanism. Moreover, BiLSTM has a simpler structure than other complex neural networks, which means it has fewer parameters and faster calculation speed. When encoding a sentence, the input sentence is R = {r1, r2, ..., r n }, then the hidden state H of the forward LSTM f for:

[0072]

[0073] The encoding result H of the reverse LSTM b for:

[0074]

[0075] The final encoded sentence hidden state is H s for:

[0076] H s ={h1,h2,…,h n }

[0077] in,

[0078] Given an entity input E = {e1, e2, ..., e m }, after BiLSTM is used to encode the entity, the hidden state of the final entity is expressed as:

[0079] H e =BLSTM{e1,e2,…,e m }

[0080] Then the average pooling result of the hidden layer state is used as the final output form of the entity after encoding. The final encoding vectors of the head and tail entities are and

[0081] 3.3 After encoding the features of sentences and entities, the attention mechanism is then used to make full use of the semantic information provided by the characters that make up the entity. The attention mechanism is widely used in many natural language processing tasks, such as question answering and machine translation, to select the most relevant parts of a given input query. In most current NLP tasks, the input state v is also used as the keyword k. This is also the case in the model of this method. The encoding output H of the input sentence is used in this model. s As k and v. The sentence after attention calculation is represented as r s , calculated as follows:

[0082] r s =αV

[0083] Where V is the hidden state H after LSTM encodes the input sentence s , α is the attention vector, α is calculated from the query q and keyword k, and k is also used in this method. s Indicates that, so the calculation process of α is as follows:

[0084] α=softmax(f attention (q,H s ))

[0085] Attention function f attention There are many ways to calculate , and multiplication is a frequently used method, so this method also uses this calculation method. In this case, α is expressed as:

[0086] α=softmax(qWH s )

[0087] where W is a parameter matrix.

[0088] In the relation extraction task, there is no query q. This method uses the encoding vectors of the head entity and the tail entity. and In the knowledge graph representation, it is assumed that given a triple (h, l, t), where h and t represent the head entity and the tail entity respectively, and l represents the relationship between the two entities, the embedded vector of the triple satisfies h + l = t, that is, the vector difference between the two entities can represent the relationship. Based on this assumption, q is expressed as:

[0089]

[0090] So the final output of the input sentence after the attention mechanism is:

[0091]

[0092] 3.4 The model then calculates the attention result r s and the joint vector representation of the head and tail entities r e Input to the fully connected layer for feature interaction, and then complete the relationship classification through the Softmax function, where r e The calculation method is as follows:

[0093]

[0094] The calculation formula of the Softmax layer is as follows:

[0095]

[0096] Among them, r s and r e Represent the feature vectors corresponding to the sentence and entity respectively, W and B represent the parameter matrix, is a probability vector, each dimension of which corresponds to the predicted probability of each label, Indicates the type of label corresponding to the maximum probability y.

[0097] The model uses cross entropy as the objective function, which is calculated as follows:

[0098]

[0099] Among them, m represents the total number of samples, y i Represents the label predicted by the Softmax function, is the corresponding true label, and θ represents the model parameters.< / se> < / end> < / start> < / strong>

Claims

1. A method for extracting structured information from medical instruction manuals, characterized in that: The following steps are involved: (1) Named entity recognition: For unlabeled instruction manual text, adversarial transfer learning technology is used to perform preliminary named entity recognition; The step (1) comprises the following sub-steps: (1.1) The input is the text of the source domain and the target domain. Character embedding is first performed, and the BERT pre-trained model is used to obtain context-based word vector representation; (1.2) The extracted feature vector x is used as the input of the next feature extraction layer, which uses BiLSTM to extract features from the input vector, where x S ∈S and x T ∈T represent the input text from the source domain and the target domain respectively; BiLSTM network adopts a bidirectional structure based on LSTM, extracting sequence features from front to back and from back to front respectively, and better extracting the features of this method from the perspective of temporal features, so it is more suitable for text processing tasks, and the state h of the hidden layer i It is expressed as follows: in, and Respectively represent the forward and backward hidden states of the i-th unit in the LSTM network, and + represents the connection operation; (1.3) The feature extraction results of BiLSTM are input into the adversarial training layer. This layer first uses the source domain data to train an entity classifier. The classifier is a feedforward neural network followed by a conditional random field (CRF) classification model. The extracted data and features are used for entity recognition. Then a domain discriminator is trained. The domain discriminator is a feedforward neural network followed by a Softmax model. Through training, the model is ultimately unable to distinguish between the source domain and the target domain, making the feature extractor domain-invariant. The model trained in the source domain is applied to the target domain. (1.4) Through the above steps, we finally get an entity recognition model in the field of medical instructions; (2) Extraction of usage and dosage table from the instruction manual: Using the features contained in the instruction manual text, a usage and dosage table extraction algorithm based on feature fusion is used to obtain structured usage and dosage table information; (3) Medical relationship extraction: The characters that make up medical entities express rich semantic information are used in combination with the attention mechanism to extract the relationship between entities, and finally obtain structured medical entity relationship triple information; The relation extraction algorithm used in step (3) includes the following steps: (3.1) The input is the sentence of the relation to be extracted and the corresponding head entity and tail entity. The first step is to map the input into a low-dimensional vector representation. Assume that the given input sentence consists of n characters, that is, S = {s1, s2, ..., s n }, the mapped vector is represented by R = {r1, r2, ..., r n }; Assume that the entity contains m characters, that is, E = {c1, c2, ...., c m }, where each symbol c i It is mapped to the eigenvector e through the mapping matrix M. i , the entity is finally represented as a vector E = {e1, e2, ...., e m }; (3.2) After completing the vector mapping, the next step is to encode the features of the input. When encoding a sentence, the input sentence is R = {r1, r2, ..., r n }, then the hidden state H of the forward LSTM f It is expressed as: The encoding result H of the reverse LSTM b It is expressed as: The final encoded sentence hidden state is H s : H s ={h1,h2,...,h n } in, Given an entity input E = {e1, e2, ...., e m }, after BiLSTM is used to encode the entity, the hidden state of the final entity is expressed as: H e =BLSTM{e1,e2,....,e m } Then the average pooling result of the hidden layer state is used as the final output form of the entity after encoding. The final encoding vectors of the head and tail entities are and (3.3) After encoding the features of sentences and entities, the attention mechanism is then used to fully utilize the semantic information provided by the characters that make up the entity. The entity encoding vector and the sentence encoding vector are paid attention to, so as to select the part that is most relevant to the given input query as the input of the next classification layer; finally, the relationship between the entities is extracted to obtain the entity relationship triple.

2. The method according to claim 1, characterized in that: Step (2) includes the following sub-steps: (2.1) The formal definition of the usage table extraction task is as follows: The input of table extraction is the text T and the entity set E contained in the text, and the output is table = {t1, t2, ...t k ..., t n }, where t k Represents a row of content in a table, and t k ={condition, dosage} represents the dosage of a drug under a certain condition; (2.2) Based on the new table extraction task, a table extraction method based on feature fusion is adopted, which includes three parts: data preprocessing, feature extraction and group matching; (2.3) Before table extraction, the text needs to be preprocessed, including location annotation, data cleaning, medical entity annotation, and reference resolution; (2.4) After completing data preprocessing, feature extraction of the instruction manual text begins. Three types of features are extracted: entity features, clue word features, and context features. (2.5) The entities are grouped according to the extracted text features, and each group is output as a row in a table.

Images