A method for constructing a framework for extracting epidemic case information based on a pre-trained language model

Abstract

The present invention proposes a method for constructing an epidemic case information extraction framework based on a pre-trained language model, and integrates a named entity recognition network and an implicit category prediction network into an epidemic case information extraction framework. The present invention designs a data labeling rule for infected cases, and proposes an information extraction framework based on a pre-trained language model to automatically extract core elements from cases, wherein the named entity recognition network can accurately identify named entities in case texts and locate key information on case transmission paths; the implicit case category prediction network can efficiently predict implicit categories and determine the main forms of case transmission paths; the framework realizes a structured representation of case texts to further assist disease prevention and control experts in formulating intervention measures for the spread of epidemic viruses.

Description

Technical Field

[0001] The present invention belongs to the technical field of text information extraction and relates to a method for constructing an epidemic case information extraction framework, and more specifically, to a method for constructing an epidemic case information extraction framework based on a pre-trained language model that integrates a named entity recognition network and an implication category prediction network. Background Art

[0002] Rapidly extracting effective information from massive amounts of public data can help control the spread of diseases and coordinate emergency responses. However, relying solely on domain experts to conduct real-time analysis of case texts is very inefficient and time-consuming, and cannot guarantee the timeliness of disease prevention and control and intervention measures.

[0003] With the rise of deep neural networks, artificial intelligence technology continues to advance. The introduction of the pre-trained language model BERT (Bidirectional Encoder Representation from Transformers) has further promoted the development of natural language processing technology. BERT learns general, contextual text representations through unsupervised training on a large amount of unlabeled text, and has achieved significant performance improvements in eleven natural language processing tasks such as question answering and machine translation. The application of natural language processing technology in information extraction tasks provides an automated recognition model for extracting key facts from case texts. Named entity recognition and text classification are the main technical means of information extraction. Deep learning models can learn parameters suitable for case information extraction from labeled data, thereby extracting structured facts from new cases.

[0004] Deep neural network is a data-driven feature learning model, and its performance depends greatly on the size and quality of data annotation. The cost of data annotation is extremely high for any field, especially for new research fields such as the spread of epidemic viruses. The programming and standardization of data annotation are particularly important. In addition, deep neural networks usually use an end-to-end processing method, which usually has high performance for a specific task, but the model is not flexible enough and has poor scalability, so it cannot fully adapt to multi-faceted case information scenarios. Summary of the invention

[0005] In order to overcome the deficiencies in the prior art, the present invention designs a data annotation rule for the particularity of virus infection cases, defines 11 fine-grained named entities and 6 implicit categories to characterize the key information of case texts, and provides an epidemic case information extraction framework based on a pre-trained language model to automatically extract the core elements in case information. The framework uses a pre-trained language model to encode the case text and obtain a vector representation of the case text. Subsequently, two different neural network structures are used to identify and classify the named entities and implicit categories in the case, respectively, to achieve a structured representation of the case text.

[0006] In order to achieve the above-mentioned invention purpose and solve the problems existing in the prior art, the technical solution adopted by the present invention is: a method for constructing an epidemic case information extraction framework based on a pre-trained language model, comprising the following steps:

[0007] Step 1: Label the original epidemic case data: divided into named entity labeling and implicit category labeling; wherein, the named entity labeling refers to constructing named entity types, including basic information related to the infected person, symptom time related, movement trajectory related and medical institution related, re-labeling the entity types using proprietary symbols, and formulating the labeling paradigm and the minimum labeling granularity, and labeling the entities using the "BIO" labeling method; the implicit category labeling refers to integrating the coding methods involved in specific scenarios and re-labeling the categories to form a variety of implicit categories; the named entity labeling data and all implicit category data are divided into training sets, validation sets and test sets in proportion;

[0008] Step 2: Use the pre-trained language model to encode the medical record text and output the word vector and sentence vector corresponding to the case text;

[0009] Step 3: Use a bidirectional long short-term memory network (LSTM) and a conditional random field (CRF) to build a named entity recognition network. Based on the word vector of the case text output in step 2, determine the correct label of each word in the case text. The entity recognition network trains network parameters on the corresponding training set, and verifies and tests the recognition effect on the validation set and test set.

[0010] Step 4: Sample the fully connected neural network to construct an implicit category prediction network. Based on the sentence vector of the case text output in step 2, predict the correct implicit category corresponding to the text. The implicit category prediction network trains the network parameters on the corresponding training set, and verifies and tests the recognition effect on the validation set and test set.

[0011] Furthermore, in step 1, the named entity types constructed include 11 fine-grained named entities: age, gender, starting place, transit place, destination, arrival time at destination, isolation time, onset time, consultation time, consultation and treatment hospital, and diagnosis time.

[0012] Furthermore, in step 1, a named entity recognition annotation system including named entity types is constructed, as shown in Table 1:

[0013] Table 1. Named entity recognition annotation system

[0014]

[0015] Among them, [ ] represents the label in a column marked with the category mentioned in the case text; XX place represents the name of the transit point mentioned in the case text; XX hospital represents the name of the consultation and treatment hospital mentioned in the case text.

[0016] Furthermore, in step 1, there are 6 implicit categories, including infection place, contact location event, contact population, isolation place, discovery method, and disease severity.

[0017] Furthermore, in step 1, an implied category annotation system including implied categories is constructed, as shown in Table 2:

[0018] Table 2. Implicit category labeling system

[0019]

[0020]

[0021] Among them, NA means that this category of information is not mentioned in the case text; returnees* means returnees from places other than the place where the epidemic occurred.

[0022] Furthermore, in step 1, the specific steps of the named entity labeling are:

[0023] (a) Using manual rough labeling to label the entities in the case to obtain rough labeling data;

[0024] (b) Statistically analyze the label distribution of different named entity types in the coarsely labeled data. To reduce the amount of manual labeling, use the label same distribution sampling method to select some case texts for subsequent fine labeling. The loss function of the same distribution sampling is shown in formula (1). Perform multiple rounds of random sampling, calculate the same distribution sampling loss, and select the round of sampling samples with the smallest loss as the final fine labeling samples.

[0025]

[0026] Where L is the total number of labels, and They represent the number of the i-th label in the coarse labeled data and the number of the i-th label in the sampled data respectively;

[0027] (c) Finely annotate the sampled case texts, that is, based on the coarsely annotated data, annotate the fields in the case texts that do not belong to the selected entity range as other entity classes, including other time, other place, and other institutions;

[0028] Furthermore, in step 2, the pre-trained language model is BERT.

[0029] Furthermore, in step 2, the pre-trained language model BERT is used to encode the medical record text; the initial input of the model is the case text set C = {c1,c,…c m …,c M},c m represents the mth case text, where m∈M; case text c represents the word set c={w1,w2,…w n …,w N},w n represents the nth character in the case text, where n∈N; the input vector E of the pre-trained language model n It is composed of a word embedding vector, a segment embedding vector, and a position encoding vector; where the word embedding vector is the value of each character w n The embedding vector obtained from the embedding matrix, where the [CLS] and [SEP] symbols indicate the beginning and end of the case text respectively; the segment embedding vector indicates the number of the case text pair; the position encoding vector indicates the position information of each word;

[0030] The process of converting text into vectors by the pre-trained language model is summarized by formula (2):

[0031] X n =Pre_trained(E n ,θ) (2)

[0032] Where n∈N, θ is the parameter set of the pre-trained language model; when X n When taking the real number vector corresponding to each character in the text, the output of the pre-trained language model is the word vector in the case text; when X n When taking the real number vector corresponding to the [CLS] sentence start symbol, the output of the pre-trained language model is the sentence vector in the case text.

[0033] Furthermore, in step 3, a named entity recognition network is constructed using bidirectional LSTM and CRF, which specifically includes the following steps:

[0034] (a) Word vectors based on the output of the pre-trained language model Bidirectional LSTM is used to learn the long-distance dependency information between entities, which is described by formula (3):

[0035]

[0036] Among them, h n is the vector output after bidirectional LSTM encoding;

[0037] (b) A CRF layer is added after the bidirectional LSTM to better learn the dependencies between named entity tags. The CRF layer is based on the output h of the bidirectional LSTM. n The probability distribution of the label path for the input learning; for a given input c = {c1, c2, ..., c N}, the probability of its label sequence y={l1,l2,…l n …,l N} is expressed by formula (4),

[0038]

[0039] Where y' is any possible label sequence, and are trainable parameters in the model;

[0040] (c) For a given M training samples The named entity recognition network calculates the network loss through the loss function described by formula (5), and optimizes the named entity recognition network by minimizing the loss function;

[0041]

[0042] Furthermore, step 4, constructing an implicit category prediction network based on a fully connected neural network, specifically includes the following steps:

[0043] (a) Given a sentence vector encoded by a pre-trained language model, a fully connected neural network is used to perform nonlinear transformation to extract text features, and then a softmax classifier is used to obtain the implicit category label of the case text;

[0044] (b) For a given M case texts and case text categories The implicit category prediction network calculates the network loss through the loss function described by formula (6), and optimizes the implicit category prediction network by minimizing the loss function.

[0045]

[0046] Where i represents the i-th training case text, represents the category predicted by the model, and y represents the true category of the case text.

[0047] The beneficial effects of the present invention are: the present invention designs a data labeling rule for epidemic infection cases, and proposes an information extraction framework based on a pre-trained language model to automatically extract core elements in cases, wherein the named entity recognition network can accurately identify named entities in case texts and locate key information on case transmission paths, and the implicit case category prediction network can efficiently predict implicit categories and determine the main forms of case transmission paths, thereby realizing a structured representation of case texts to further assist disease prevention and control experts in formulating intervention measures for the spread of epidemic viruses. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the steps of extracting case text information in an embodiment of the present invention.

[0049] Figure 2 It is a model framework diagram of an embodiment of the present invention.

[0050] Figure 3 This is the result of named entity recognition in an embodiment of the present invention. The bar chart shows the recognition precision, recall rate and F value of the model for 9 types of named entities. The precision is marked with a black triangle, the recall rate is marked with a gray diamond, and the length of the bar represents the F value. The pie-shaped area chart shows the number of labels for each type of fine-grained named entity in the case text.

[0051] Figure 4 It is the implicit category prediction result of the embodiment of the present invention.

[0052] Figure 5 This is a comparison chart of the F values ​​of 9 fine-grained named entities under rough annotation (RA) and fine annotation (FA) in the embodiment of the present invention. The bar graph in the figure represents the F value under fine annotation, and the line graph represents the F value under rough annotation. The square on the line graph is lower than the column height, indicating that the F value of the rough annotation is lower than the F value of the fine annotation. DETAILED DESCRIPTION

[0053] The present invention will be further described below in conjunction with the accompanying drawings.

[0054] like Figure 2 As shown in the model framework diagram of the embodiment of the present invention, the annotation data includes two forms: named entity annotation and implicit category annotation. "B-" indicates the starting position of a named entity, and "I-" indicates the middle position of a named entity. The epidemic case information extraction framework CCIE consists of a pre-trained language model, a named entity recognition network, and an implicit category prediction network.

[0055] 1. The input of the pre-trained language model is each word in the case text, [CLS] marks the beginning of a sentence, and [SEP] marks the separator between different sentences; "T" represents Transformer, and its internal structure is shown in sub-figure (a). The ⊕ in (a) represents the vector concatenation operation.

[0056] 2. The named entity recognition network is composed of bidirectional LSTM and CRF. “L” represents LSTM, and its internal structure is shown in sub-figure (b). The ⊕ in (b) represents vector concatenation. represents the bitwise multiplication of the vector elements. σ is the sigmoid function, tanh is the activation function, and x t and h t are the input and output of LSTM respectively.

[0057] 3. The implication category prediction network is composed of a fully connected neural network, and the [CLS] vector is a sentence vector.

[0058] 4. Model evaluation includes named entity recognition evaluation and implicit category prediction evaluation. The evaluation method is to analyze the error between the output of the model and the golden annotation.

[0059] like Figure 1 As shown, a method for constructing an epidemic case information extraction framework based on a pre-trained language model that integrates a named entity recognition network and an implicit category prediction network includes the following steps:

[0060] Step 1: Based on the data requirements of information extraction technology, a set of annotation rules for virus infection cases is designed, including two annotation systems: named entity and implied category. The specific steps include the following:

[0061] (a) Guided by the data needs of information extraction technology, a set of annotation rules for virus infection cases is designed, including two annotation systems: named entities and implied categories. The named entity annotation system covers 11 fine-grained named entity types. The corresponding proprietary symbols, annotation paradigms, and minimum annotation granularity are shown in Table 1. In order to allow the information extraction model to focus on key features, the minimum annotation granularity of the location categories QSL and MDL is limited to "city", and the minimum annotation granularity of the time category is limited to "day". The "BIO" annotation method is used to annotate the entities. Specifically, the characters at the starting position of the entity are marked as "BL", the characters in the middle position of the entity are marked as "IL", and the characters at the end position of the entity are marked as "OL", where L is the re-annotated label of the category to which the entity belongs;

[0062] (b) For the implicit category annotation system, the coding methods of the specific scenes involved are integrated, and the specific scenes with similar semantics are classified into one category, and finally integrated into 6 implicit categories, as shown in Table 3;

[0063] Table 3. Implicit category labeling system of this embodiment

[0064]

[0065]

[0066] Among them, NA means that this category of information is not mentioned in the case text; returnees* means returnees from places other than the place where the epidemic occurred.

[0067] Step 2: Process and label the original epidemic case data. According to the labeling rules of step 1, perform named entity labeling on the case text through coarse labeling and fine labeling; perform implicit category labeling through text matching to build a medical record information extraction data set; divide the small amount of finely labeled named entity data and all implicit category data into training sets, validation sets, and test sets according to a certain ratio to obtain the information extraction data set required by the epidemic case extraction framework CCIE;

[0068] Named entity annotation specifically includes the following sub-steps:

[0069] (a) According to the rules in Table 1, the entities in the case are labeled manually to obtain coarsely labeled data;

[0070] (b) The label distribution of the nine named entities in the coarsely labeled data was statistically analyzed. To reduce the amount of manual labeling, approximately 12% of the case texts were selected for subsequent fine labeling using the label same distribution sampling method. The loss function of the same distribution sampling is as shown in the above formula (1). 100 rounds of random sampling were performed, and the same distribution sampling loss was calculated. The sampling sample with the smallest loss was selected as the final fine labeling sample.

[0071] (c) Finely annotate the sampled case texts, that is, based on the coarsely annotated data, annotate the fields in the case texts that do not belong to the selected entity range as other entity classes, including other time (OTT), other location (OTL), and other institution (OTI);

[0072] For the above implied category annotation, a vocabulary is constructed according to Table 3 to store all implied category labels and all scenarios under the label. Each case text is matched with all scenarios in the vocabulary through text matching to determine the category label to which the infected case belongs.

[0073] Step 3: Use the pre-trained language model BERT to encode the medical record text and obtain word vectors that can express long-range semantic information. The initial input of the model is the case text set C = {c1, c, …c m …,c M}, c m represents the mth case text, where m∈M; case text c represents the word set c={w1,w2,…w n …,w N},w nrepresents the nth character in the case text, where n∈N; the input vector E of the pre-trained language model n It is composed of a word embedding vector, a segment embedding vector, and a position encoding vector; where the word embedding vector is the value of each character w n The embedding vector obtained from the embedding matrix, where [CLS] and [SEP] respectively indicate the beginning and end of the case text; the segment embedding vector indicates the number of the case text pair; and the position encoding vector indicates the position information of each word. The vector conversion process of the pre-trained language model to the text can be used with formula (2). When X n When taking the real number vector corresponding to each character in the text, the output of the pre-trained language model is the word vector in the case text. n When taking the real number vector corresponding to the [CLS] sentence start symbol, the output of the pre-trained language model is the sentence vector in the case text.

[0074] Step 4: Use bidirectional LSTM and CRF to build a named entity recognition network. The goal of this network is to determine the correct label of any given word in the case text. It specifically includes the following sub-steps:

[0075] (a) Vector output based on the pre-trained language model Bidirectional LSTM is used to learn the long-distance dependency information between entities, which is described by formula (3);

[0076] (b) A CRF layer is added after the bidirectional LSTM to better learn the dependencies between named entity tags. The CRF layer is based on the output h of the bidirectional LSTM. n For a given input c = {c1, c2, ..., c N}, the probability of its label sequence y={l1,l2,…,l N} is expressed by formula (4).

[0077] (c) For a given M training samples The named entity recognition network calculates the network loss through the loss function described by formula (5), and the named entity recognition network is optimized by minimizing the loss function.

[0078] Step 5: Construct an implicit category prediction network based on a fully connected neural network. Its goal is to predict the correct category corresponding to a complete case text s. Specifically, it includes the following sub-steps:

[0079] (a) Given a sentence vector encoded by a pre-trained language model, a fully connected neural network is used to perform nonlinear transformation to extract text features, and then a softmax classifier is used to obtain the implicit category label of the case text;

[0080] (b) For a given M case texts and case text categories The implicit category prediction network calculates the network loss through the loss function described by formula (6), and optimizes the implicit category prediction network by minimizing the loss function.

[0081] Step 6: Evaluate the epidemic case information extraction framework, and evaluate the information extraction effect of the framework for two different tasks: named entity recognition and implied category prediction.

[0082] In order to verify the effectiveness of the present invention, the present invention conducted experiments on 10,017 infected cases. For the named entity recognition task, 1,000 cases were selected from 1,200 accurately labeled cases as the training set for the task, and 100 cases were selected from the remaining 200 cases as the validation set and 100 cases as the test set. The model effect was evaluated on the test set. For the implicit category prediction task, all 10,017 infected cases were selected and divided into training set, validation set and test set in a ratio of 8:1:1. The model effect was evaluated on the test set. For the two different tasks of named entity recognition and implicit category prediction, two different evaluation systems were used to evaluate the model effect. For the named entity recognition task, the evaluation indicators are precision, recall and F-value, as shown in formula (7),

[0083]

[0084] Where TP represents the number of positive samples predicted correctly, FP represents the number of positive samples predicted incorrectly, FN represents the number of negative samples predicted incorrectly, and the F value is the harmonic mean of precision and recall;

[0085] For the implicit category prediction task, the accuracy (Acc.), macro-F value (macro-F) and weighted-F value (weighted-F) are used for evaluation, as shown in formula (8),

[0086]

[0087] Where m represents the number of samples in the test set, and K represents the number of sample label types. i Indicates the F value of each class label. W i Represents the weight of each type of label. Here, the number of samples of each type of label is used as the weight for calculating the weighted F value.

[0088] The epidemic case information extraction framework based on the pre-trained language model was used to identify 11 fine-grained named entities. The descriptions of the two fine-grained named entities, age (AGE) and gender (GED), in the case text were very standardized and fixed, so the regular rule matching method was used for extraction with an accuracy of 100%. The model's recognition results for the remaining 9 fine-grained named entities are as follows: Figure 3 As shown in the figure, the average F-value of the recognition of 9 types of fine-grained named entities is 81.004%. The recognition results of the five types of entities, namely, arrival time at destination (MDT), onset time (FBT), diagnosis time (QZT), destination (MDL), and hospital for treatment and admission (ZSI), are higher than the average F-value. Among them, the F-value of ZSI entities reached the highest 95.9%. At the same time, the model can also achieve a high F-value for the recognition rate of transit points (ZZL) with more specific description granularity and quarantine points (GLT) with a number accounting for less than 5%.

[0089] The epidemic case information extraction framework based on the pre-trained language model is used to predict the implicit categories of the case text. The classification results of the model for the six implicit categories are as follows: Figure 4 As shown, it can be seen that the classification accuracy (Acc.) of the epidemic case information extraction framework based on the pre-trained language model for the six implicit categories is more than 80%, which means that the model can accurately judge the label to which it belongs based on the content of the case text, thereby clarifying the contact objects of the infected person during the illness cycle and the current vital signs. The macro-F value (macro-F) reflects the classification performance of the model in different labels. It can be seen that the performance of the epidemic case information extraction framework based on the pre-trained language model in the contact location event (Event) class with a large number of label types and extremely unbalanced distribution is still competitive. However, it is lower than the classification results of the other five implicit categories. This is mainly because in the Event category, the classification results of Event_travel and Event_hospital with too few labels are low, which affects the overall macro-F value. To this end, we introduce the weighted F value to balance the impact of the number of samples on the macro-F value. The evaluation results of the weighted F value (weighted-F) are basically consistent with the accuracy. Comprehensive Figure 4 The results in the paper prove that, in different implicit categories, the epidemic case information extraction framework model based on the pre-trained language model can learn the semantic features of the case text according to different labels and better realize the implicit category prediction.

[0090] In addition, the present invention also uses the epidemic case information extraction framework based on the pre-trained language model to conduct experiments on the coarsely labeled dataset and the finely labeled dataset. The comparison results are as follows: Figure 5As shown, it can be seen that in the comparison of 9 fine-grained named entities, the result of fine annotation is better than the result of coarse annotation, especially the F value of ZSI entity is improved by 118%. The experimental results prove that the annotation rules for epidemic virus infection cases proposed in the present invention are suitable for the application of artificial intelligence technology in case text information extraction.

Claims

1. A method for constructing a framework for extracting epidemic case information based on a pre-trained language model, characterized in that: The steps include: Step 1: Label the original epidemic case data: It is divided into named entity labeling and implicit category labeling; the named entity labeling refers to constructing named entity types, including basic information related to the infected person, symptom time related, movement trajectory related, and medical institution related, re-labeling the entity types using proprietary symbols, and formulating the labeling paradigm and minimum labeling granularity, as shown in Table 1; Table 1. Named entity recognition annotation system Among them, [] indicates the label in a column of the category mentioned in the case text; XX place indicates the name of the transit place mentioned in the case text; XX hospital indicates the name of the hospital for treatment and admission mentioned in the case text; The implied category annotation refers to integrating and re-annotating the coding methods involved in specific scenarios to form a variety of implied categories, as shown in Table 2; Table 2. Implicit category labeling system Among them, NA means that the case text does not mention this category of information; returnees* means returnees from other places except where the epidemic occurred; Divide the named entity annotation data and all the implicit category data into training set, validation set and test set in proportion; Step 2: Use the pre-trained language model to encode the medical record text and output the word vector and sentence vector corresponding to the case text; specifically: use the pre-trained language model BERT to encode the medical record text; the initial input of the model is the case text set C = {c1,c,…c m …,c M }, c m represents the mth case text, where m∈M; case text c represents the word set c={w1,w2,…w n …,w N },w n represents the nth character in the case text, where n∈N; the input vector E of the pre-trained language model n It is composed of a word embedding vector, a segment embedding vector, and a position encoding vector; where the word embedding vector is the value of each character w n The embedding vector obtained from the embedding matrix, where the [CLS] and [SEP] symbols indicate the beginning and end of the case text respectively; the segment embedding vector indicates the number of the case text pair; the position encoding vector indicates the position information of each word; The process of converting text into vectors by the pre-trained language model is summarized by formula (2): X n =Pre_trained(E n ,θ) (2) Where n∈N, θ is the parameter set of the pre-trained language model; when X n When taking the real number vector corresponding to each character in the text, the output of the pre-trained language model is the word vector in the case text; when X n When taking the real number vector corresponding to the [CLS] sentence start symbol, the output of the pre-trained language model is the sentence vector in the case text; Step 3, using a bidirectional long short-term memory network LSTM and a conditional random field CRF to construct a named entity recognition network, based on the word vector of the case text output in step 2, determine the correct label of each word in the case text, the entity recognition network trains network parameters on the corresponding training set, and verifies and tests the recognition effect on the verification set and the test set; the use of bidirectional LSTM and CRF to construct a named entity recognition network specifically includes the following steps: (a) Word vectors based on the output of the pre-trained language model Bidirectional LSTM is used to learn the long-distance dependency information between entities, which is described by formula (3): Among them, h n is the vector output after bidirectional LSTM encoding; (b) The CRF layer is based on the output h of the bidirectional LSTM n The probability distribution of the label path for the input learning; for a given input c = {c1, c2, ..., c N }, the probability of its label sequence y={l1,l2,…l n …,l N } is expressed by formula (4), Where y' is any possible label sequence, and are trainable parameters in the model; (c) For a given M training samples The named entity recognition network calculates the network loss through the loss function described by formula (5), and optimizes the named entity recognition network by minimizing the loss function; Step 4: Sampling a fully connected neural network to construct an implicit category prediction network. Based on the sentence vector of the case text output in step 2, the correct implicit category corresponding to the text is predicted. The implicit category prediction network trains network parameters on the corresponding training set, and verifies and tests the recognition effect on the validation set and the test set. The sampling fully connected neural network to construct the implicit category prediction network specifically includes the following steps: (a) Given a sentence vector encoded by a pre-trained language model, a fully connected neural network is used to perform nonlinear transformation to extract text features, and then a softmax classifier is used to obtain the implicit category label of the case text; (b) For a given M case texts and case text categories The implicit category prediction network calculates the network loss through the loss function described by formula (6), and optimizes the implicit category prediction network by minimizing the loss function. Where i represents the i-th training case text, represents the category predicted by the model, and y represents the true category of the case text.

2. The method for constructing an epidemic case information extraction framework based on a pre-trained language model according to claim 1, characterized in that: In step 1, the named entity types constructed include age, gender, starting place, transit place, destination, time of arrival at destination, quarantine time, onset time, consultation time, consultation and treatment hospital, and diagnosis time.

3. The method for constructing an epidemic case information extraction framework based on a pre-trained language model according to claim 1, characterized in that: In step 1, the implicit categories include infection site, contact site event, contact population, quarantine site, discovery method, and disease severity.

4. The method for constructing an epidemic case information extraction framework based on a pre-trained language model according to claim 1, characterized in that: In step 1, the specific steps of the named entity labeling are: (a) Using manual rough labeling to label the entities in the case to obtain rough labeling data; (b) Statistically analyze the label distribution of different named entity types in the coarsely labeled data, and use the label same-distribution sampling method to select some case texts for subsequent fine-labeling. The loss function of the same-distribution sampling is shown in formula (1). Perform multiple rounds of random sampling, calculate the same-distribution sampling loss, and select the round of sampling samples with the smallest loss as the final fine-labeled samples; Where L is the total number of labels, and They represent the number of the i-th label in the coarse labeled data and the number of the i-th label in the sampled data respectively; (c) The sampled case texts are precisely annotated, that is, based on the coarsely annotated data, the fields in the case text that do not belong to the selected entity range are annotated as other entity classes, including other time, other place, and other institution.

Images