A method and system for automatic entity relation extraction in clinical trial eligibility criteria based on multi-turn question answering

Abstract

This invention discloses a method and system for automatically extracting entity relationships from clinical trial eligibility criteria based on multi-turn question answering. The method includes the following steps: preprocessing the acquired clinical trial eligibility criteria text to obtain free text; defining entity categories and relationship categories and labeling the free text, and designing head entity question templates and tail entity question templates; constructing positive and negative samples of head and tail entity question answers to train a BERT-BiLSTM-CRF model; concatenating the head entity question with the free text and inputting it into the trained model to predict the head entity name; then replacing the head entity name in the tail entity question and concatenating it with the free text again and inputting it into the model to predict the tail entity name; through multi-turn question answering, the model finally predicts all entities and their relationships in the free text. This invention can achieve high-precision automatic entity relationship extraction in the complex clinical scenario of clinical trial eligibility criteria.

Description

Technical Field

[0001] This invention belongs to the field of medical text mining technology, specifically relating to a method and system for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering. Background Technology

[0002] Clinical trials offer new hope to patients when traditional treatments have failed. Clinical trial websites allow physicians to search for suitable trials using keywords; however, due to the limitations of keywords, search results are often lengthy, the screening process is time-consuming and inefficient, and the semantic complexity of eligibility criteria makes manual review tedious and labor-intensive. Therefore, research into automated extraction methods is particularly important.

[0003] Initially, automatic extraction of qualification criteria primarily employed rule-based and dictionary-based methods. The core of this approach lies in using predefined rules and vocabularies to identify entities in text and the relationships between them. Due to its simplicity and speed of implementation, it was widely adopted in the early stages. However, this method also has significant limitations: the rules and dictionary require continuous manual updates to accommodate newly emerging entities or relationships, and it struggles to cover all expressions and linguistic diversity within the text. Therefore, when dealing with complex and ever-changing natural language, this method demands substantial manual effort, and its effectiveness is often limited.

[0004] With the rapid development of machine learning and deep learning technologies, various effective methods have been proposed for accurately identifying and extracting eligibility criteria in clinical trial information extraction.

[0005] The Criteria2Query system (Yuan C, Ryan PB, Ta C, et al (2019) Criteria2Query: a natural language interface to clinical databases for cohort definition. Journal of the American Medical Informatics Association, 26(4), 294-305.) is an early attempt at machine learning in this field. It has achieved excellent performance in entity recognition and relation extraction through steps such as sentence segmentation, named entity recognition, relation extraction and logic detection.

[0006] Tseo Y et al. further explored the application of deep neural networks in eligibility criteria information extraction, proposing the Att-BiLSTM-CRF model (Tseo Y, Salkola MI, Mohamed A, et al (2020) Information extraction of clinical trial eligibility criteria. arXiv preprint arXiv:2006.07296.). By introducing attention mechanisms and conditional random fields, this model showed comparable or even better performance than Criteria2Query in entity recognition and relation extraction.

[0007] Furthermore, the emergence of the pre-trained language model BERT has brought new opportunities to the field of information extraction. The CT-BERT model developed by Liu X et al. (Liu X, Hersch GL, Khalil I, et al (2021) Clinical trial information extraction with BERT. 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI). IEEE: p.505-506) utilizes BERT's powerful text representation capabilities and achieves efficient extraction of entities and relations by fine-tuning the pre-trained model, demonstrating performance superior to the aforementioned methods and proving the potential of pre-trained language models in information extraction tasks.

[0008] While these methods have achieved remarkable accuracy, they generally suffer from relatively low recall. Due to the complexity, diverse semantic categories and patterns of clinical trial eligibility criteria, and the existence of multiple patterns for the criteria, these texts often contain various entity types and complex relationships, frequently using technical terms and mixed English-Chinese expressions, placing high demands on the model's semantic understanding. Furthermore, current research on eligibility criterion information extraction relies on pipelined methods, which inherently suffer from error accumulation, also leading to low recall. Errors in each subtask can result in the omission of key entities or misidentification of relationships; these accumulated errors can bias the final extraction results, negatively impacting the system's recall. Summary of the Invention

[0009] In view of the above, the purpose of this invention is to provide a method and system for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering. By inputting the head entity question and the tail entity question into the BERT-BiLSTM-CRF model for multi-turn question answering, high-precision automatic extraction of entity relationships in clinical scenarios of complex clinical trial eligibility criteria can be achieved.

[0010] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0011] In a first aspect, embodiments of the present invention provide a method for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering, comprising the following steps:

[0012] The obtained clinical trial eligibility criteria text is preprocessed, including sentence segmentation and cleaning, to obtain the free text of the clinical trial eligibility criteria.

[0013] Based on the entity categories and relation categories defined in the free text, the free text is annotated according to the defined entity categories and relation categories to obtain annotated data. Head entity question templates and tail entity question templates are designed for the head entity category and tail entity category, respectively.

[0014] Positive samples for head entity question answering and tail entity question answering are constructed based on free text, labeled data and question templates. Negative samples are constructed based on free text and unlabeled entity categories. Positive and negative samples are then input into the BERT-BiLSTM-CRF model for model training.

[0015] In each round of question answering, the head entity question is concatenated with the free text and then input into the trained BERT-BiLSTM-CRF model to predict the head entity name. The head entity name is then replaced in the corresponding tail entity question and concatenated with the free text again before being input into the model to predict the tail entity name. Through multiple rounds of question answering, the model can finally predict all entities and their relationships in the free text.

[0016] Specifically, the entity categories and relation categories defined based on free text include:

[0017] Analyze entity words or phrases in free text and define head entity categories and tail entity categories, as well as the relationship categories between head entity categories and tail entity categories, where the head entity category is the entity category that serves as the starting point of the relationship and the tail entity category is the entity category that serves as the ending point of the relationship.

[0018] Specifically, the step of annotating the free text according to the defined entity categories and relation categories to obtain annotated text data includes:

[0019] Using the BART tool, entities in free text are labeled according to entity and relation categories and established labeling guidelines, resulting in labeled text data for each piece of free text.

[0020] Specifically, the step of designing header entity question templates and tail entity question templates for the header entity category and tail entity category in the entity category respectively includes:

[0021] Simulates the way humans ask questions. Design head entity question templates based on head entity category and tail entity question templates that lack a subject based on tail entity category. When answering questions, add the answer to the head entity question as the subject to the tail entity question template to form the tail entity question.

[0022] Specifically, the positive samples for constructing head entity question-answering and tail entity question-answering based on free text, labeled data, and question templates include:

[0023] For a single header entity category, its corresponding header entity question is concatenated with free text to construct a positive header entity question input and construct a true label for the positive answer;

[0024] For head and tail entity category pairs, the head entity question corresponding to the head entity category is concatenated with the free text to construct a positive head entity question input and the corresponding positive answer real label is given. The answer of the head entity question is added to the tail entity question template corresponding to the tail entity category to form the tail entity question. The tail entity question is concatenated with the free text to construct a positive tail entity question input and the corresponding positive answer real label is given.

[0025] Specifically, the construction of negative samples based on free text and unlabeled entity categories includes:

[0026] Select head entity categories that do not appear in the free text and are therefore not labeled, concatenate their corresponding head entity questions with the free text to construct a negative head entity question input, and provide the corresponding negative answer empty label;

[0027] Select tail entity categories that do not appear in the free text and are therefore not labeled, replace the labeled head entity names into the tail entity question template to form a tail entity question, concatenate the tail entity question with the free text to construct a negative tail entity question input, and provide the corresponding negative answer empty label.

[0028] Specifically, the process of using multiple rounds of question-and-answer to enable the model to ultimately predict all entities and their relationships in the free text includes:

[0029] In multi-turn question answering, the model identifies all head entity names in the free text by iterating through different head entity questions. For each head entity name, the subject part of the tail entity question is replaced, and the model identifies all tail entity names and corresponding relationship categories that are related to the head entity name in the free text by iterating through different tail entity questions. If there is no entity category corresponding to the question in the free text, the model will output "No relevant entity".

[0030] Secondly, in order to achieve the above-mentioned objectives, the present invention also provides an automatic entity relation extraction system for clinical trial eligibility criteria based on multi-turn question answering, including: a data acquisition and preprocessing module, a model training preparation module, a model training module, and an entity relation extraction and prediction module;

[0031] The data acquisition and preprocessing module is used to perform preprocessing on the acquired clinical trial eligibility criteria text, including sentence segmentation and cleaning, to obtain the free text of the clinical trial eligibility criteria.

[0032] The model training preparation module is used to define entity categories and relation categories in the free text, and to annotate the free text according to the defined entity categories and relation categories to obtain annotated data. It also designs head entity question templates and tail entity question templates for the head entity category and tail entity category, respectively.

[0033] The model training module is used to construct positive samples of head entity question answering and tail entity question answering based on free text, labeled data and question templates, and construct negative samples based on free text and unlabeled entity categories. The positive and negative samples are then input into the BERT-BiLSTM-CRF model for model training.

[0034] The entity relationship extraction and prediction module is used to concatenate the head entity question with the free text in each round of question answering and input it into the trained BERT-BiLSTM-CRF model to predict the head entity name. Then, the head entity name is replaced in the corresponding tail entity question and concatenated with the free text again and input into the model to predict the tail entity name. Through multiple rounds of question answering, the model can finally predict all entities and their relationships in the free text.

[0035] Thirdly, to achieve the above-mentioned objectives, embodiments of the present invention also provide an automatic entity relationship extraction device for clinical trial eligibility criteria based on multi-turn question answering, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-mentioned automatic entity relationship extraction method for clinical trial eligibility criteria based on multi-turn question answering when the computer program is executed.

[0036] Fourthly, to achieve the above-mentioned objectives, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-mentioned method for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering.

[0037] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0038] (1) This invention preprocesses the qualification standard text and deeply analyzes its content, which can comprehensively cover the different categories of the qualification standard. By defining the head entity category and tail entity category and their relationship category and annotating the free text, it tailors the corresponding head entity question and tail entity question for the multi-turn question answering paradigm. Through the multi-turn question answering of the model, it achieves high-precision automatic extraction of entity relationships.

[0039] (2) This invention integrates the multi-turn question answering paradigm with the BERT-BiLSTM-CRF model, thereby integrating and leveraging the high semantic understanding capabilities of the BERT-BiLSTM-CRF model and the multi-turn question answering paradigm. At the same time, it incorporates prior knowledge in entity questions, effectively alleviating the problem of error accumulation and improving the recall rate of the model. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the automatic entity relationship extraction method in the clinical trial eligibility criteria based on multi-turn question answering provided in this embodiment of the invention.

[0042] Figure 2 This is a schematic diagram of preprocessed text data provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the entity categories and relationship categories defined in the embodiments of the present invention;

[0044] Figure 4 This is a simplified illustration of a multi-round question-and-answer method for automatically extracting clinical trial eligibility criteria based on the BERT-BiLSTM-CRF model, provided in this embodiment of the invention.

[0045] Figure 5This is a schematic diagram of the structure of the automatic entity relationship extraction system in the clinical trial eligibility criteria based on multi-turn question answering provided in this embodiment of the invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0047] The inventive concept of this invention is as follows: Addressing the problems of low recall and error accumulation in current methods for automatically extracting entity relationships from clinical trial eligibility criteria, this invention provides a method and system for automatically extracting entity relationships from clinical trial eligibility criteria based on multi-turn question answering. The method and system employ a multi-turn question answering paradigm, selecting a BERT-BiLSTM-CRF model (deep learning and pre-trained language model) to preprocess the crawled original text of the clinical trial eligibility criteria, generating positive entity question samples and constructing corresponding negative entity question samples, which together constitute the training dataset. The BERT-BiLSTM-CRF model is then trained to be suitable for automatically extracting entity relationships from clinical trial eligibility criteria. In the prediction phase, the head entity name is predicted by concatenating free text with the head entity question template and inputting it into the model. This name is then used to replace the tail entity question template, concatenated with the free text, and input into the model again to predict the tail entity name. Through a multi-turn iterative question answering process, comprehensive extraction of entity relationships from clinical trial eligibility criteria is achieved.

[0048] Figure 1 This is a flowchart illustrating the automatic entity relation extraction method in the clinical trial eligibility criteria based on multi-turn question answering provided in this embodiment of the invention. Figure 1 As shown in the embodiment, an automatic entity relation extraction method for clinical trial eligibility criteria based on multi-turn question answering is provided, including the following steps:

[0049] S1. Preprocessing the obtained clinical trial eligibility criteria text, including sentence segmentation and cleaning, yields the free text of the clinical trial eligibility criteria.

[0050] S1.1, Data Acquisition. On the Drug Clinical Trial Registration and Information Disclosure Platform website, a search was conducted using "tumor" as the keyword. The Selenium tool was used to automatically extract eligibility criteria paragraphs from the fixed-format clinical trial webpages, and the obtained data was stored as a text file for subsequent reading and preprocessing.

[0051] S1.2, Data Preprocessing. The collected raw text of clinical trial eligibility criteria is segmented into sentences, using periods and semicolons as segmentation criteria to form single sentences of moderate length containing complete semantics. The segmented text is then cleaned by removing redundant spaces and special characters, and standardizing the character format to Chinese to ensure text cleanliness and format consistency. For example... Figure 2 As shown, the left image is the original text of the obtained clinical trial eligibility criteria, and the right image is the free text obtained after preprocessing.

[0052] S2 defines entity and relation categories based on the free text, and annotates the free text according to the defined entity and relation categories to obtain annotated data. It designs head entity question templates and tail entity question templates for the head entity category and tail entity category, respectively.

[0053] S2.1, Entity Relationship Category Definition. Referring to the table "General Data Elements for Common and Precision Oncology Clinical Trials" (Source: Zeng J, Shufean MA, Khotskaya Y, et al (2019) OCTANE: Oncology Clinical Trial Annotation Engine. JCO Clinical Cancer Informatics, 3, 1-11. https: / / doi.org / 10.1200 / CCI.18.00145), and combining the content of the free text corresponding to the actual crawled clinical trial eligibility criteria, the eligibility criteria text is analyzed in depth, and the definition of entity categories is refined to ensure that each category has clear boundaries and descriptions. In this example, 11 entity categories and 5 relationship categories are defined, such as... Figure 3As shown, this aims to comprehensively cover the various descriptions in the eligibility criteria. The 11 entity categories are divided into 8 head entity categories and 9 tail entity categories. The 8 head entity categories include: cancer type, treatment, drug, biopsy, gene type, stage, examination, and variant type. The 9 tail entity categories include: treatment, drug, biopsy, gene type, stage, time, outcome, variant type, and value. The five types of relationships include conditional relationships (has_condition), time relationships (has_time), outcome relationships (has_result), variation relationships (has_change), and value relationships (has_value). Conditional relationships are used to represent the relationship between head entity categories (cancer type) and tail entity categories (treatment, drug, biopsy, gene type, and stage). Time relationships are used to represent the relationship between head entity categories (treatment and drug) and tail entity categories (time). Outcome relationships are used to represent the relationship between head entity categories (biopsy) and tail entity categories (outcome). Variation relationships are used to represent the relationship between head entity categories (gene type) and tail entity categories (variation type). Value relationships are used to represent the relationship between head entity categories (examination) and tail entity categories (value).

[0054] S2.2, Data Annotation. 400 free texts of clinical trial eligibility criteria were randomly selected. An initial annotation guideline was developed before entity annotation and continuously refined during the annotation process to ensure consistency. Using the brat tool, entity and relation names in the free texts were manually annotated according to predefined entity and relation categories and the annotation guideline, and .ann annotation files were exported for each text. Subsequently, a Python script was written to read the entity and relation annotation data from each .ann file.

[0055] S2.3, Question Template Design. Simulating human questioning, head entity question templates are designed based on head entity categories, and tail entity question templates (lacking subjects) are designed based on tail entity categories. During question-and-answer sessions, the answer to the head entity question is added as the subject to the tail entity question template to form the tail entity question. The head entity question template aims to guide the model in identifying key entities, while the tail entity question template, considering its relevance to the previous round of question-and-answer, omits the subject to include the answer from the previous round, thus forming a coherent question-and-answer flow. By designing multi-round question-and-answer templates, the model can better understand the extraction task. Table 1 shows examples of head entity question and tail entity question templates.

[0056] Table 1. Template Examples for Head Entity Problems and Tail Entity Problems

[0057]

[0058] In the embodiments, as shown in Table 1, "Check" and "Value" form a type of relation, where "Check" is the head entity and "Value" is the tail entity. Each type of relation has a corresponding question template. In the question template for "Value," "Head3" refers to the name of the head entity extracted by the model. The tail entity question template integrates predefined relation categories, enabling the simultaneous extraction of relations and entities to a certain extent.

[0059] S3 constructs positive samples for head entity question answering and tail entity question answering based on free text, labeled data and question templates, and constructs negative samples based on free text and unlabeled entity categories. The positive and negative samples are then input into the BERT-BiLSTM-CRF model for model training.

[0060] Positive and negative samples are used as the training sample set. The training sample set contains question samples in text form and corresponding real label samples in the form of numerical sequences.

[0061] Before constructing real-valued label samples in the form of numerical sequences, the BIO annotation format is used to store entity boundaries and types. In this example, 11 entity categories are defined, each with two annotation results: "B-" and "I-", and one "O" annotation, resulting in 23 possible single-entity word annotation results. To facilitate model processing, these 23 category characters are assigned integer values, with "O" corresponding to the number 0, and the integer values ​​for other categories increasing sequentially. When constructing labels, a sequence of all zeros is built based on the known entity boundary index values, and the zeros are replaced with the corresponding entity numbers, thus forming real-valued labels in the form of numerical sequences used for model training.

[0062] When constructing text-based question samples, the free text and the entity question corresponding to the entity category are concatenated in the format "[CLS,Q,SEP,C,SEP]". If it is a tail entity question, the corresponding head entity name is replaced in the tail entity question template.

[0063] S3.1, Constructing positive samples. In constructing positive samples, for individual entities or entity pairs obtained by manual annotation, construct the head entity question input and its corresponding ground truth label, and the tail entity question input and its corresponding ground truth label, respectively.

[0064] (1) For individual entities existing in free text:

[0065] The free text and the corresponding header entity question for each entity category are concatenated in the format "[CLS,Q,SEP,C,SEP]" to generate a text-based sample of positive header entity questions. Then, a sequence of all zeros is constructed as the answer label. For each entity category, if its entity question has only one answer in the free text, the zero value at that answer position is replaced with the corresponding number. If multiple answers exist, the zero values ​​at all answer positions are replaced with the corresponding numbers, resulting in the true label for the positive answer.

[0066] (2) For entity pairs with constitutive relations existing in free text:

[0067] When constructing the head entity questions, the free text and the head entity questions corresponding to the head entity categories in the entity relations are concatenated according to the format "[CLS,Q,SEP,C,SEP]" to generate positive head entity question samples in text form. Similarly, a sequence of all zeros is constructed as the answer label. For each head entity question, if it has only one answer in the free text, the 0 value at that answer position is replaced with the corresponding number; if it has multiple answers, the 0 values ​​at all answer positions are replaced with the corresponding number, thus obtaining the true labels of the positive answers for the head entity questions.

[0068] When constructing tail entity questions, if the head entity question has multiple answers in the free text, the head entity name is replaced in the tail entity question template corresponding to the tail entity category in the entity relationship, forming multiple tail entity questions. These are then concatenated with the free text in the format "[CLS,Q,SEP,C,SEP]" to obtain positive tail entity question samples in text form. A sequence of all zeros is constructed as answer labels. For each tail entity question, if it has only one answer in the free text, the zero value at that answer position is replaced with the corresponding number; if it has multiple answers, the zero values ​​at all answer positions are replaced with the corresponding number, resulting in the true labels for positive answers to the tail entity questions.

[0069] S3.2, Construct negative samples.

[0070] (1) When constructing negative head entity samples, head entity categories that do not appear in the free text and are therefore unlabeled are selected. The corresponding question templates are concatenated with the free text in the format "[CLS,Q,SEP,C,SEP]" to generate negative head entity question samples in text form. Since the head entities in these samples do not exist in the text, they are assigned empty labels. A sequence of all zero numbers is constructed as the negative sample of the corresponding answer label to simulate entity categories that do not exist in the text. This trains the model to learn to ignore entities that are irrelevant to the text.

[0071] (2) When constructing negative tail entity samples, for known head entity names, tail entity categories that do not appear in the free text and are therefore unlabeled are selected. Using question templates of these categories, the labeled head entity names are replaced in the tail entity question templates to form tail entity questions. The tail entity questions are then concatenated with the free text in the format "[CLS,Q,SEP,C,SEP]" to generate negative tail entity question samples in text form. Similarly, since these tail entities also do not exist in the text, these samples are also given empty labels. A sequence of all zero numbers is constructed as the negative sample of the corresponding answer label to simulate entity relationships that do not exist in the text, so as to help the model learn to identify and eliminate false entity relationships.

[0072] The negative sample set constructed using the above method is large in number. A portion of the sample data is randomly selected, and the number of negative samples is continuously adjusted based on the training results to optimize the model's generalization ability and training efficiency.

[0073] S3.3, Model Training. The BERT-BiLSTM-CRF model constructed in this embodiment combines BERT, BiLSTM (Bidirectional Long Short-Term Memory Network), and CRF (Conditional Random Field). It employs a full-word masking Chinese BERT model, a pre-trained language model optimized for Chinese scenarios. The full-word masking strategy is particularly crucial, masking the entire Chinese vocabulary as a unit during pre-training to more accurately preserve and learn the integrity and context of Chinese words. Based on the pre-processed free text length statistics, the maximum input text length that BERT can handle is set to 150 characters. Regardless of the question / answer format, both positive and negative samples use the format "[CLS,Q,SEP,C,SEP]", where "C" represents free text, "Q" represents entity question, "CLS" represents input sequence markers, and "SEP" represents separator markers. After the samples are processed by the BERT word segmenter, the output vector undergoes high-dimensional processing by the BERT-BiLSTM-CRF model to predict the most likely label sequence.

[0074] S4: In each round of question answering, the head entity question is concatenated with the free text and then input into the trained BERT-BiLSTM-CRF model to predict the head entity name. The head entity name is then replaced in the corresponding tail entity question and concatenated with the free text again before being input into the model to predict the tail entity name. Through multiple rounds of question answering, the model can finally predict all entities and their relationships in the free text.

[0075] like Figure 4 The example shown is a simple example of the embodiment, which uses the qualifying sentence about cancer type "unresectable advanced hepatocellular carcinoma" in free text as an example to illustrate the question-and-answer process of entity and relation extraction.

[0076] S4.1, Head Entity Extraction. For example... Figure 4 As shown, the model focuses on identifying entities in text that do not have inherent relations or are head entities in relations. Head entity questions are concatenated with free text in the format "[CLS,Q,SEP,C,SEP]" and input into the BERT-BiLSTM-CRF model. These questions are fed into the BERT tokenizer and encoded into three vectors: tokenembeddings, segmentembeddings, and positionembeddings. All the encoded vectors for the head entity questions are then stacked together and input into the BERT-BiLSTM-CRF model for further high-dimensional processing. The model outputs the most probable prediction sequence, which is in numeric sequence format. Based on the mapping relationship between numeric encodings and category characters established during model training, the numeric sequence is converted into a BIO-formatted character sequence. The continuous strings (excluding "O") in the sequence are extracted and converted into the corresponding head entity names in the free text, thus obtaining the entity name answers for all head entity questions.

[0077] S4.2, Tail entity extraction. For example... Figure 4 As shown, the model aims to extract tail entities that can be related to previously identified head entities. Based on the head entity category corresponding to the head entity name, the head entity name is replaced in the "Head" part of all corresponding tail entity question templates. This head entity name is then concatenated with the free text in the format "[CLS,Q,SEP,C,SEP]" to generate one or more tail entity questions. These questions are also superimposed and input into the model for prediction. The predicted numeric sequence is converted into a character sequence, where the consecutive strings (excluding "O") in the character sequence correspond to the entity name answers of the tail entity questions in the free text.

[0078] In multi-turn question answering, the model identifies all head entity names in the free text by iterating through different head entity questions. For each head entity name, the subject part of the tail entity question is replaced, and the model identifies all tail entity names and their corresponding relationship categories in the free text by iterating through different tail entity questions. If no entity category corresponding to the question exists in the free text, the model outputs None, indicating that there is no relevant entity.

[0079] In summary, a multi-turn question-answering-based automatic entity relation extraction method for clinical trial eligibility criteria employs the BERT-BiLSTM-CRF model, demonstrating high effectiveness in information extraction. Benefiting from BiLSTM's ability to capture word order interactions, the model effectively integrates contextual information, thereby enhancing semantic understanding. BERT's strengths in grasping sentence-level semantics and long-distance dependencies further improve extraction accuracy. Simultaneously, the multi-turn question-answering paradigm transforms entity extraction into a human question-answering task, showcasing effective semantic understanding. This method provides prior information on target entity categories, merging entity and relation extraction into a unified task, contributing to accurate results. Combining the multi-turn question-answering paradigm with the excellent BERT-BiLSTM-CRF model further enhances the model's semantic understanding capabilities and serves as a joint extraction framework, reducing error accumulation and improving recall.

[0080] Based on the same inventive concept, such as Figure 5 As shown, this embodiment of the invention also provides an automatic entity relation extraction system 500 for clinical trial eligibility criteria based on multi-turn question answering, including: a data acquisition and preprocessing module 510, a model training preparation module 520, a model training module 530, and an entity relation extraction and prediction module 540.

[0081] The data acquisition and preprocessing module 510 is used to perform preprocessing on the acquired clinical trial eligibility criteria text, including sentence segmentation and cleaning, to obtain the free text of the clinical trial eligibility criteria.

[0082] The model training preparation module 520 is used to define entity categories and relation categories based on free text, and to annotate the free text according to the defined entity categories and relation categories to obtain annotated data. Head entity question templates and tail entity question templates are designed for the head entity category and tail entity category, respectively.

[0083] The model training module 530 is used to construct positive samples of head entity question answering and tail entity question answering based on free text, labeled data and question templates, and construct negative samples based on free text and unlabeled entity categories. The positive and negative samples are then input into the BERT-BiLSTM-CRF model for model training.

[0084] The entity relation extraction and prediction module 540 is used to concatenate the head entity question with the free text in each round of question answering and input it into the trained BERT-BiLSTM-CRF model to predict the head entity name. Then, the head entity name is replaced in the corresponding tail entity question and concatenated with the free text again and input into the model to predict the tail entity name. Through multiple rounds of question answering, the model can finally predict all entities and their relations in the free text.

[0085] Based on the same inventive concept, embodiments of the present invention also provide an automatic entity relation extraction device for clinical trial eligibility criteria based on multi-turn question answering, including a memory and one or more processors. The memory is used to store a computer program, and the processor is used to implement the above-described automatic entity relation extraction method for clinical trial eligibility criteria based on multi-turn question answering when the computer program is executed.

[0086] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering.

[0087] It should be noted that the above embodiments of the automatic entity relation extraction system, the automatic entity relation extraction device, and the computer-readable storage medium for the qualification criteria of clinical trials based on multi-turn question answering all belong to the same inventive concept as the method for automatic entity relation extraction in the qualification criteria of clinical trials based on multi-turn question answering. For details of the specific implementation process, please refer to the embodiment of the method for automatic entity relation extraction in the qualification criteria of clinical trials based on multi-turn question answering, which will not be repeated here.

[0088] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically extracting entity relations in clinical trial eligibility criteria based on multi-turn question answering, characterized in that, Includes the following steps: The obtained clinical trial eligibility criteria text is preprocessed, including sentence segmentation and cleaning, to obtain the free text of the clinical trial eligibility criteria. Based on the entity categories and relation categories defined in the free text, the free text is annotated according to the defined entity categories and relation categories to obtain annotated data. Head entity question templates and tail entity question templates are designed for the head entity category and tail entity category, respectively. Positive samples for head entity question answering and tail entity question answering are constructed based on free text, labeled data and question templates. Negative samples are constructed based on free text and unlabeled entity categories. Positive and negative samples are then input into the BERT-BiLSTM-CRF model for model training. In each round of question answering, the head entity question is concatenated with the free text and then input into the trained BERT-BiLSTM-CRF model to predict the head entity name. The head entity name is then replaced in the corresponding tail entity question and concatenated with the free text again before being input into the model to predict the tail entity name. Through multiple rounds of question answering, the model can finally predict all entities and their relationships in the free text.

2. The method for automatic entity relation extraction in clinical trial eligibility criteria based on multi-turn question answering as described in claim 1, characterized in that, The entity categories and relation categories defined in the free text include: Analyze entity words or phrases in free text and define head entity categories and tail entity categories, as well as the relationship categories between head entity categories and tail entity categories, where the head entity category is the entity category that serves as the starting point of the relationship and the tail entity category is the entity category that serves as the ending point of the relationship.

3. The method for automatic entity relation extraction in clinical trial eligibility criteria based on multi-turn question answering as described in claim 1, characterized in that, The annotation of the free text based on the defined entity categories and relation categories to obtain annotated text data includes: Using the BART tool, entities in free text are labeled according to entity and relation categories and established labeling guidelines, resulting in labeled text data for each piece of free text.

4. The method for automatic extraction of entity relationships in clinical trial eligibility criteria based on multi-turn question answering as described in claim 1, characterized in that, The above describes the design of header and tail entity question templates for the header and tail entity categories, respectively, including: Simulates the way humans ask questions. Design head entity question templates based on head entity category and tail entity question templates that lack a subject based on tail entity category. When answering questions, add the answer to the head entity question as the subject to the tail entity question template to form the tail entity question.

5. The method for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering as described in claim 1, characterized in that, The positive samples for constructing head entity question-answering and tail entity question-answering based on free text, labeled data, and question templates include: For a single header entity category, its corresponding header entity question is concatenated with free text to construct a positive header entity question input and construct a true label for the positive answer; For head and tail entity category pairs, the head entity question corresponding to the head entity category is concatenated with the free text to construct a positive head entity question input and the corresponding positive answer real label is given. The answer of the head entity question is added to the tail entity question template corresponding to the tail entity category to form the tail entity question. The tail entity question is concatenated with the free text to construct a positive tail entity question input and the corresponding positive answer real label is given.

6. The method for automatic extraction of entity relations in clinical trial eligibility criteria based on multi-turn question answering as described in claim 1, characterized in that, The construction of negative samples based on free text and unlabeled entity categories includes: Select head entity categories that do not appear in the free text and are therefore not labeled, concatenate their corresponding head entity questions with the free text to construct a negative head entity question input, and provide the corresponding negative answer empty label; Select tail entity categories that do not appear in the free text and are therefore not labeled, replace the labeled head entity names into the tail entity question template to form a tail entity question, concatenate the tail entity question with the free text to construct a negative tail entity question input, and provide the corresponding negative answer empty label.

7. The method for automatic extraction of entity relations in clinical trial eligibility criteria based on multi-turn question answering as described in claim 1, characterized in that, The process involves multiple rounds of question-and-answering to enable the model to ultimately predict all entities and their relationships in the free text, including: In multi-turn question answering, the model identifies all head entity names in the free text by iterating through different head entity questions. For each head entity name, the subject part of the tail entity question is replaced, and the model identifies all tail entity names and corresponding relationship categories that are related to the head entity name in the free text by iterating through different tail entity questions. If there is no entity category corresponding to the question in the free text, the model will output "No relevant entity".

8. An automatic entity relation extraction system for clinical trial eligibility criteria based on multi-turn question answering, characterized in that, include: The module includes a data acquisition and preprocessing module, a model training preparation module, a model training module, and an entity relationship extraction and prediction module. The data acquisition and preprocessing module is used to perform preprocessing on the acquired clinical trial eligibility criteria text, including sentence segmentation and cleaning, to obtain the free text of the clinical trial eligibility criteria. The model training preparation module is used to define entity categories and relation categories in the free text, and to annotate the free text according to the defined entity categories and relation categories to obtain annotated data. It also designs head entity question templates and tail entity question templates for the head entity category and tail entity category, respectively. The model training module is used to construct positive samples of head entity question answering and tail entity question answering based on free text, labeled data and question templates, and construct negative samples based on free text and unlabeled entity categories. The positive and negative samples are then input into the BERT-BiLSTM-CRF model for model training. The entity relationship extraction and prediction module is used to concatenate the head entity question with the free text in each round of question answering and input it into the trained BERT-BiLSTM-CRF model to predict the head entity name. Then, the head entity name is replaced in the corresponding tail entity question and concatenated with the free text again and input into the model to predict the tail entity name. Through multiple rounds of question answering, the model can finally predict all entities and their relationships in the free text.

9. An automatic entity relation extraction device for clinical trial eligibility criteria based on multi-turn question answering, comprising a memory and one or more processors, wherein the memory is used to store a computer program, characterized in that, The processor is configured to, when executing the computer program, implement the method for automatic extraction of entity relations in clinical trial eligibility criteria based on multi-turn question answering as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the method for automatically extracting entity relationships in clinical trial eligibility criteria based on multi-turn question answering as described in any one of claims 1-7.

Images