PICO extraction method and device based on large language model and storage medium

Abstract

The invention relates to a PICO extraction method and device based on a large language model and a storage medium, and is applied to the technical field of medical information, and the method comprises the steps: obtaining and preprocessing titles and abstracts of a large number of medical literatures, then combining preset cues related to PICO with texts to construct an instruction data set, and storing the instruction data set in a database; a text in an instruction data set is subjected to word segmentation processing to be converted into an input sequence, a large language model framework is constructed, a false positive suppression loss function is set, the false positive rate of PICO element extraction by the model is remarkably reduced, the large language model framework is trained through the input sequence, and a trained PICO element extraction large language model is obtained. Preprocessing medical text data to be retrieved, inputting the preprocessed medical text data into the trained model, and outputting a PICO extraction text by the model; and finally, each PICO classification result is output through the PICO classifier. The data are data related to the crowd / problem, intervention, comparison and result in the to-be-retrieved data.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and in particular to a PICO extraction method, device and storage medium based on a large language model. Background Art

[0002] The application of evidence-based medicine (EBM) in clinical practice faces many challenges, especially in the rapid retrieval and application of the best evidence. EBM requires doctors to consciously, explicitly, prudently and rationally use the latest best evidence when treating patients. However, with the rapid increase in the number of biomedical science literature, it is difficult for clinicians to quickly and effectively retrieve evidence and make decisions. In order to retrieve relevant evidence, it is first necessary to construct a structured clinical question; researchers widely use the PICO (Population / Question, Intervention, Comparison, Outcome) framework to decompose evidence needs into searchable keywords and form answerable research questions. The PICO framework helps to improve the efficiency of retrieving evidence in databases such as PubMed. Although the PICO framework is very useful, the literature in the literature database usually does not contain information about the PICO structure, which results in the search results may contain many literature that do not match the research question PICO. At the same time, using PICO information to screen relevant literature requires knowledge of evidence-based medicine, which is time-consuming and difficult to master. Therefore, being able to automatically extract PICO descriptions from biomedical literature is very valuable for promoting clinicians and the public to retrieve, evaluate and synthesize evidence.

[0003] Currently, there are three main PICO extraction methods for evidence in medical literature:

[0004] (1) Rule extraction algorithm: Multiple regular expressions are summarized based on the rules of the training samples in the data set, and then these regular expressions are used to match the corresponding PICO elements. The advantage of this method is that it is simple to run, and the corresponding PICO elements can be obtained by simply matching the rules. However, the disadvantages are also obvious. It lacks generalization ability, and the rules cannot cover all PICO elements, so there will be a large number of omissions. In addition, it is extremely dependent on manual summary of regular expressions, which is labor-intensive.

[0005] (2) PICO recognition algorithm based on machine learning: In order to solve the problem of missed elements in traditional rule extraction algorithms, researchers have developed classification algorithms based on machine learning, including machine learning methods based on naive Bayes, machine learning methods based on logistic regression, machine learning methods based on multi-layer perceptrons, and machine learning methods based on conditional random fields. The advantage of this type of method is that it can build a model to uniformly identify PICO elements and alleviate the problem of too many missed elements by training the model. The disadvantage is that this type of method has limited modeling capabilities and is prone to ignoring the correlation between attributes in the data set. In addition, most of these methods detect PICO elements one by one, so it is necessary to build and train multiple models separately, which consumes resources.

[0006] (3) PICO recognition algorithm based on deep learning: Considering the limited modeling ability of machine learning-based methods, the need to build multiple models to identify PICO elements separately, and the problem of consuming resources, researchers use deep learning and multi-label classification methods to perform PICO recognition. This method mainly includes PICO recognition algorithms based on long short-term memory networks, PICO recognition algorithms based on BERT-CRF, PICO recognition algorithms based on cascaded pointer networks, etc. This type of method has strong modeling ability, can well understand the semantics of the input text, and can effectively extract its PICO elements; however, it is extremely necessary to have a large and labeled data set for training the model, and different disease data sets require different models, which requires a good data foundation. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a PICO extraction method, device and storage medium based on a large language model to solve the problem that the regular expression of the existing rule extraction algorithm cannot cover all PICO elements, resulting in a large number of omissions, and is extremely dependent on manual summary of regular expressions, which is labor-intensive; at the same time, it solves the problem that the PICO recognition algorithm based on machine learning needs to establish and train multiple models according to the category of PICO, which consumes resources; and solves the problem that the PICO recognition algorithm based on deep learning requires a large regular annotated data set for training the model, and different models are required for data sets of different diseases.

[0008] According to a first aspect of an embodiment of the present invention, a PICO extraction method based on a large language model is provided, the method comprising:

[0009] Obtain titles and abstracts of multiple medical articles;

[0010] Preprocessing the titles and abstracts of the plurality of medical documents;

[0011] Setting prompt words related to PICO elements for large language model training, combining the prompt words with the preprocessed titles and abstracts of multiple medical documents to obtain an instruction data set;

[0012] Performing word segmentation processing on the text in the instruction data set to obtain an input sequence;

[0013] Constructing a large language model framework and setting a false positive suppression loss function for the large language model framework;

[0014] Inputting the input sequence into the large language model framework, iteratively training the large language model framework until a preset stop condition is met, and obtaining a trained PICO element extraction large language model;

[0015] Acquire medical text data to be retrieved, preprocess the medical text data to be retrieved, and input the preprocessed medical text data to be retrieved into the trained PICO element extraction large language model;

[0016] The trained PICO element extraction large language model outputs PICO extracted text;

[0017] The PICO extracted text is input into a pre-trained PICO classifier, and the PICO classifier outputs a PICO classification result.

[0018] Preferably,

[0019] The word segmentation processing of the text in the instruction data set includes:

[0020] Splitting the text in the instruction data set into multiple tokens, converting each token into a unique digital ID according to a preset vocabulary to obtain an input sequence;

[0021] Add a special marker [CLS] at the beginning of each input sequence and a special marker [SEP] at the end of each input sequence.

[0022] Preferably, it also includes:

[0023] Perform part-of-speech tagging on the split tokens;

[0024] The length of the input sequence is judged. If the length of the input sequence is greater than a preset standard length range, the input sequence is truncated; if the length of the input sequence is less than the preset standard length range, the input sequence is padded to maintain a consistent length of the input sequence.

[0025] Preferably,

[0026] The preset stop condition includes:

[0027] The loss value of the loss function no longer decreases or reaches a preset number of iterations.

[0028] Preferably,

[0029] The preprocessing of the titles and abstracts of the plurality of medical documents comprises:

[0030] Performing data cleaning on the titles and abstracts of the plurality of medical documents to remove special characters and punctuation marks in the titles and abstracts;

[0031] Performing format conversion on the titles and abstracts of the plurality of medical documents after data cleaning, so as to convert letters or English words in the titles and abstracts into the same uppercase and lowercase format;

[0032] Stop word processing is performed on the titles and abstracts of the multiple medical documents after format conversion to remove stop words in the titles and abstracts.

[0033] Preferably,

[0034] The expression of the false positive suppression loss function is:

[0035]

[0036] In the formula, length represents the length of the input sequence, class num It represents the number of categories of each PICO, which includes participants, intervention measures, control measures and outcome indicators; the FP represents a false positive, which refers to the situation in which a negative sample is mistakenly predicted as a positive sample during the prediction process.

[0037] According to a second aspect of an embodiment of the present invention, a PICO extraction device based on a large language model is provided, the device comprising:

[0038] Training data acquisition module: used to obtain titles and abstracts of multiple medical documents;

[0039] Preprocessing module: used for preprocessing the titles and abstracts of the plurality of medical documents;

[0040] Instruction data construction module: used to set prompt words related to PICO elements for large language model training, and combine the prompt words with the titles and abstracts of multiple pre-processed medical documents to obtain an instruction data set;

[0041] Word segmentation module: used to perform word segmentation processing on the text in the instruction data set to obtain an input sequence;

[0042] Model framework setting module: used to build a large language model framework and set a false positive suppression loss function for the large language model framework;

[0043] Training module: used for inputting the input sequence into the large language model framework, iteratively training the large language model framework until a preset stop condition is met, and obtaining a trained PICO element extraction large language model;

[0044] Input module: used for acquiring medical text data to be retrieved, preprocessing the medical text data to be retrieved, and inputting the preprocessed medical text data to be retrieved into the trained PICO element extraction large language model;

[0045] Output module: used for the trained PICO element extraction large language model to output PICO extracted text;

[0046] Classification module: used to input the PICO extracted text into a pre-trained PICO classifier, and the PICO classifier outputs various PICO classification results.

[0047] According to a third aspect of an embodiment of the present invention, there is provided a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a main controller, each step in the above method is implemented.

[0048] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0049] The present application obtains the titles and abstracts of a large number of medical documents and preprocesses them, then combines the pre-set PICO-related prompt words with the text to construct an instruction data set, converts the text in the instruction data set into an input sequence by performing word segmentation processing, constructs a large language model framework, and sets a false positive suppression loss function to explicitly reduce the false positive rate of the model for PICO element extraction, trains the large language model framework through the input sequence to obtain a trained PICO element extraction large language model, preprocesses the medical text data to be retrieved and inputs it into the trained model, and the model outputs the PICO extracted text; finally, the PICO classification results are output through the PICO classifier, that is, the data related to the four categories of population / problem, intervention, comparison, and result in the data to be retrieved; thereby solving the problems in the prior art of relying on manual summary of regular expressions, which consumes manpower, or the need to separately establish and train multiple models, which consumes resources, or the need for large-scale regular annotated data sets for training models.

[0050] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0052] Figure 1 is a flowchart of a PICO extraction method based on a large language model according to an exemplary embodiment;

[0053] Figure 2 is a schematic diagram of a PICO element extraction model architecture according to another exemplary embodiment;

[0054] Figure 3 is a system schematic diagram of a PICO extraction device based on a large language model according to another exemplary embodiment;

[0055] In the accompanying drawings: 1- training data acquisition module, 2- preprocessing module, 3- instruction data construction module, 4- word segmentation module, 5- large language model framework setting module, 6- training module, 7- input module, 8- output module, 9- classification module. DETAILED DESCRIPTION

[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0057] Embodiment 1

[0058] Figure 1 is a flow chart of a PICO extraction method based on a large language model according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0059] S1, obtain titles and abstracts of multiple medical articles;

[0060] S2, preprocessing the titles and abstracts of the plurality of medical documents;

[0061] S3, setting prompt words related to PICO for large language model training, combining the prompt words with the preprocessed titles and abstracts of multiple medical documents to obtain an instruction data set;

[0062] S4, performing word segmentation processing on the text in the instruction data set to obtain an input sequence;

[0063] S5. Build a large language model framework and set a false positive suppression loss function for the large language model framework;

[0064] S6. Input the input sequence into the large language model framework and perform iterative training on the large language model framework until a preset stop condition is met, obtaining a trained large language model for PICO element extraction;

[0065] S7. Obtain the medical text data to be retrieved, preprocess the medical text data to be retrieved, and input the preprocessed medical text data to be retrieved into the trained large language model for PICO element extraction;

[0066] S8. The trained large language model for PICO element extraction outputs PICO-related results;

[0067] S9. Input the PICO-related results into a pre-trained PICO classifier, and the PICO classifier outputs PICO classification results;

[0068] It can be understood that, as shown in the appendix Figure 2 In this embodiment, by collecting the titles and abstracts of a large number of medical documents to be processed and preprocessing the collected titles and abstracts of medical documents, the preprocessing includes:

[0069] Perform data cleaning on the titles and abstracts of multiple medical documents to remove special characters and punctuation marks in the titles and abstracts. Improving data quality is one of the core purposes of data cleaning; perform format conversion on the titles and abstracts of medical documents after data cleaning to convert letters or English words in the titles and abstracts into the same case format, so as to process words with different cases in the same way. For example, "text", "Text", and "TEXT" are uniformly processed; remove stop words in the titles and abstracts. Stop words are common words that contribute little to text analysis, such as "de" (of), "shi" (is), etc. During the preprocessing process, these words are usually removed to reduce noise and improve model efficiency; through these preprocessing steps, it can be ensured that the input text meets the requirements of the model, improve the performance and accuracy of the model, and thus be better applied to the classification of PICO;

[0070] Based on the pre-processed title summary data combined with PICO-related prompt words, an "instruction dataset" for large language model training is constructed. Prompt words refer to a method of enhancing model performance by inputting specific instructions or questions during model training without adjusting the model structure itself; by inputting specific instructions or questions to the model, prompt words can guide the model to generate more accurate and targeted outputs. When dealing with complex tasks, carefully designed prompt words can improve the accuracy and efficiency of the model. Compared with traditional model adjustment methods, prompt words do not require modification of the model structure, thereby simplifying the training process. In practical applications, tips for using prompts include: asking questions directly and clearly, avoiding unnecessary polite language, such as "describe the structure of human cells"; clearly indicating the intended audience type in the prompt, such as providing different explanations for users of different age groups; breaking down complex tasks into a series of clear and specific prompts, gradually guiding the model to a deeper understanding; using positive guidance words such as "do" or "execute" and avoiding negative expressions; when a deeper understanding is required, directly asking the model to explain in the simplest and clearest language; combining thought chains with a small number of example prompts to help the model better understand and generate answers.

[0071] Next, the text in the instruction dataset is segmented to obtain the input sequence, specifically:

[0072] Attached Figure 2 The Tokenizer in the model training is responsible for converting the text string into a structured data form that the model can process. It usually divides the text into "tokens" or units such as words, phrases, and subwords. For example, the Chinese word segmenter will split "natural language processing" into "natural", "language", and "processing", while the English Tokenizer may use SubwordTokenization such as Byte-Pair Encoding (BPE) to process rare words; these tokens are the basis for the model to understand the text. The type and complexity of the Tokenizer can vary according to the task requirements, from simple space-based segmentation to more complex rule-based or machine learning-based word segmentation methods. This embodiment does not impose any restrictions on this.

[0073] Tokenizer encodes text into numbers according to the dictionary, which is called token embedding. When the input contains two sentences, the [SEP] flag is used to separate the two sentences to obtain segment embedding. The encoding of the first sentence is 0 and the encoding of the second sentence is 1. In addition, the [CLS] flag is placed at the beginning of the first sentence. The representation vector obtained by BERT can be used for subsequent classification tasks.

[0074] Add special markers at the beginning and end of the sequence, such as [CLS] and [SEP], for task-specific sequence classification or distinguishing input segments;

[0075] Convert tokens to numeric IDs, which are the input to the model. Each token has a unique ID in the vocabulary.

[0076] To ensure consistent length of input sequences, Tokenizer can pad shorter sequences and truncate longer sequences;

[0077] After obtaining an input sequence with a length within the preset range, a large language model framework is constructed. The large language model framework is usually: Transformers, DeepSpeed, Megatron-LM, LangChain, Haystack, etc. The most suitable large language model framework for medical information extraction is the Transformer model architecture, which can process a large amount of medical data;

[0078] A loss function for a large language model framework is set. This embodiment constructs a false positive suppression loss function as a loss function for model training. This loss function enables the model to predict the target area more accurately and significantly reduces the situation where the background is misjudged as the target (i.e., false positive). The false positive suppression loss function is as follows:

[0079]

[0080] In the formula, length represents the length of the input sequence, class num represents the number of categories of each PICO, which includes participants, intervention measures, control measures, and outcome indicators; FP represents false positive, which refers to the situation in which a negative sample is mistakenly predicted as a positive sample during the prediction process;

[0081] The trained large language model is fine-tuned using the input sequence, and the parameters of the large language model are updated until a preset number of iterations is reached or the loss value of the loss function no longer decreases, thereby obtaining a trained PICO element extraction large language model;

[0082] For new medical text data, after preprocessing, the PICO element extraction model obtained above is input to perform PICO extraction, and the output is output according to the format agreed in the prompt word (such as Json format). The output of the model is then classified by a pre-trained classifier, and the classification results are divided into participants, intervention measures, control measures and outcome indicators.

[0083] Embodiment 2

[0084] Figure 3 is a system schematic diagram of a PICO extraction device based on a large language model according to another exemplary embodiment, the device comprising:

[0085] Training data acquisition module 1: used to obtain titles and abstracts of multiple medical documents;

[0086] Preprocessing module 2: used for preprocessing the titles and abstracts of the plurality of medical documents;

[0087] Instruction data construction module 3: used to set prompt words related to PICO elements for large language model training, and combine the prompt words with the titles and abstracts of multiple pre-processed medical documents to obtain an instruction data set;

[0088] Word segmentation module 4: used for performing word segmentation processing on the text in the instruction data set to obtain an input sequence;

[0089] Large language model framework setting module 5: used to construct a large language model framework and set a false positive suppression loss function for the large language model framework;

[0090] Training module 6: used for inputting the input sequence into the large language model framework, iteratively training the large language model framework until a preset stop condition is met, and obtaining a trained PICO element extraction large language model;

[0091] Input module 7: used for acquiring medical text data to be retrieved, preprocessing the medical text data to be retrieved, and inputting the preprocessed medical text data to be retrieved into the trained PICO element extraction large language model;

[0092] Output module 8: used for the trained PICO element extraction large language model to output PICO extracted text;

[0093] Classification module 9: used for inputting the PICO extracted text into a pre-trained PICO classifier, and the PICO classifier outputs various PICO classification results.

[0094] Embodiment three:

[0095] This embodiment provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a main controller, each step in the above method is implemented;

[0096] It is understandable that the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0097] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0098] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0099] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0100] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0101] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0102] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0103] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0104] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0105] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A PICO extraction method based on a large language model, characterized in that: The method comprises: Obtain titles and abstracts of multiple medical articles; Preprocessing the titles and abstracts of the plurality of medical documents; Setting prompt words related to PICO elements for large language model training, combining the prompt words with the preprocessed titles and abstracts of multiple medical documents to obtain an instruction data set; Performing word segmentation processing on the text in the instruction data set to obtain an input sequence; Constructing a large language model framework and setting a false positive suppression loss function for the large language model framework; Inputting the input sequence into the large language model framework, iteratively training the large language model framework until a preset stop condition is met, and obtaining a trained PICO element extraction large language model; Acquire medical text data to be retrieved, preprocess the medical text data to be retrieved, and input the preprocessed medical text data to be retrieved into the trained PICO element extraction large language model; The trained PICO element extraction large language model outputs PICO extracted text; The PICO extracted text is input into a pre-trained PICO classifier, and the PICO classifier outputs a PICO classification result.

2. The method according to claim 1, characterized in that: The word segmentation processing of the text in the instruction data set includes: Splitting the text in the instruction data set into multiple tokens, converting each token into a unique digital ID according to a preset vocabulary to obtain an input sequence; Add a special marker [CLS] at the beginning of each input sequence and a special marker [SEP] at the end of each input sequence.

3. The method according to claim 2, characterized in that Also includes: Perform part-of-speech tagging on the split tokens; The length of the input sequence is judged. If the length of the input sequence is greater than a preset standard length range, the input sequence is truncated; if the length of the input sequence is less than the preset standard length range, the input sequence is padded to maintain a consistent length of the input sequence.

4. The method according to claim 3, characterized in that The preset stop condition includes: The loss value of the loss function no longer decreases or reaches a preset number of iterations.

5. The method according to claim 4, characterized in that The preprocessing of the titles and abstracts of the plurality of medical documents comprises: Performing data cleaning on the titles and abstracts of the plurality of medical documents to remove special characters and punctuation marks in the titles and abstracts; Performing format conversion on the titles and abstracts of the plurality of medical documents after data cleaning, so as to convert letters or English words in the titles and abstracts into the same uppercase and lowercase format; Stop word processing is performed on the titles and abstracts of the multiple medical documents after format conversion to remove stop words in the titles and abstracts.

6. The method according to claim 5, characterized in that The expression of the false positive suppression loss function is: In the formula, length represents the length of the input sequence, class num It represents the number of categories of each PICO, which includes participants, intervention measures, control measures and outcome indicators; the FP represents a false positive, which refers to the situation in which a negative sample is mistakenly predicted as a positive sample during the prediction process.

7. A PICO extraction device based on a large language model, characterized in that: The device comprises: Training data acquisition module: used to obtain titles and abstracts of multiple medical documents; Preprocessing module: used for preprocessing the titles and abstracts of the plurality of medical documents; Instruction data construction module: used to set prompt words related to PICO elements for large language model training, and combine the prompt words with the titles and abstracts of multiple pre-processed medical documents to obtain an instruction data set; Word segmentation module: used to perform word segmentation processing on the text in the instruction data set to obtain an input sequence; Model framework setting module: used to build a large language model framework and set a false positive suppression loss function for the large language model framework; Training module: used for inputting the input sequence into the large language model framework, iteratively training the large language model framework until a preset stop condition is met, and obtaining a trained PICO element extraction large language model; Input module: used for acquiring medical text data to be retrieved, preprocessing the medical text data to be retrieved, and inputting the preprocessed medical text data to be retrieved into the trained PICO element extraction large language model; Output module: used for the trained PICO element extraction large language model to output PICO extracted text; Classification module: used to input the PICO extracted text into a pre-trained PICO classifier, and the PICO classifier outputs various PICO classification results.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the main controller, each step of the PICO extraction method based on a large language model as described in any one of claims 1 to 6 is implemented.

Images