Literature screening method and device, electronic equipment and storage medium

Abstract

The invention provides a literature screening method and device, electronic equipment and a storage medium, and the method comprises the steps: extracting all literature data related to a screening topic from a literature library, and downloading a full text of the literature data and a corresponding abstract part; generating a cue word according to the screening theme, inputting an abstract part corresponding to the literature data and the cue word into an intelligent screening model, judging whether the literature data meets a PICO requirement or not through the intelligent screening model, and outputting a first screening result; the cue words and the literature data are input into an intelligent screening model in a full text mode, whether the literature data meet the PICO requirement or not is judged through the intelligent screening model, a second screening result is output, and the intelligent screening model is one or more large language models with the natural language understanding ability; and performing cross validation according to the first screening result and the second screening result, and screening literatures meeting PICO requirements or related to the current theme by using a large language model, thereby improving the accuracy and efficiency of literature screening.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a document screening method, device, electronic equipment and storage medium. Background Art

[0002] Medical literature is an important reference for basic medical theory and clinical research. It is an important task to select the required literature from a large amount of literature. The literature screening method based on the intelligent literature screening model can improve the efficiency of literature screening. The traditional intelligent literature screening model focuses on removing duplicate literature and then uses basic keywords for screening. It lacks precise screening for specific research questions such as PICO (P-subjects, I-intervention methods, C-controls, O-outcome indicators) guidelines, and is prone to missing important literature or including irrelevant literature. Summary of the invention

[0003] The present invention provides a document screening method, device, electronic device and storage medium, which are used to solve the defects of traditional document screening methods that important documents are easily missed or irrelevant documents are included.

[0004] The present invention provides a document screening method, comprising: Extract all literature data related to the screening topic from the literature library, and download the full text of the literature data and its corresponding abstract part; Generate prompt words according to the screening theme, input the abstract part and the prompt words corresponding to the document data into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output the first screening result; Input the prompt word and the full text of the document data into the intelligent screening model, determine whether the document data meets the PICO requirement through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; A cross-validation is performed based on the first screening result and the second screening result to obtain a final screening result.

[0005] According to the literature screening method provided by the present invention, the screening subject is vaccine, and judging whether the literature data meets the PICO requirements by the intelligent screening model includes: Outputting evaluation details of each document through the intelligent screening model; Determine, based on the evaluation details, whether the literature research is vaccine-related, whether it includes at least one of safety research, vaccine application targets, vaccine uses, effectiveness research, immunogenicity research, vaccination rate research, immunization program policy research, vaccine development, economic evaluation, literature type, clinical research type, and secondary research type related to vaccines; If at least one of the above conditions is met, the literature data is determined to meet the PICO requirements.

[0006] According to the document screening method provided by the present invention, generating prompt words according to the screening topics includes: Preset multiple prompt words related to vaccines, input the prompt words into a first large language model, and fine-tune the multiple prompt words according to output results of the first large language model to generate new prompt words; Inputting the new prompt word into the second largest language model, and adjusting the temperature of the output result of the second largest language model according to the output result of the first largest language model; Use a script to compare the output result of the first language model and the output result of the second language model, and when the output result of the first language model and the output result of the second language model are consistent, obtain a final prompt word; The first largest language model and the second largest language model are the same or different models.

[0007] According to the document screening method provided by the present invention, after outputting the first screening result, the method further comprises: Extracting some documents from the first screening results, verifying the some documents, and if the verification pass rate of the some documents does not reach a preset threshold, screening multiple documents from all document data related to the screening topic; The prompt word, the abstract part corresponding to the document included in the first screening result, and the abstract part corresponding to multiple documents screened from all the document data related to the screening topic are re-input into the intelligent screening model to update the first screening result.

[0008] According to the document screening method provided by the present invention, the step of inputting the prompt word and the full text of the document data into the intelligent screening model comprises: Obtaining text data of the full text of the document data; Converting the text data into a text vector; Obtaining relevant sentences and paragraphs according to the distances between the text vectors; The relevant sentences and paragraphs are input into the intelligent screening model.

[0009] According to the document screening method provided by the present invention, extracting all document data related to the screening topic from the document library includes: Acquire multiple document data from multiple document libraries; perform deduplication preprocessing on the document data; Through keyword technology, all literature data related to the screening topic are extracted from the deduplicated literature.

[0010] The document screening method provided by the present invention further comprises: The final screening results are classified and sorted through the intelligent screening model to obtain multiple types of literature data.

[0011] The present invention also provides a document screening device, comprising: The extraction module is used to extract all the literature data related to the screening topic from the literature library, and download the full text of the literature data and its corresponding abstract part; A first output module is used to generate prompt words according to the screening theme, input the abstract part corresponding to the document data and the prompt words into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output the first screening result; A second output module is used to input the prompt word and the full text of the document data into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; The verification module is used to perform cross-validation based on the first screening result and the second screening result to obtain a final screening result.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the document screening method as described in any one of the above items is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned document screening methods when executed by a processor.

[0014] The document screening method, device, electronic device and storage medium provided by the present invention extract all document data related to the screening topic from the document library, and download the full text of the document data and the corresponding abstract part; generate prompt words according to the screening topic, input the abstract part corresponding to the document data and the prompt words into the intelligent screening model, judge whether the document data meets the PICO requirement through the intelligent screening model, and output a first screening result; input the prompt words and the full text of the document data into the intelligent screening model, judge whether the document data meets the PICO requirement through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding ability; cross-validate the first screening result and the second screening result to obtain the final screening result, and use the large language model to screen documents that meet the PICO requirement or are related to the current topic, thereby improving the accuracy and efficiency of document screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 Schematic diagram of the process of the document screening method provided by the embodiment of the present invention; Figure 2 is a functional structure diagram of a document screening device provided by an embodiment of the present invention; Figure 3 It is a functional structure diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Figure 1 A flowchart of the document screening method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the document screening method provided by the embodiment of the present invention includes: Step 101: extract all document data related to the screening topic from the document library, and download the full text of the document data and its corresponding abstract part; In the embodiment of the present invention, extracting all document data related to the screening topic from the document library includes: Acquire multiple document data from multiple document libraries; perform deduplication preprocessing on the document data; Through keyword technology, all literature data related to the screening topic are extracted from the deduplicated literature.

[0019] For example, by using keyword search technology, all vaccine-related literature can be automatically extracted from online evidence-based medicine literature databases such as PubMed, Embase, and other databases, and the abstract part can be automatically downloaded.

[0020] Step 102: Generate prompt words according to the screening theme, input the abstract part and the prompt words corresponding to the document data into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output the first screening result; Step 103: input the prompt word and the full text of the document data into the intelligent screening model, determine whether the document data meets the PICO requirement through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; Step 104: cross-validate the first screening result and the second screening result to obtain a final screening result.

[0021] The literature screening model used in the traditional literature screening method focuses on removing duplicate literature, and then uses basic keywords for screening. It lacks the precise screening required by specific research questions such as PICO (P-subjects, I-interventions, C-controls, O-outcome indicators) guidelines, and is prone to missing important literature or including irrelevant literature.

[0022] The document screening method provided by the embodiment of the present invention extracts all document data related to the screening topic from the document library, and downloads the full text of the document data and the corresponding abstract part; generates prompt words according to the screening topic, inputs the abstract part corresponding to the document data and the prompt words into the intelligent screening model, judges whether the document data meets the PICO requirement through the intelligent screening model, and outputs a first screening result; inputs the prompt words and the full text of the document data into the intelligent screening model, judges whether the document data meets the PICO requirement through the intelligent screening model, and outputs a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; cross-validates the first screening result and the second screening result to obtain a final screening result, and uses the large language model to screen documents that meet the PICO requirement or are related to the current topic, thereby improving the accuracy and efficiency of document screening.

[0023] Based on any of the above embodiments, the screening subject is vaccine, and judging whether the literature data meets the PICO requirements by the intelligent screening model includes: Step 201: Output evaluation details of each document through the intelligent screening model; Step 202: determine whether the literature research is related to the vaccine, whether it includes at least one of safety research, vaccine application objects, vaccine use, effectiveness research, immunogenicity research, vaccination rate research, immunization program policy research, vaccine development, economic evaluation, literature type, clinical research type, and secondary research type related to the vaccine according to the evaluation details; Step 203: If at least one of the above conditions is met, it is determined that the literature data meets the PICO requirements.

[0024] In the embodiment of the present invention, the large language model has a strong natural language understanding ability and can accurately understand the semantics and contextual relationships in the document. By performing semantic analysis on the content of the document, the LLM can determine whether the document meets the PICO requirements, such as whether the research is related to vaccines, the application objects of vaccines, the use of vaccines, etc.

[0025] Based on any of the above embodiments, generating prompt words according to the screening topic includes: Step 301: preset a plurality of prompt words related to vaccines, input the prompt words into a first large language model, and fine-tune the plurality of prompt words according to output results of the first large language model to generate new prompt words; Step 302: input the new prompt word into the second largest language model, and adjust the temperature of the output result of the second largest language model according to the output result of the first largest language model; Step 303: using a script to compare the output result of the first language model and the output result of the second language model, and when the output result of the first language model and the output result of the second language model are consistent, obtaining a final prompt word; The first largest language model and the second largest language model are the same or different models.

[0026] In the embodiment of the present invention, the prompt words are, for example, the age of the subject of interest and the corresponding annotated data, the vaccine recipient and the corresponding annotated data, etc. By using LLM to build an automated screening process, a large amount of literature can be processed quickly, avoiding the inefficiency and error-prone problems of manual screening, and improving overall work efficiency.

[0027] Based on any of the above embodiments, after outputting the first screening result, the method further includes: Step 401: extract some documents from the first screening results, verify the some documents, and if the verification pass rate of the some documents does not reach a preset threshold, screen multiple documents from all document data related to the screening topic; Step 402: re-input the prompt word, the abstract part corresponding to the documents included in the first screening result, and the abstract parts corresponding to multiple documents selected from all the document data related to the screening topic into the intelligent screening model to update the first screening result.

[0028] The prior art lacks targeted screening and is prone to missing important documents or including irrelevant documents. When the first screening result does not meet the requirements, the present invention re-screens from all document data related to the screening topic, which can introduce more documents, avoid missing screening, and further improve the screening accuracy.

[0029] Based on any of the above embodiments, the step of inputting the prompt word and the full text of the document data into the intelligent screening model includes: Step 501: Obtain the text data of the full text of the document data; Step 502: convert the text data into a text vector; Step 503: Obtain relevant sentences and paragraphs according to the distances between the text vectors; Step 504: input the relevant sentences and paragraphs into the intelligent screening model.

[0030] In an embodiment of the present invention, a Retrieval Augmented Generation (RAG) architecture is applied. The working principle of the RAG architecture includes the steps of indexing, retrieval, and generation. In the indexing phase, documents are divided into small chunks, encoded into vector form, and then stored in a vector database. This process involves dividing the document content into small chunks and mapping these document chunks into a high-dimensional vector space using an encoder (such as a pre-trained language model, such as BERT or RoBERTa). In this way, each document can be quickly retrieved by its vector representation. In the retrieval phase, a query is input into a query encoder, which also maps the query into a vector space. Then, the most relevant documents to the query are found by calculating the similarity between the query vector and all stored document vectors. ‌This process leverages the Dual-Encoder architecture, which consists of two separate encoders: one for encoding the query and the other for encoding the document. By computing the similarity between them, efficient retrieval can be achieved. The generation phase feeds the original question and the retrieved relevant documents together into a large language model to generate the final answer. ‌This step leverages the power of the generator, which integrates and transforms the retrieved information into coherent and fluent text. The generator is guided by the retrieved text to ensure consistency between the generated text and the retrieved information. ‌This comprehensive input enables the generator to gain deep understanding of the context of the question, thereby providing more informative and contextual responses. ‌

[0031] The RAG architecture combines the retrieval and generation processes, finds the most relevant documents to the query through the retrieval model, and uses the generation model to conduct in-depth analysis of these documents, which can provide more accurate and relevant responses when processing natural language tasks. The RAG architecture can efficiently process large-scale document data and dynamically generate screening criteria and results during the screening process. Therefore, the full text of the document data is input into the intelligent screening model to implement the RAG architecture, which can quickly extract useful information from a large amount of information and further improve the screening efficiency.

[0032] Traditional intelligent screening models such as the XGBoost algorithm have complex construction and optimization processes. The RAG architecture simplifies the model construction process and improves screening efficiency by combining retrieval and generation. The prompt word engineering of the embodiment of the present invention can guide LLM to better understand the content of the document abstract, and the RAG technology can introduce external full-text document retrieval results. By cross-validating the screening results based on the abstract content with the screening results of the full text of the document, the accuracy of the screening results can be further verified and improved.

[0033] Based on any of the above embodiments, the document screening method further includes: The final screening results are classified and sorted through the intelligent screening model to obtain multiple types of literature data.

[0034] In some embodiments of the present invention, manual sampling verification is also included to ensure the accuracy of the screening model.

[0035] In the embodiment of the present invention, the screened literature data includes evaluations on different screening criteria, and the literature that meets the requirements is classified and sorted to generate a detailed report, which is applied to subsequent evidence-based medicine research and clinical practice.

[0036] The embodiment of the present invention is to screen a batch of literature about the new coronavirus vaccine (COVID-19 vaccine), with the goal of finding all research literature related to the effectiveness, safety, immunogenicity, vaccination rate, etc. of the new coronavirus vaccine.

[0037] (1) Data preprocessing: 5,000 relevant articles were retrieved from the PubMed database using the keyword "COVID-19 vaccine" and the abstracts were downloaded. These articles were preprocessed to remove duplicate articles, and finally 4,500 independent articles were obtained.

[0038] (2) Initial screening of keywords: The keyword screening model was used to conduct a preliminary screening of the literature, filter out literature not related to vaccines, and retain 2,500 potentially relevant literature.

[0039] (3) Abstract screening using a large language model: 2,500 literature abstracts were input into a large language model, and the input was optimized through prompt word engineering to guide the model to perform deep semantic analysis to determine whether each literature meets the following criteria: whether the research is vaccine-related, the target population of the vaccine (such as adults, children, and the elderly), the purpose of the vaccine (prevention, treatment, etc.), whether it contains safety studies, whether it contains efficacy studies, whether it contains immunogenicity studies, whether it contains vaccination rate studies, whether it contains immunization program policy studies, whether it contains vaccine development, whether it contains economic evaluation, literature type (such as RCT, meta-analysis, etc.), clinical research type, secondary research type, and 1,500 literature that meets the criteria was screened out.

[0040] (4) Full-text screening using a large language model: The full texts of 2,500 articles were input into a large language model. The input was optimized through prompt word engineering, which guided the model to perform deep semantic analysis to determine whether each article met the following criteria: whether the research was vaccine-related, the target population of the vaccine (such as adults, children, and the elderly), the purpose of the vaccine (prevention, treatment, etc.), whether it included safety studies, whether it included efficacy studies, whether it included immunogenicity studies, whether it included vaccination rate studies, whether it included immunization program policy studies, whether it included vaccine development, whether it included economic evaluation, document type (such as RCT, meta-analysis, etc.), clinical research type, and secondary research type. A total of 1,600 articles that met the criteria were screened out.

[0041] (5) Cross-validation: compare the 1,500 documents that meet the criteria screened out through abstracts with the 1,600 documents that meet the criteria screened out through full-text screening, output the screening results, and obtain 1,000 documents; (6) Classify and organize the evaluation of the 1,000 documents based on different screening criteria.

[0042] (7) Manual sampling verification was conducted. 100 articles were randomly selected from 1,000 articles for manual review to ensure the accuracy of the screening model. The verification results showed that the model accuracy rate reached 95%.

[0043] (8) Data output and application: Generate a detailed report based on the screening results, which includes the evaluation details and classification results of each document. The screening results will be applied to subsequent COVID-19 vaccine research and clinical practice to provide researchers with accurate literature support.

[0044] The embodiment of the present invention has been independently verified by more than 500 documents and has achieved an accuracy rate of more than 99%, which exceeds the 85% accuracy rate of manual extraction and can completely replace manual work.

[0045] The document screening method provided by the embodiment of the present invention uses a multi-level screening mechanism. In the initial screening stage, a keyword screening model is used to quickly remove obviously irrelevant documents; in the deep screening stage, a large language model is used to perform detailed semantic analysis on the documents to ensure that the document content meets the PICO requirements. Through the combination of initial screening and deep screening, the accuracy of the screening results is ensured. A real-time updated vaccine dynamic monitoring mechanism is formed. A high-quality vaccine-related document collection can be updated in real time, and real-time risk warning and safety evaluation can be performed.

[0046] The document screening device provided by the present invention is described below. The document screening device described below and the document screening method described above can be referenced to each other.

[0047] Figure 2 A schematic diagram of a document screening device provided by an embodiment of the present invention, such as Figure 2 As shown, the document screening device provided by the embodiment of the present invention includes: Extraction module 201, used to extract all document data related to the screening topic from the document library, and download the full text of the document data and its corresponding abstract part; The first output module 202 is used to generate prompt words according to the screening theme, input the abstract part and the prompt words corresponding to the document data into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output the first screening result; The second output module 203 is used to input the prompt word and the full text of the document data into the intelligent screening model, determine whether the document data meets the PICO requirement through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; The verification module 204 is used to perform cross-validation based on the first screening result and the second screening result to obtain a final screening result.

[0048] The document screening device provided by the embodiment of the present invention extracts all document data related to the screening topic from the document library, and downloads the full text of the document data and the corresponding abstract part; generates prompt words according to the screening topic, inputs the abstract part corresponding to the document data and the prompt words into the intelligent screening model, judges whether the document data meets the PICO requirement through the intelligent screening model, and outputs a first screening result; inputs the prompt words and the full text of the document data into the intelligent screening model, judges whether the document data meets the PICO requirement through the intelligent screening model, and outputs a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding ability; cross-validates the first screening result and the second screening result to obtain the final screening result, and uses the large language model to screen documents that meet the PICO requirement or are related to the current topic, thereby improving the accuracy and efficiency of document screening.

[0049] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the server may include: a processor 310 , a communications interface 320 , a memory 330 and a communications bus 340 , wherein the processor 310 , the communications interface 320 , and the memory 330 communicate with each other via the communications bus 340 . The memory 330 includes a computer program, an operating system, and acquired data. The processor 310 can call the logic instructions in the memory 330 to execute a document screening method, which includes: extracting all document data related to the screening topic from the document library, and downloading the full text of the document data and its corresponding abstract part; generating prompt words according to the screening topic, inputting the abstract part corresponding to the document data and the prompt words into an intelligent screening model, judging whether the document data meets the PICO requirement through the intelligent screening model, and outputting a first screening result; inputting the prompt words and the full text of the document data into the intelligent screening model, judging whether the document data meets the PICO requirement through the intelligent screening model, and outputting a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; and performing cross-validation based on the first screening result and the second screening result to obtain a final screening result.

[0050] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0051] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the document screening method provided by the above methods, the method comprising: extracting all document data related to the screening topic from a document library, and downloading the full text of the document data and its corresponding abstract part; generating prompt words according to the screening topic, inputting the abstract part corresponding to the document data and the prompt words into an intelligent screening model, judging whether the document data meets the PICO requirements through the intelligent screening model, and outputting a first screening result; inputting the prompt words and the full text of the document data into the intelligent screening model, judging whether the document data meets the PICO requirements through the intelligent screening model, and outputting a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; cross-validating the first screening result and the second screening result to obtain a final screening result.

[0052] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0053] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for screening documents, characterized in that: include: Extract all literature data related to the screening topic from the literature database, and download the full text of the literature data and its corresponding abstract part; Generate prompt words according to the screening theme, input the abstract part and the prompt words corresponding to the document data into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output the first screening result; Input the prompt word and the full text of the document data into the intelligent screening model, determine whether the document data meets the PICO requirement through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; A cross-validation is performed based on the first screening result and the second screening result to obtain a final screening result.

2. The method for document screening according to claim 1, characterized in that: The screening subject is vaccine, and judging whether the literature data meets the PICO requirements by the intelligent screening model includes: Outputting evaluation details of each document through the intelligent screening model; Determine, based on the evaluation details, whether the literature research is vaccine-related, whether it includes at least one of safety research, vaccine application targets, vaccine uses, effectiveness research, immunogenicity research, vaccination rate research, immunization program policy research, vaccine development, economic evaluation, literature type, clinical research type, and secondary research type related to vaccines; If at least one of the above conditions is met, the literature data is determined to meet the PICO requirements.

3. The method for document screening according to claim 1, characterized in that: The step of generating prompt words according to the screening topic includes: Preset multiple prompt words related to vaccines, input the prompt words into a first large language model, and fine-tune the multiple prompt words according to output results of the first large language model to generate new prompt words; Inputting the new prompt word into the second largest language model, and adjusting the temperature of the output result of the second largest language model according to the output result of the first largest language model; Use a script to compare the output result of the first language model and the output result of the second language model, and when the output result of the first language model and the output result of the second language model are consistent, obtain a final prompt word; The first largest language model and the second largest language model are the same or different models.

4. The method for document screening according to claim 1 or 3, characterized in that: After outputting the first screening result, the method further includes: Extracting some documents from the first screening results, verifying the some documents, and if the verification pass rate of the some documents does not reach a preset threshold, screening multiple documents from all document data related to the screening topic; The prompt word, the abstract part corresponding to the document included in the first screening result, and the abstract part corresponding to multiple documents screened from all the document data related to the screening topic are re-input into the intelligent screening model to update the first screening result.

5. The method for document screening according to claim 1, characterized in that: The step of inputting the prompt word and the full text of the literature data into the intelligent screening model comprises: Obtaining text data of the full text of the document data; Converting the text data into a text vector; Obtaining relevant sentences and paragraphs according to the distances between the text vectors; The relevant sentences and paragraphs are input into the intelligent screening model.

6. The method for document screening according to claim 1, characterized in that: The method of extracting all literature data related to the screening topic from the literature library includes: Acquire multiple document data from multiple document libraries; perform deduplication preprocessing on the document data; Through keyword technology, all literature data related to the screening topic are extracted from the deduplicated literature.

7. The method for document screening according to claim 1, characterized in that: Also includes: The final screening results are classified and sorted through the intelligent screening model to obtain multiple types of literature data.

8. A document screening device, characterized in that: include: An extraction module is used to extract all document data related to the screening topic from the document library, and download the full text of the document data and its corresponding abstract part; A first output module is used to generate prompt words according to the screening theme, input the abstract part corresponding to the document data and the prompt words into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output the first screening result; A second output module is used to input the prompt word and the full text of the document data into the intelligent screening model, determine whether the document data meets the PICO requirements through the intelligent screening model, and output a second screening result, wherein the intelligent screening model is one or more large language models with natural language understanding capabilities; The verification module is used to perform cross-validation based on the first screening result and the second screening result to obtain a final screening result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the document screening method according to any one of claims 1 to 7 is implemented.

10. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the document screening method according to any one of claims 1 to 7 is implemented.

Images