Literature recommendation method and system in clinical research field and computer program product

Abstract

The invention provides a literature recommendation method and system in the clinical research field and a computer program product, and relates to the technical field of clinical research, and the literature recommendation method comprises the following steps: obtaining demand information input by a user; based on the demand information, a corresponding demand sub-specialized subject term is generated through a first large language model, and the demand sub-specialized subject term comprises a combination of one or more of a demand research field, a demand research direction, a demand research method and a demand research result corresponding to the demand information; determining a matching degree between the required sub-specialized subject term and a literature sub-specialized subject term corresponding to the literature information, wherein the literature sub-specialized subject term comprises one or a combination of more of a research field, a research direction, a research method and a research result corresponding to the literature information; and determining a literature recommendation list based on the matching degree.

Description

Technical Field

[0001] The present invention relates to the field of clinical research technology, and in particular to a method, system and computer program product for recommending literature in the field of clinical research. Background Art

[0002] In the field of clinical research, in order to keep up with the latest clinical medical progress and specialized field knowledge, researchers and clinicians need to keep track of the latest clinical guidelines and journal articles in related fields to obtain cutting-edge research results and clinical experience. Today, the number of journal articles in the field of clinical research is exploding, and traditional literature retrieval and management methods can no longer meet the needs of researchers, resulting in low efficiency in information acquisition and time-consuming and laborious search and reading of papers.

[0003] In view of this, some embodiments of the present specification provide a method, system and computer program product for literature recommendation in the field of clinical research, which can make the literature recommendation process faster, more efficient and more accurate. Summary of the invention

[0004] One or more embodiments of the present specification provide a method for recommending literature in the field of clinical research, the method comprising: obtaining demand information input by a user; generating corresponding demand sub-specialty subject words through a first language model based on the demand information, the demand sub-specialty subject words including a combination of one or more of the demand research field, demand research direction, demand research method, and demand research results corresponding to the demand information; determining the degree of match between the demand sub-specialty subject words and the literature sub-specialty subject words corresponding to the literature information, the literature sub-specialty subject words including a combination of one or more of the research field, research direction, research method, and research results corresponding to the literature information; and determining a literature recommendation list based on the degree of match.

[0005] In some embodiments, based on the demand information, corresponding demand sub-specialty subject terms are generated through the first largest language model, including: based on the semantic analysis of the demand information, corresponding demand prompt words are generated; based on the demand prompt words, demand sub-specialty subject terms are generated through the first largest language model; wherein the first largest language model at least uses the content of the first knowledge database when generating the demand sub-specialty subject terms, and the first knowledge database at least stores research data information related to the demand information.

[0006] In some embodiments, based on the semantic analysis of the demand information, corresponding demand prompt words are generated, including: based on the semantic analysis of the demand information, obtaining the demand type and demand keywords corresponding to the demand information; based on the preset prompt word template and demand keywords corresponding to the demand type, generating demand prompt words.

[0007] In some embodiments, based on the semantic analysis of the demand information, corresponding demand prompt words are generated, including: performing semantic analysis on the demand information through a second largest language model to generate demand prompt words; wherein the second largest language model at least uses the content of the first knowledge database when generating the demand prompt words.

[0008] In some embodiments, the method also includes: generating corresponding literature sub-specialty subject terms based on the acquired literature information, specifically including: generating corresponding literature prompt words based on the literature information; generating corresponding literature sub-specialty subject terms through a third language model based on the literature information and the literature prompt words; wherein the third language model at least uses the content of a second knowledge database when generating literature sub-specialty subject terms, and the second knowledge database at least stores research data information related to the literature information.

[0009] In some embodiments, corresponding document prompt words are generated based on document information, including: obtaining the document type corresponding to the document information, and generating the document prompt words according to a preset prompt word template corresponding to the document type; or analyzing the document information through a fourth language model to generate the document prompt words; wherein the fourth language model at least uses the content of the second knowledge database when generating the document prompt words.

[0010] In some embodiments, the method also includes: obtaining influence information corresponding to the document information, the influence information including a combination of one or more of the publication date of the document information, the publication partition of the document information, the publication journal of the document information, and the impact factor of the publication journal; determining a document recommendation list based on the degree of matching includes: determining a recommended order of the document recommendation list based on the degree of matching and the influence information.

[0011] In some embodiments, the method also includes: marking the document information based on the influence information so that each document information corresponds to at least one influence marking information; when the required sub-specialty subject words include influence marking information, selecting matching document information based on the influence marking information, and determining the degree of matching between the required sub-specialty subject words and the document sub-specialty subject words corresponding to the matching document information; the documents in the document recommendation list are derived from the matching document information.

[0012] One or more embodiments of the present specification also provide a literature recommendation system in the field of clinical research, the system comprising: an acquisition module for acquiring demand information input by a user; a generation module for generating corresponding demand sub-specialty subject words based on the demand information through a first language model, the demand sub-specialty subject words including a combination of one or more of the demand research field, demand research direction, demand research method and demand research results corresponding to the demand information; a matching module for determining the degree of matching between the demand sub-specialty subject words and the literature sub-specialty subject words corresponding to the literature information, wherein the literature sub-specialty subject words include a combination of one or more of the research field, research direction, research method and research results corresponding to the literature information; and a recommendation module for determining a literature recommendation list based on the degree of matching.

[0013] Some embodiments of the present specification also provide a computer program product, including computer instructions or a computer program. When at least part of the computer instructions or the computer program is executed by a processor, it can implement the literature recommendation method in the field of clinical research provided in any embodiment of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] This specification will be further explained by way of exemplary embodiments, which will be described in detail by way of the accompanying drawings. The same numbers in the drawings represent the same structures or steps.

[0015] Figure 1 It is a schematic diagram of a document recommendation system according to some embodiments of this specification.

[0016] Figure 2 It is an exemplary flow chart of a literature recommendation method in the field of clinical research according to some embodiments of this specification.

[0017] Figure 3 This is an exemplary flow chart of generating demand sub-specialty keywords corresponding to demand information through a first language model according to some embodiments of this specification.

[0018] Figure 4 This is an exemplary flow chart for generating sub-specialty subject terms for documents according to some embodiments of this specification.

[0019] Figure 5 It is a data flow diagram of a document recommendation system according to some embodiments of this specification.

[0020] Figure 6 It is a schematic diagram of the functional modules of a literature recommendation system in the field of clinical research according to some embodiments of this specification. DETAILED DESCRIPTION

[0021] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the contents described below are some examples or embodiments of this specification. For ordinary technicians in this field, without paying creative work, the technical solutions or means disclosed in this specification can also be applied to other scenarios based on these technical contents.

[0022] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the above words can be replaced by other expressions.

[0023] Unless otherwise specified, technical terms used in this specification to describe components, elements, etc. do not refer to the singular, but may also include the plural. Generally speaking, terms such as "include", "comprise", etc. only indicate that the steps, elements, or components that have been clearly identified are included, and these steps, elements, and components do not constitute an exclusive list, such as the method or device described may also include other steps or components.

[0024] Flowcharts are used in this specification to illustrate the operation steps performed by the device or system of the relevant embodiments, but unless otherwise specified, the order used to describe these steps should not be understood as a limitation on the order in which the steps are performed. A person of ordinary skill in the art can adjust the order in which these steps are performed based on the knowledge and information conveyed by the embodiments of this specification, and the above adjustment includes but is not limited to swapping the order, merging multiple steps, and splitting a certain step.

[0025] Figure 1 Schematic diagram of a document recommendation system according to some embodiments of this specification. Figure 1 As shown, the document recommendation system 100 may include a processing device 110, a network 120, a storage medium 130, and a terminal device 140. The processing device 110 and the terminal device 140 may perform data transmission via the network 120.

[0026] Among them, the processing device 110 can be a computer device with high computing performance, which is used to obtain updated literature in the field of clinical research, respond to literature recommendation requirements proposed by users, etc. In some embodiments, the processing device 110 can be a single computer device, or a computing cluster composed of multiple computer devices, so as to provide more powerful computing power and more efficient response to the acquisition of literature resources and user's literature recommendation requests. In some embodiments, the processing device 110 can be a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Distribute Network) and big data and artificial intelligence platforms.

[0027] The network 120 may be any form of wired or wireless network, or any combination thereof. As just an example, the network 120 may be one or more combinations of a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, etc. The network 120 may have multiple access points, and the processing device 110 and the terminal device 140 may access the network 120 through the access points.

[0028] The storage medium 130 can be used to store data and / or instructions related to the document recommendation system 100. In some embodiments, the storage medium 130 can store data and / or information obtained from the processing device 110, the terminal device 140, etc. As an example, the storage medium 130 can store crawled document information, etc. In some embodiments, the storage medium 130 may include one or more storage components, each of which may be an independent device or a part of other devices. In some embodiments, the storage medium 130 may be set in the processing device 110. In some embodiments, the storage medium 130 may include a random access memory (RAM), a read-only memory (ROM), a large capacity memory, a removable memory, a volatile read-write memory, etc. or any combination thereof. Exemplarily, the large capacity storage may include a magnetic disk, an optical disk, a solid-state disk, etc. In some embodiments, the storage medium 130 may be implemented on a cloud platform.

[0029] The terminal device 140 may include, but is not limited to, a desktop computer, a smart phone, a laptop computer, a VR (Virtual Reality) device, and a tablet computer. The terminal device 140 may be a document recommendation application client, a browser client or an instant messaging client carrying a document search program and a file recommendation program. The user may install and run the application corresponding to the document recommendation system on the terminal device 140, display the document recommendation list through the terminal device, input the document recommendation requirements through the human-machine interface device, etc., wherein the human-machine interface device may be a part of the terminal device 140, or may be separated from the terminal device 140, but have a signal connection with each other. As an example, the terminal device 140 may have a signal connection with the processing device 110 through the network 120, and the user may send a document recommendation requirement to the processing device 110, obtain the document recommendation list from the processing device 110, and conduct a document search.

[0030] It should be noted that Figure 1 The document recommendation system 100 shown is only an example. The document recommendation system and scenario described in the embodiment of this specification are intended to more clearly illustrate the technical solution of the embodiment of this specification, and do not constitute a limitation on the technical solution provided by the embodiment of this specification. Figure 1 The processing device 110, network 120 and / or storage medium 130 shown can be omitted. It is known to those skilled in the art that with the evolution of document carriers and the emergence of new business scenarios, the technical solutions provided in the embodiments of this specification are also applicable to similar technical problems. For example, in some application scenarios, the processing device 110, storage medium 130 and terminal device 140 can be integrated into one, in which case the network 120 can be omitted. For example, in some application scenarios, the processing device 110 and the terminal device 140 can be integrated into one.

[0031] In the field of clinical research, in order to keep pace with the latest clinical medical progress and specialized field knowledge, researchers and clinicians need to constantly track the latest clinical guidelines and journal literature in related fields to obtain cutting-edge research results and clinical experience. Today, the number of journal literature in the field of clinical research is exploding, and traditional literature retrieval and management methods can no longer meet the needs of researchers, resulting in low efficiency in information acquisition and time-consuming and laborious searching and reading of papers. In order to overcome the above problems, this specification provides a literature recommendation method, system and computer program product in the field of clinical research. Specifically, Figure 2 is an exemplary flow chart of a method for recommending literature in the field of clinical research according to some embodiments of this specification. In some embodiments, Figure 2The process 200 shown may be executed by the processing device 110. In some embodiments, the process 200 may be implemented by a document recommendation system 600 deployed on the processing device 110.

[0032] In some embodiments, Figure 2 As shown, process 200 may include the following steps.

[0033] Step 210 : Obtaining the requirement information input by the user. In some embodiments, step 210 may be implemented by the acquisition module 610 .

[0034] In some embodiments, the demand information input by the user may include the user's knowledge needs in a specific field or a specific direction in the field of clinical research, for example, it may be the clinical response of a certain type of therapeutic drug in the field of clinical research, clinical usage data of a certain type of medical device, diagnosis and treatment methods for a certain type of disease, research results of a certain type of therapeutic drug, updates of clinical guidelines, etc., without limitation here.

[0035] In some embodiments, the user can freely input the demand information according to their own needs. In some embodiments, the user can input the demand information according to the input method supported by the processing device or terminal device deployed in process 200, for example, the user can use an external human-machine interface device such as a mouse and a keyboard to input the text of the demand information, or can use an external human-machine interface device such as a microphone to input the voice of the demand information, which is not limited here.

[0036] In some embodiments, in order to ensure that the demand information input by the user matches the corresponding literature information, the demand information input by the user at a time can be limited to explicitly point to a branch field in the clinical research field. The branch field can be a disease branch, such as gastric malignant tumors, pulmonary micronodules, etc.; it can also be a diagnosis and treatment method branch, such as chemotherapy treatment methods, surgical treatment methods, etc.; it can also be a diagnosis and treatment drug branch, such as antagonist drugs, enzyme inhibitor drugs, etc., which are not limited here. In some embodiments, the demand information input by the user at a time can also point to a combination of different branch fields such as the above-mentioned disease branches, diagnosis and treatment methods, and diagnosis and treatment drugs. For example, it may be necessary to obtain relevant literature information on the clinical use of antagonist drugs on gastric malignant tumors, etc., which is not limited here. In some embodiments, if the user has literature recommendation requirements for multiple different branch fields in the clinical research field, multiple different demand information can be input. The literature recommendation method and system provided in this specification can independently generate and provide literature recommendation lists for different demand information.

[0037] Step 220 : Based on the demand information, generate corresponding demand sub-specialty subject words through the first language model. In some embodiments, step 220 can be implemented by the generation module 620 .

[0038] In some embodiments, a large language model (LLM) is a natural language processing model based on deep learning that has been trained with a large amount of text data and has the ability to process and generate natural language text. In some embodiments, "subspecialty" is a medical term, which generally refers to a more subdivided or more specialized field within a certain specialty field. In the literature recommendation method and system in the field of clinical research provided in this specification, subspecialty can be understood as a subfield or specific direction under a major specialty (such as internal medicine, surgery, oncology, etc.) in the field of clinical research, and these subfields generally involve more specific diseases, treatment methods or research directions. In some embodiments, the demand subspecialty subject term is a paragraph obtained by standardizing and arranging the demand information input by the user. In some embodiments, the generation module 620 can match the closest subspecialty topic according to the demand information freely input by the user, and rewrite, summarize or expand the demand information input by the user in a standardized manner according to the above-mentioned subspecialty topic, so as to accurately understand the subspecialty topic of interest to the user and make literature recommendations based on this. In some embodiments, the demand sub-specialty subject words may specifically include a combination of one or more of the demand research fields, demand research directions, demand research methods, and demand research results corresponding to the demand information. In some embodiments, the demand research fields, demand research directions, demand research methods, and / or demand research results included in the demand sub-specialty subject words may be limited to the sub-specialty level. The technical solutions provided in the embodiments of this specification can make full use of the advantages of the large language model in understanding reasoning and text generation, and can accurately and quickly generate corresponding demand sub-specialty subject words for the demand information freely input by the user. How to use the large language model to generate demand sub-specialty subject words will be specifically described in conjunction with the embodiments below, and will not be repeated here.

[0039] Step 230: Determine the degree of matching between the required sub-specialty subject terms and the document sub-specialty subject terms corresponding to the document information. In some embodiments, step 230 may be implemented by a matching module 630 .

[0040] In some embodiments, the literature information may include literature articles published in academic journals of all levels and types in the field of clinical research, academic books published by publishing houses, conference papers submitted at academic conferences containing research results, technical reports recording clinical research cases or clinical medical device development project related information, clinical diagnosis and treatment guidelines summarized and updated regularly or irregularly, patent application texts and authorization texts containing research results, etc., which are not limited here. In some embodiments, the processing device 110 can collect recently disclosed literature information by crawling data regularly or irregularly. For example, Figure 6 The acquisition module 610 shown can crawl and collect data on newly published literature information in the field of clinical research every natural day. For example, the acquisition module 610 can use natural language processing (NLP) and machine learning algorithms to automatically crawl relevant literature information from multiple well-known clinical research databases and / or paper platforms (such as PubMed, etc.), and determine the literature sub-specialty subject terms for the crawled literature information.

[0041] In some embodiments, the sub-specialty subject words of the literature are a paragraph obtained after a standardized summary of the literature information, which can determine one or more sub-specialty topics corresponding to the literature information, and accurately refine, summarize and generalize the literature information according to the above sub-specialty topics. In some embodiments, the sub-specialty subject words of the literature can specifically include a combination of one or more of the research fields, research directions, research methods and research results corresponding to the literature information. In some embodiments, each document in the literature information can correspond to an independent sub-specialty subject word of the literature, which is used to summarize the literature information in a professional and standardized manner, and can concisely represent the characteristics of the literature information such as the main theme, innovation points, scientific research results, etc. The acquisition of the sub-specialty subject words of the literature will be specifically described in conjunction with the embodiments below, and will not be repeated here.

[0042] In some embodiments, based on the relevant description of the above embodiments, the demand sub-specialty subject words can be used to characterize the real needs of users, and the document sub-specialty subject words can be used to characterize the specific content of documents. Then, by determining the matching degree of the demand sub-specialty subject words and the document sub-specialty subject words, the matching degree between the user's needs and the document content can be characterized by the above matching degree, and the documents with high matching degree can be selected as recommended documents to provide to users. It can be understood that compared with directly using the demand sub-specialty subject words or directly using the input demand information to calculate the matching degree of document information, calculating the matching degree of the demand sub-specialty subject words and the document sub-specialty subject words greatly simplifies the computational load of similarity judgment, improves processing efficiency and recommendation generation speed, and can ensure that the recommended documents accurately meet the actual needs of users, so that the user's document acquisition needs can be responded to in a timely, efficient and accurate manner.

[0043] In some embodiments, the degree of match between the demand subspecialty subject terms and the document subspecialty subject terms can be characterized by the semantic similarity between the demand subspecialty subject terms and the document subspecialty subject terms, wherein the semantic similarity can refer to the degree of similarity between two or more texts or paragraphs in terms of expression meaning, and is usually used to evaluate the relevance and / or similarity between different texts or paragraphs; the higher the semantic similarity between the demand subspecialty subject terms and the document subspecialty subject terms, the higher the semantic similarity between the document information corresponding to the document subspecialty subject terms and the demand information input by the user, and the higher the degree of match between the corresponding demand subspecialty subject terms and the document subspecialty subject terms. In some embodiments, the degree of match can also be achieved according to the actual needs of those skilled in the art by using other methods such as text similarity measurement, edit distance measurement (Levenshtein distance, i.e., calculating the minimum number of editing operations required to convert one string into another, such as insertion, deletion, replacement, etc.), which is not limited here.

[0044] Step 240 : Determine a document recommendation list based on the matching degree. In some embodiments, step 240 may be implemented by a recommendation module 640 .

[0045] In some embodiments, based on the matching degree of the demand subspecialty subject words and the literature subspecialty subject words obtained in the aforementioned step 230, the literature recommendation list can be determined. It can be understood that the higher the matching degree between the demand subspecialty subject words and the literature subspecialty subject words, the higher the matching degree between the literature information corresponding to the literature subspecialty subject words and the demand information input by the user, and the higher the probability that the literature information corresponding to the literature subspecialty subject words contains clinical research field knowledge and / or clinical research field results that the user is interested in. Therefore, the literature information corresponding to the literature subspecialty subject words can be pushed to the user in order of matching degree from high to low to meet the user's literature review needs in the field of clinical research. In some embodiments, the literature information corresponding to the literature subspecialty subject words can be sorted in order of matching degree from high to low, and a literature recommendation list is generated according to the literature name / paper name corresponding to each piece of literature information, and the user can intuitively obtain the literature information that meets his or her needs by consulting the literature recommendation list. In some embodiments, the document recommendation list may include the document names / paper names corresponding to the document information in all clinical research fields. The user can check the document recommendation list through the terminal device, and check the corresponding specific document information by clicking on the document name / paper name in the document recommendation list. In some embodiments, in addition to the document name / paper name corresponding to the document information, the document recommendation list may further include the visual matching degree information of the document sub-specialty subject words and the demand sub-specialty subject words, the publication date of the document information, the publication partition of the document information, the publication journal of the document information, the abstract diagram of the document information, and other document related information of the user's concern, which is not limited here; the user can intuitively select the document information of interest from the document recommendation list through the above-mentioned document related information, combined with the document name / paper name, for further specific reference, thereby improving the user's browsing experience for the document recommendation list. In some embodiments, considering that users often only consult the document information that meets their own demand information, it can also be limited that the matching degree of the document sub-specialty subject words corresponding to the document information in the generated document recommendation list and the demand sub-specialty subject words is greater than a preset threshold, so as to exclude the document information in the clinical research field that does not meet or is irrelevant to the current user's needs, while streamlining the document recommendation list, further improving the user's browsing experience for the document recommendation list.

[0046] Based on the relevant description of the above embodiment, the processing device 110 can generate a document recommendation list according to the demand information input by the user, and because a large language model is introduced in the process of generating the document recommendation list to professionally interpret and / or expand the demand information input by the user, the document information in the generated document recommendation list can be highly matched with the actual needs of the user, so that the user can obtain the latest, high-quality document information that meets their own needs in a timely manner. The specific implementation of the above process 200 will be further described and explained in conjunction with specific embodiments below.

[0047] Figure 3 is an exemplary flow chart of generating demand subspecialty keywords corresponding to demand information through a first language model according to some embodiments of this specification. In some embodiments, Figure 3 The process 300 shown may be executed by the processing device 110. In some embodiments, the process 300 may be implemented by the document recommendation system 600 (such as the generation module 620) deployed on the processing device 110. Figure 3 As shown, process 300 may include the following steps.

[0048] Step 310: Generate corresponding demand prompt words based on semantic analysis of demand information.

[0049] In some embodiments, in the process of generating demand sub-specialty subject words through a large language model, the demand information input by the user can be directly provided to the first large language model for generating demand sub-specialty subject words. However, considering that the demand information input by the user has a high degree of freedom, directly inputting the demand information in the large language model may cause the generation quality of the demand sub-specialty subject words to fluctuate greatly. In some embodiments, in order to improve the generation quality of demand sub-specialty subject words, a relevant demand information filling template or demand information filling guide can be provided to the user when the user inputs the demand information, so that the user can input the demand information according to the demand information filling template or demand information filling guide. In some embodiments, in order to improve the generation quality of demand sub-specialty subject words while taking into account the degree of freedom of the user inputting the demand information, the demand information input by the user can be pre-processed by performing semantic analysis on the demand information and generating corresponding demand prompt words, and the demand prompt words are used to guide the generation of demand sub-specialty subject words, thereby improving the generation quality of demand sub-specialty subject words. In some embodiments, based on the semantic analysis of the demand information, the demand type and demand keywords corresponding to the demand information can be obtained, and then based on the preset prompt word template and demand keywords corresponding to the demand type, the demand prompt words are generated. For example, the demand information input by the user may be "clinical cases of gastric cancer treatment this year". After semantic analysis, the user's demand type is obtained as "consult relevant literature containing clinical cases". The demand keywords are "gastric cancer" and the time span of this year. The corresponding demand prompt words can be expressed as "consult relevant literature containing clinical cases, pay attention to gastric cancer disease, and the time span is the newly disclosed literature information for the whole year of 2025."

[0050] Step 320: Based on the demand prompt words, the demand sub-specialty subject words are generated by the first large language model. It can be understood that the demand prompt words obtained by the above-mentioned embodiment can increase the contextual understanding ability of the large language model for the demand information input by the user, and can guide the large language model to rewrite, summarize or expand the demand information in a standardized manner, so that the demand sub-specialty subject words generated by the large language model can clearly represent the sub-specialty subject corresponding to the demand information. In some embodiments, taking the user-input demand information of "treatment plan in the field of gastric cancer" as an example, the large language model can expand the "gastric cancer" in the user-input demand information into "malignant tumor of the stomach" during the generation of demand sub-specialty keywords, and obtain multiple standardized disease occurrence sites according to the site of disease occurrence, such as "gastric body", "gastric fundus", "cardia", "pylorus", "lesser curvature of stomach", "greater curvature of stomach", etc. At the same time, multiple standardized disease names are obtained according to the histological type of malignant tumors, such as "adenocarcinoma", "lymphoma", "mesothelioma", etc. The demand sub-specialty keywords finally obtained may at least include: "Literature on treatment plans for gastric malignant tumor diseases, wherein the disease occurrence sites involved in the literature may include gastric body, gastric fundus, cardia, pylorus, lesser curvature of stomach, greater curvature of stomach, etc., and the disease names involved in the literature may include adenocarcinoma, lymphoma, mesothelioma, etc...." Compared with the traditional method of directly conducting full-text retrieval and / or semantic analysis on the demand information and literature information input by the user, the use of a large language model to standardize the demand information and generate demand sub-specialty subject terms can avoid missing relevant literature on "pyloric malignant tumor" or "cardiac malignant tumor" due to differences in specific word expressions during the literature recommendation process, thereby causing the problem of omission of literature recommendation. While ensuring the accuracy of literature recommendation, it also improves the coverage of literature recommendation, allowing users to fully obtain updated literature content under the sub-specialty topics of interest in the field of clinical research.

[0051] In some embodiments, considering that there are a large amount of professional knowledge and academic terms in the field of clinical research, in order to avoid the problem of hallucination when the large language model processes the professional knowledge in the field of clinical research, that is, although the content generated by the large language model looks reasonable on the surface, the actual content is fictitious or inaccurate information, the first knowledge database containing relevant knowledge information or research data information in the field of clinical research can be accessed in the first large language model. In some embodiments, in the process of generating the demand sub-specialty subject words through the first large language model, at least the content of the first knowledge database is used, so that the generated demand sub-specialty subject words can be output according to the content of the first knowledge database, wherein the first knowledge database at least stores research data information related to the demand information, for example, it can include preset clinical trial plans, clinical guidelines, journal literature data, etc., which are not limited here. In some embodiments, the research data information stored in the first knowledge database and the relevant knowledge information in the field of clinical research can be updated regularly or irregularly, so as to effectively avoid the problem of hallucination when the large language model generates sub-specialty subject words due to the emergence of new professional knowledge.

[0052] In some embodiments, in the specific implementation process of the aforementioned step 310, in addition to generating and optimizing demand prompt words according to preset prompt word templates and demand keywords, a large language model can also be used to generate and optimize demand prompt words. In some embodiments, the demand information can be semantically analyzed by a second large language model to generate demand prompt words, wherein the second large language model can be the same large language model as the first large language model, or it can be two different large language models independent of the first large language model, which is not limited here. In some embodiments, considering that the second large language model is mainly used to convert the demand information input by the user into demand prompt words, the model scale of the second large language model can be smaller than the model scale of the first large language scale. In some embodiments, in order to avoid the problem of processing hallucinations in the process of generating demand prompt words by the second large language model, the content of the first knowledge database provided in the aforementioned embodiment can be also used in the process of generating demand prompt words by the second large language model, which is not repeated here.

[0053] Based on the relevant description of the above embodiments, the processing device 110 can use the large language model to process the user input demand information and generate standardized demand sub-specialty subject words that can fully reflect the actual needs of the user. The following will describe and explain the acquisition of literature sub-specialty subject words in conjunction with specific embodiments.

[0054] In some embodiments, the document recommendation method and system provided in this specification can generate corresponding document sub-specialty subject terms based on the acquired document information. In some embodiments, in the process of generating document sub-specialty subject terms, a pre-trained model can be used to extract and summarize the main information of the document. For example, a pre-trained T5 (Text-to-Text Transfer Transformer) model can be used to generate a summary of the document information, or a Graph Neural Networks (GNNs) model can be used to process the structural information in the document information, which is particularly suitable for summarizing long and complex documents. In some embodiments, in the process of generating document sub-specialty subject terms, the large language model involved in the aforementioned embodiments can also be used for implementation. Specifically, Figure 4 is an exemplary flow chart for generating sub-specialty subject terms of a document according to some embodiments of this specification. In some embodiments, Figure 4 The process 400 shown may be executed by the processing device 110. In some embodiments, the process 400 may be implemented by the document recommendation system 600 (such as the generation module 620) deployed on the processing device 110. Figure 4 As shown, process 400 may include the following steps.

[0055] Step 410: Generate corresponding document prompt words based on the document information.

[0056] In some embodiments, in the process of using a large language model to generate document sub-specialty subject words, the generation quality of document sub-specialty subject words can be improved by first generating document prompt words. In some embodiments, in the process of generating document prompt words, the document type corresponding to the document information can be obtained first, and then the document prompt words can be generated according to the preset prompt word template corresponding to the document type. In some embodiments, the documents can be divided into review documents, experimental documents and communication documents according to the specific content corresponding to the documents, wherein the review documents are mainly used to summarize and evaluate the existing research in a certain field or theme, provide comprehensive background information and the direction of future research, and users are more concerned about the sample statistics therein, so the preset prompt word template corresponding to the review documents may include "summarize the sample statistics in the document" and other similar expressions; experimental documents are mainly used to report new experimental research results, introduce experimental design, methods, results and discussions, and users are more concerned about the experimental / statistical methods, experimental data and experimental results in the documents, so the preset prompt word template corresponding to the experimental documents may include "summarize the experimental / statistical methods, experimental data and experimental results in the document" and other similar expressions; communication documents are mainly used to quickly disseminate important research results or new discoveries, and users are more concerned about the key information, main discoveries and innovations therein, so the preset prompt word template corresponding to the communication documents may include "summarize the main discoveries and innovations in the document" and other similar expressions. Those skilled in the art can also classify documents and set the corresponding preset prompt word templates according to actual needs, which are not limited here.

[0057] Step 420: Generate corresponding document sub-specialty subject terms through the third language model based on the document information and document prompt words. In some embodiments, the third language model can be the same large language model as the first language model, or two different large language models that are independent of the first language model, which is not limited here. In some embodiments, the same large language model can be used as the third language model in the current embodiment and the first language model in the aforementioned embodiment, so that the generation of document sub-specialty subject terms and demand sub-specialty subject terms can be realized simultaneously using a large language model, saving the deployment and use costs of the large language model.

[0058] In some embodiments, also considering that there are a large number of professional knowledge and academic terms in the field of clinical research, in order to avoid the problem of processing hallucinations, a second knowledge database containing knowledge information or research data information related to the field of clinical research can be accessed in the third language model. In some embodiments, in the process of generating sub-specialty subject words of the literature through the third language model, at least the content of the second knowledge database is used, so that the generated sub-specialty subject words of the literature can be output according to the content of the second knowledge database, wherein the second knowledge database at least stores research data information related to the literature information, for example, it may include preset clinical trial programs, clinical guidelines, journal literature data, etc., which are not limited here. In some embodiments, the second knowledge database and the first knowledge database provided in the aforementioned embodiment can be the same knowledge database, or they can be two independent and different knowledge databases, which are not limited here. In some embodiments, the research data information stored in the second knowledge database and the relevant knowledge information in the field of clinical research can be updated regularly or irregularly, thereby effectively avoiding the problem of processing hallucinations in the large language model due to the emergence of new professional knowledge during the generation of sub-specialty subject words of the literature.

[0059] In some embodiments, a large language model is used to summarize and generalize the literature information in a standardized manner, so that the literature sub-specialty subject terms generated by the large language model can clearly characterize the sub-specialty topics corresponding to the literature information. In some embodiments, taking the literature information "Preparation of implantable bone healing auxiliary devices using hydroxyapatite (HA) and tricalcium phosphate (TCP) as materials" as an example, the large language model will determine that the sub-specialty topic corresponding to the literature information is the field of "advanced orthopedic material technology" based on the literature prompt words and the full-text semantic understanding of the literature information during the generation of literature sub-specialty subject terms. The final obtained literature sub-specialty subject terms can at least include: "The literature involves advanced orthopedic material technology, and the specific materials used involve hydroxyapatite (HA) and tricalcium phosphate (TCP)...", which can achieve accurate summarization of different sub-specialty topics involved in the literature information, and facilitate subsequent matching with the required sub-specialty subject terms.

[0060] In some embodiments, in the specific implementation process of the aforementioned step 410, in addition to generating and optimizing document prompt words according to the preset prompt word template and document type, the large language model can also be used to generate and optimize document prompt words. In some embodiments, the demand information can be semantically analyzed by the fourth large language model to generate demand prompt words, wherein the fourth large language model can be a large language model that is the same as one or more of the large language models in the aforementioned embodiments, or a large language model that is independent of the large language models in the aforementioned embodiments, which is not limited here. In some embodiments, in order to avoid the problem of processing hallucinations in the process of generating document prompt words by the fourth large language model, the content of the second knowledge database provided in the aforementioned embodiment can also be used in the process of generating demand prompt words by the fourth large language model, which is not repeated here.

[0061] Figure 5 is a data flow diagram of a document recommendation system according to some embodiments of this specification. Figure 5 As shown, the system 500 may include a document information acquisition module 510, which is used to crawl updated document information regularly or irregularly; a demand information acquisition module 520, which is used to obtain demand information input by a user; a prompt word generation module 530, which is connected to the document information acquisition module 510 and the demand information acquisition module 520, respectively, and is used to independently generate and / or optimize document prompt words and demand prompt words for the acquired document information and / or demand information, and output the optimized document prompt words and / or demand prompt words; a large language model 540 is used to receive the document prompt words and / or demand prompt words input by the prompt word generation module 530, and generate and / or optimize the document prompt words and / or demand prompt words through the large language model; The natural language processing capability outputs literature sub-specialty subject terms and / or demand sub-specialty subject terms; in the process of generating literature sub-specialty subject terms and / or demand sub-specialty subject terms by the large language model 540, the professional knowledge and academic terms in the clinical research field are obtained by accessing the clinical research knowledge base 550 to avoid processing hallucination problems in the generation process; the literature recommendation module 560 is used to receive literature sub-specialty subject terms and / or demand sub-specialty subject terms output by the large language model 540, and determine the degree of matching between the literature sub-specialty subject terms and the demand sub-specialty subject terms, and finally generate a literature recommendation list for users to obtain and consult based on the determined degree of matching.

[0062] In some embodiments, Figure 5In the system 500 shown, the document information acquisition module 510 can acquire updated document information regularly or irregularly, while the demand information acquisition module 520 only acquires demand information when the user updates the input demand information; for the prompt word generation module 530 and the large language model 540, the generation of document prompt words, document sub-specialty subject words, demand prompt words, and demand sub-specialty subject words are all performed independently; for the document recommendation module 560, when the user has not updated the input demand information, it can be matched according to the historical demand sub-specialty subject words that the user has most recently updated. In some embodiments, when the document information acquisition module 510 acquires updated document information once every natural day, the document recommendation module 560 can update the document recommendation list according to the updated document sub-specialty subject words every natural day; when the user updates and inputs new demand information, the document recommendation module 560 will update the document recommendation list in real time according to the updated demand sub-specialty subject words every natural day.

[0063] Based on the relevant descriptions of the foregoing embodiments, the literature recommendation method and system provided in this specification can obtain the demand sub-specialty subject terms and the literature sub-specialty subject terms, and determine the literature recommendation list according to the matching degree between the demand sub-specialty subject terms and the literature sub-specialty subject terms. Further, considering that users often pay more attention to the latest clinical medical progress and specialty field knowledge in the clinical research field, and at the same time require that the obtained clinical medical progress and specialty field knowledge have a certain degree of authority, influence information corresponding to the literature information can also be introduced in the process of determining the literature recommendation list. In some embodiments, for the literature recommendation method and system provided in this specification, while obtaining the literature information, the influence information corresponding to the literature information can also be synchronously obtained, where the influence information can specifically include one or a combination of multiple items such as the publication date of the literature information, the publication section of the literature information, the publication journal of the literature information, and the impact factor of the publication journal. Those skilled in the art can also select other contents as the influence information according to actual needs, such as the number of citations of the literature information, etc., which is not limited herein; correspondingly, in the process of determining the literature recommendation list, the recommendation ranking of the literature recommendation list can be determined based on the matching degree and the influence information at the same time. In some embodiments, it can be understood that the higher the matching degree, the higher the recommendation ranking of the corresponding literature in the literature recommendation list; the higher the influence level of the literature, the higher the recommendation ranking of the corresponding literature in the literature recommendation list, where the influence level of the literature can be determined according to the influence information of the literature. For example, the closer the publication date of the literature information is, it indicates that the research result corresponding to the literature information is newly obtained, and its corresponding influence level is higher; the more authoritative the publication journal of the literature information is, it indicates that the research result corresponding to the literature information has been reviewed and evaluated by authoritative persons, and its corresponding influence level is also higher. Those skilled in the art can determine the influence level based on the influence information according to actual needs, which is not limited herein. In the process of determining the literature recommendation list by comprehensively considering the matching degree and the influence information, different weights can be assigned to the matching degree and the influence information, and the specific recommendation ranking of the literature can be determined by means of weighting, which is not limited herein.

[0064] In some embodiments, in addition to using the influence information of the document information to optimize the recommended sorting of the document recommendation list, the document recommendation system can also support users to filter the documents in the document recommendation list by the influence information of the document information. In some embodiments, the document information can be marked based on the obtained influence information so that each document information corresponds to at least one influence marking information, such as the publication journal marking corresponding to the document information, the publication partition marking corresponding to the document information, etc.; when the demand sub-specialty subject words include influence marking information, matching document information can be selected based on the influence marking information, and then the matching degree between the demand sub-specialty subject words and the document sub-specialty subject words corresponding to the matching document information can be determined, so that the documents in the document recommendation list are all derived from the matching document information. In some embodiments, for example, when the user inputs the demand information, the description content "needs to track the paper documents in the Q1 partition of the clinical orthopedic treatment field" is included, then the corresponding demand sub-specialty subject words can include the influence marking information "limited to focus on the document information in the Q1 partition under the JCR partition (Journal Citation Reports)", at this time, the document information belonging to the Q1 partition under the JCR partition can be preferentially screened out according to the above influence marking information, and the screened document information can be recommended and sorted to meet the actual needs of the user.

[0065] Some embodiments of the present specification also provide a literature recommendation system in the field of clinical research. Figure 6 is a functional module diagram of a literature recommendation system in the field of clinical research according to some embodiments of this specification. Figure 6 As shown, the document recommendation system 600 may include an acquisition module 610 , a generation module 620 , a matching module 630 , and a recommendation module 640 . Each module of the document recommendation system 600 may be configured in the processing device 110 .

[0066] In some embodiments, the acquisition module 610 can be used to acquire data and / or information during the execution of the document recommendation method of the embodiment of this specification. In some embodiments, the acquisition module 610 can be used to acquire demand information input by a user. In some embodiments, the acquisition module 610 can be used to acquire document information. In some embodiments, the acquisition module 610 can include a document information acquisition module 510 and a demand information acquisition module 520.

[0067] In some embodiments, the generation module 620 can be used to generate relevant contents such as prompt words and subject words. In some embodiments, the generation module 620 can be used to generate corresponding demand sub-specialty subject words based on demand information through the first large language model, wherein the demand sub-specialty subject words can specifically include one or more combinations of demand research fields, demand research directions, demand research methods and demand research results corresponding to the demand information. In some embodiments, the generation module 620 can generate corresponding demand prompt words based on semantic analysis of demand information. In some embodiments, the generation module 620 can generate demand sub-specialty subject words based on demand prompt words through the first large language model. In some embodiments, the generation module 620 can generate corresponding document prompt words based on document information. In some embodiments, the generation module 620 can generate corresponding document sub-specialty subject words based on document information and document prompt words through the third large language model. In some embodiments, the generation module 620 can include a prompt word generation module 530 and / or a large language model 540.

[0068] In some embodiments, the matching module 630 can be used to match the demand subspecialty subject words with the literature subspecialty subject words. In some embodiments, the matching module 630 can be used to determine the degree of matching between the demand subspecialty subject words and the literature subspecialty subject words corresponding to the literature information, wherein the literature subspecialty subject words can specifically include one or more combinations of the research field, research direction, research method and research results corresponding to the literature information.

[0069] In some embodiments, the recommendation module 640 may be used to implement document recommendation. In some embodiments, the recommendation module 640 may be used to determine a document recommendation list based on a matching degree. In some embodiments, the recommendation module 640 may include the document recommendation module 560.

[0070] For more information about each module, see Figures 2 to 5 The relevant description of will not be repeated here. It should be understood that Figure 6The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will understand that the above methods and systems can be implemented using computer executable instructions and / or control codes contained in a processor, such as a disk, CD or DVD-ROM, etc., and such codes are provided in a carrier medium or a memory of a programmable device. The system and its modules of this specification can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above hardware circuits and software (e.g., firmware).

[0071] It should be noted that the above description of the system and its modules is only for convenience of description and does not limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules without deviating from this principle to form a subsystem connected to other modules. Or some modules may be split to obtain more modules or multiple units under the module. Such variations are all within the scope of the disclosure of this specification.

[0072] Some embodiments of the present specification also provide a literature recommendation device in the field of clinical research, which includes a processor and a storage medium, wherein the storage medium stores computer program instructions, and the processor is used to execute at least part of the computer program instructions to implement the literature recommendation method in the field of clinical research provided in the aforementioned embodiments of the present specification.

[0073] Some embodiments of the present specification also provide a computer program product, including computer instructions or a computer program. When at least part of the computer instructions or the computer program is executed by a processor, it can implement the literature recommendation method in the field of clinical research provided in the aforementioned embodiments of the present specification.

[0074] In some embodiments, the processor may be a combination of one or more of the following processors: a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a programmable logic controller (PLC), a reduced instruction set computer (RISC), a microprocessor, etc. In some embodiments, the processor may be the processing device 110.

[0075] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) The demand sub-specialty subject terms and document sub-specialty subject terms can be generated according to the demand information input by the user and the literature information obtained by crawling, and the document recommendation list is generated by matching the demand sub-specialty subject terms with the document sub-specialty subject terms, which greatly simplifies the matching calculation and makes the recommendation process faster and more efficient. (2) In the process of generating demand sub-specialty subject terms, a large language model is introduced to perform standardized interpretation and / or expansion of the demand information input by the user, while supporting the user to freely input the demand, ensuring the accuracy of the document recommendation, and improving the coverage of the document recommendation, so that the user can fully obtain the updated document content under the sub-specialty topic of interest in the clinical research field. (3) In the process of generating document sub-specialty subject terms, a large language model is introduced to accurately obtain the content theme, innovation points and other features of the corresponding document. (4) A knowledge database containing knowledge information or research data information related to the clinical research field is connected to the large language model, which effectively avoids the problem of processing hallucinations caused by professional knowledge and academic terms in the clinical research field. (5) In the process of matching the required sub-specialty subject terms with the literature sub-specialty subject terms, the influence information corresponding to the literature information is introduced to support users to independently select the information quality of the recommended literature, and enable users to preferentially obtain the recommended literature with recent publication time and high quality. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.

[0076] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are taught in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

Claims

1. A method for recommending literature in the field of clinical research, characterized in that: The method comprises: Obtain the demand information input by the user; Based on the demand information, corresponding demand sub-specialty subject words are generated through the first language model, and the demand sub-specialty subject words include a combination of one or more of the demand research field, demand research direction, demand research method and demand research results corresponding to the demand information; Determine the degree of match between the required sub-specialty subject terms and the literature sub-specialty subject terms corresponding to the literature information, wherein the literature sub-specialty subject terms include one or more combinations of the research field, research direction, research method and research results corresponding to the literature information; A document recommendation list is determined based on the matching degree.

2. The method according to claim 1, characterized in that Based on the demand information, the corresponding demand sub-specialty subject words are generated by the first language model, including: Based on the semantic analysis of the demand information, generate corresponding demand prompt words; Based on the demand prompt words, generating the demand subspecialty subject words through the first language model; Among them, the first language model at least uses the content of the first knowledge database when generating the demand sub-specialty subject terms, and the first knowledge database at least stores research data information related to the demand information.

3. The method according to claim 2, characterized in that The generating corresponding demand prompt words based on the semantic analysis of the demand information includes: Based on the semantic analysis of the demand information, obtaining the demand type and demand keywords corresponding to the demand information; The demand prompt word is generated based on a preset prompt word template corresponding to the demand type and the demand keyword.

4. The method according to claim 2, characterized in that: The generating corresponding demand prompt words based on the semantic analysis of the demand information includes: Performing semantic analysis on the demand information through a second language model to generate the demand prompt words; The second language model at least uses the content of the first knowledge database when generating the demand prompt word.

5. The method according to claim 1, characterized in that The method further includes: generating corresponding sub-specialty subject words of the literature based on the acquired literature information, specifically including: Based on the document information, generate corresponding document prompt words; Based on the document information and the document prompt words, the corresponding sub-specialty subject words of the document are generated through the third language model; The third language model at least uses the content of the second knowledge database when generating the sub-specialty subject terms of the document, and the second knowledge database at least stores research data information related to the document information.

6. The method according to claim 5, characterized in that The step of generating corresponding document prompt words based on the document information includes: Acquire the document type corresponding to the document information, and generate the document prompt word according to a preset prompt word template corresponding to the document type; or Analyzing the document information by using a fourth language model to generate the document prompt words; The fourth language model at least uses the content of the second knowledge database when generating the document prompt word.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining influence information corresponding to the document information, the influence information including one or more combinations of the publication date of the document information, the publication partition of the document information, the publication journal of the document information, and the impact factor of the publication journal; Determining a document recommendation list based on the matching degree includes: determining a recommendation ranking of the document recommendation list based on the matching degree and the influence information.

8. The method according to claim 7, characterized in that The method further comprises: Marking the document information based on the influence information so that each document information corresponds to at least one influence marking information; When the demand subspecialty subject term includes the influence mark information, selecting matching document information based on the influence mark information, and determining the matching degree between the demand subspecialty subject term and the document subspecialty subject term corresponding to the matching document information; The documents in the document recommendation list are derived from the matching document information.

9. A literature recommendation system in the field of clinical research, characterized in that: The system comprises: The acquisition module is used to obtain the demand information input by the user; A generating module, configured to generate corresponding demand sub-specialty subject words based on the demand information through a first language model, wherein the demand sub-specialty subject words include a combination of one or more of the demand research field, demand research direction, demand research method, and demand research results corresponding to the demand information; A matching module, used to determine the matching degree between the required sub-specialty subject words and the literature sub-specialty subject words corresponding to the literature information, wherein the literature sub-specialty subject words include a combination of one or more of the research field, research direction, research method and research results corresponding to the literature information; A recommendation module is used to determine a document recommendation list based on the matching degree.

10. A computer program product comprising computer instructions or a computer program, characterized in that When at least part of the computer instructions or the computer program is executed by a processor, the method for recommending literature in the field of clinical research as claimed in any one of claims 1 to 8 can be implemented.

Images