Literature classification system, literature classification device, and literature classification method

Abstract

This system enables highly accurate extraction of topics and concepts desired by the user from a collection of literature categorized according to the user's interests. [Solution] The literature classification system 100 includes a storage device 13 that holds knowledge information about literature information, and a computing device 11 that performs a first clustering process of literature using a first combined feature vector obtained by combining feature quantities of literature based on user-specified keywords and feature quantities of text in the literature that contain the keywords, and a second clustering process that performs clustering of the literature using a second combined feature vector based on feature quantities of literature included in the cluster.

Description

Technical Field

[0001] The present invention generally relates to the technology of a document classification system, a document classification apparatus, and a document classification method. Specifically, it relates to a technology capable of accurately extracting topics and concepts desired by a user from a group of documents classified from the viewpoints of interest to the user.

Background Art

[0002] The purpose of using document classification technology is to reach the target document in a short time by classifying a huge collection of documents by topic. Document clustering used in such document classification technology is a method of expressing each document as a vector based on words in the document and clustering using the distance or cosine between each document as a similarity measure.

[0003] As a result of the above clustering, the document set to be clustered is divided into several clusters consisting of mutually similar documents. Also, representative topics and topics included in the document set can be obtained from the document sets of each such cluster. By checking the representative topics and topics of each such cluster, the user can easily reach the target document without reading all the documents.

[0004] As such a document clustering technology, for example, in Patent Document 1, a concept extraction technology capable of accurately extracting various topics and concepts of text data by repeating clustering while reflecting the intention of the user is disclosed.

[0005] The above technology relates to a concept extraction system that extracts concepts from text data by interactively classifying multiple text data sets multiple times, and is characterized by comprising: a text vector generation means that counts the frequency of occurrence of each language appearing in the text data for each text data set and generates matrix information of each text data set and each language with the frequency of occurrence as matrix elements; a text vector storage means that stores the matrix information; a clustering means that classifies each text data into multiple clusters based on the matrix information stored in the text vector storage means; a cluster selection means that causes a subset of clusters to be selected from the multiple clusters; a selected cluster storage means that stores cluster identification information indicating the selected clusters; and a text vector modification means that modifies the matrix information stored in the text vector storage means based on the cluster identification information.

[0006] Furthermore, Patent Document 2 discloses a document classification device that generates a classification structure suitable for the user's perspective and enables classification and analysis that meets the user's objectives.

[0007] The above technology includes a document storage unit for storing document data, a category storage unit for storing a hierarchical structure of categories and classification rules for classifying the document data into the categories, a category operation unit that receives input of a category to be used as a classification perspective and a target category which is the category to be classified, and reads a set of axis categories which are lower categories of the category to be used as a classification perspective from the category storage unit as an axis category set, a feature value calculation unit that calculates the feature value of words included in the target document data set, which is a set of document data from the document storage unit that satisfies the classification rules of the target category, and selects the words that represent the characteristics of the document based on the feature value calculated by the feature value calculation unit, and for each of the selected words, calculates the feature value based on the frequency of occurrence of the word in the target document data set that satisfies the classification rules of each axis category in the axis category set. A trend vector generation unit calculates a metric and generates a trend vector in which the statistic is set as the value of an element corresponding to the axis category; a clustering unit clusters the words based on the similarity of the trend vectors generated by the trend vector generation unit; a category generation processing unit generates a feature word category for each cluster obtained as a result of clustering by the clustering unit, with the target category as the higher-level category and a classification rule using the words belonging to the cluster as filter words, and registers it in the category storage unit; a two-axis map display unit displays information in each cell of a two-axis map in which the axis category is the classification item of the first axis and the feature word category is the classification item of the second axis, representing the number of document data stored in the document storage unit that satisfy the classification rule of the axis category corresponding to the cell and the classification rule of the feature word category corresponding to the cell. The present invention relates to a document classification device characterized by comprising the following: [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2004-341948 [Patent Document 2] Patent No. 6173848 [Overview of the project] [Problems that the invention aims to solve]

[0009] In the conventional technology described above, the technology shown in Patent Document 1 presents the clustering results to the user, and the user repeatedly selects clusters that they judge to have useful concepts and performs clustering again. However, each such clustering requires a considerable amount of computation time. Therefore, after selecting clusters, the user has to wait until the results of re-clustering are available. Furthermore, when presenting the clustering results, the clusters are sorted using importance scores, and clusters with many affiliated texts and high cluster cohesion are displayed at the top. As a result, the clustering results presented to the user tend to be sorted by data cohesion rather than reflecting the user's interests.

[0010] Furthermore, in the technology described in Patent Document 2, while it is possible to assign a classification perspective desired by the user during clustering, such perspectives are limited to those assigned by the user.

[0011] Therefore, the present invention has been made in view of the above problems, and aims to provide a technology that can extract topics and concepts desired by a user with high accuracy from a collection of literature classified according to the user's interests. [Means for solving the problem]

[0012] The present invention includes several means for solving the above problems, but an example thereof is as follows. To solve the above problems, a literature classification system according to one aspect of the present invention is a literature classification system for classifying a collection of literature, and includes a storage device that holds knowledge information relating to literature information; a first feature summing process that combines feature quantities of literature searched based on user-specified keywords with feature quantities of text containing the keywords in the literature; a first clustering process that performs clustering of the literature using a first combined feature vector created by the first feature summing process; a concept extension keyword identification process that matches the keywords with the knowledge information and identifies concept extension keywords; a second feature summing process that combines feature quantities relating to literature included in at least one cluster generated by the first clustering process, feature quantities of text containing the keywords in the literature, and feature quantities relating to the concept extension keywords; and a second clustering process that performs clustering of the literature using a second combined feature vector created by the second feature summing process.

[0013] Furthermore, in order to solve the above problems, a literature classification device according to one aspect of the present invention is a literature classification device for classifying a collection of literature, and is characterized by including a storage device that holds knowledge information relating to literature information, a first feature summing process that combines feature quantities of literature searched based on user-specified keywords and feature quantities of text containing the keywords in the literature, a first clustering process that performs clustering of the literature using a first combined feature vector created by the first feature summing process, a concept extension keyword identification process that matches the keywords with the knowledge information and identifies concept extension keywords, a second feature summing process that combines feature quantities relating to literature included in at least one cluster generated by the first clustering process, feature quantities of text containing the keywords in the literature, and feature quantities relating to the concept extension keywords, and a second clustering process that performs clustering of the literature using a second combined feature vector created by the second feature summing process.

[0014] Furthermore, in order to solve the above problems, a literature classification method according to one aspect of the present invention is characterized in that a literature classification system for classifying a set of literature stores knowledge information relating to the literature information in a storage device, performs a first feature merging process which combines the feature quantities of literature searched based on user-specified keywords with the feature quantities of text containing the keywords in the literature, performs a first clustering process which clusters the literature using a first merged feature vector created by the first feature merging process, performs a concept extension keyword identification process which matches the keywords with the knowledge information and identifies concept extension keywords, performs a second feature merging process which combines the feature quantities relating to the literature included in at least one cluster generated by the first clustering process, the feature quantities of text containing the keywords in the literature, and the feature quantities relating to the concept extension keywords, and performs a second clustering process which clusters the literature using a second merged feature vector created by the second feature merging process.

Effect of the Invention

[0015] According to the present invention, it becomes possible to extract, with high precision, topics and concepts desired by the user from a group of documents classified from the viewpoints interesting to the user.

Brief Description of the Drawings

[0016] [Figure 1] It is a diagram showing an example of the network configuration of the document classification system in the present embodiment. [Figure 2] It is a diagram showing an example of the hardware configuration of the server (document classification device) in the present embodiment. [Figure 3] It is a diagram showing an example of the flow of the document classification method in the present embodiment. [Figure 4] It is a diagram showing an example of the configuration of the document information DB in the present embodiment. [Figure 5] It is a diagram showing an example of the flow of the document classification method in the present embodiment. [Figure 6] It is a diagram showing an example of the configuration of the thesaurus information DB in the present embodiment. [Figure 7] It is a diagram showing an example of the configuration of the knowledge information DB in the present embodiment. [Figure 8] It is a diagram showing an example of the flow of the document classification method in the present embodiment.

Mode for Carrying Out the Invention

[0017] In the following description, the communication device may be one or more communication interface devices. The one or more communication interface devices may be one or more of the same type of communication interface devices (for example, one or more NICs (Network Interface Cards)), or two or more different types of communication interface devices (for example, a NIC and an HBA (Host Bus Adapter)).

[0018] Also, in the following description, "memory" is one or more memory devices, which are examples of one or more storage devices. At least one of the memory devices in the memory may be a volatile memory device or a non-volatile memory device.

[0019] Also, in the following description, "storage device" may be one or more persistent storage devices, which are examples of one or more storage devices. Persistent storage devices are typically non-volatile storage devices, and specifically, for example, may be HDD (Hard Disk Drive), SSD (Solid State Drive), or NVMe (Non-Volatile Memory Express) drive.

[0020] Also, in the following description, "CPU" is an arithmetic unit and may be one or more processor devices. At least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may also be other types of processor devices such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core. At least one processor device may be a processor core. At least one processor device may also be a processor device in a broad sense that performs part or all of the processing, such as a hardware circuit (e.g., FPGA (Field-Programmable Gate Array), CPLD (Complex Programmable Logic Device), or ASIC (Application Specific Integrated Circuit)).

[0021] Furthermore, in the following explanation, we may use expressions such as "xxx table" or "xxx database" to describe information from which an output is obtained for a given input. This information can be data of any structure (for example, structured data or unstructured data), or it can be a neural network that generates an output for a given input, a learning model such as a genetic algorithm or a random forest. Therefore, "xxx table" or "xxx database" can be referred to as "xxx information." Also, in the following explanation, the configuration of each database or table is just an example, and one database or table may be divided into two or more databases or tables, or all or part of two or more databases or tables may be a single database or table.

[0022] Furthermore, in the following explanation, the subject of the process may be "program," but since a program is executed by the CPU and performs defined processes using memory and / or interface devices as appropriate, the subject of the process may also be the CPU (or a device such as a controller having that processor). A program may be installed from a program source into a device such as a computer. The program source may be, for example, a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Also, in the following explanation, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0023] Furthermore, in the following explanation, when describing similar elements without distinction, the common part of the reference code may be used, and when describing similar elements with distinction, the reference code or element identifier may be used.

[0024] <Regarding the network configuration of the literature classification system> Figure 1 shows an example of the network configuration of the literature classification system 1 in this embodiment. The literature classification system 1 in this embodiment is a system that can extract topics and concepts desired by the user with high accuracy from a group of literature classified according to the user's interests. For example, for a user who wants to plan and create a clinical trial plan while referring to various literature, this literature classification system 1 not only searches and presents useful literature through simple keyword-based clustering, but also efficiently and accurately searches and provides the literature (information) that the user actually desires, taking into account the hierarchy of its concepts.

[0025] The literature classification system 1 of this embodiment consists of a server 10, which is a literature classification device, and a user terminal 20. The server 10 and the user terminal 20 are assumed to be connected via an appropriate network N that enables communication. The server 10 is an information processing device that primarily executes the literature classification method of this embodiment. The user terminal 20 is a client operated by a person who plans and creates a clinical trial plan, or a person who assists such a person, and who appropriately utilizes the literature clustered by the server 10.

[0026] As shown in Figure 2, the server 10 comprises a computing unit 11, memory 12, storage device 13, and communication interface 14. The computing unit 11 is a processor that implements the literature classification processing unit 131, which will be described later. The memory 12 is a volatile storage device that temporarily stores input / output data associated with each process in the literature classification processing unit 131, which will be described later. The storage device 13 holds programs corresponding to the literature classification processing unit 131, as well as a literature information DB 132, a thesaurus information DB 133, and a knowledge information DB 134.

[0027] The above-mentioned literature classification processing unit 131 comprises a literature download processing unit 1311, a text extraction processing unit 1312, a target keyword text extraction processing unit 1313, a concept extension keyword creation processing unit 1314, a feature vector transformation processing unit 1325, a feature vector merging processing unit 1316, and a clustering processing unit 1317.

[0028] Of these, the literature download processing unit 1311 is a processing unit that downloads literature necessary for creating a clinical trial plan from an appropriate external system, etc., and stores it in the literature information DB 132, which will be described later. The above-mentioned external system, etc., includes, but is not limited to, systems that publish various types of literature such as academic papers and clinical trial results.

[0029] Furthermore, the text extraction processing unit 1312 is a processing unit that extracts text to be used for clustering from the documents downloaded by the document download processing unit 1311. The target keyword text extraction processing unit 1313 is a processing unit that extracts the target words (keywords) specified by the user from each document in the set of documents to be clustered.

[0030] Furthermore, the concept extension keyword creation processing unit 1314 is a processing unit that creates extended concept keywords using thesaurus information DB 133 for focus words specified by the user regarding viewpoints of interest. The feature vector transformation processing unit 1315 is a processing unit that represents text (e.g., obtained by the text extraction processing unit 1312, the target keyword text extraction processing unit 1313, the concept extension keyword creation processing unit 1314, etc.) as a vector. The feature vector merging processing unit 1316 is a processing unit that takes two vectors as input and outputs a combined vector. The clustering processing unit 1317 is a processing unit that creates clusters using the feature vectors of each document as input.

[0031] The details of each processing unit in the above-mentioned document classification processing unit 131 will be specifically explained in the flow description described later. Note that some of the processing performed by the arithmetic unit 11 when executing a program may be performed by other arithmetic units (e.g., hardware such as ASICs or FPGAs). Furthermore, the memory 12 may be a non-volatile memory element, such as ROM (Read Only Memory). ROM stores immutable programs (e.g., BIOS). RAM is a high-speed, volatile memory element such as DRAM (Dynamic Random Access Memory), and temporarily stores the program executed by the arithmetic unit 11 and the data used during program execution.

[0032] Furthermore, a user terminal 20 connected to the literature classification system 1 via network N may provide input and output devices to the server 10. The program executed by the arithmetic unit 11 is provided to the server 10 via removable media (CD-ROM, flash memory, etc.) or network N and stored in a non-volatile storage device 13, which is a non-temporary storage medium.

[0033] Therefore, the server 10 of the literature classification system 1 reads data from removable media via the communication interface 14. The server 10 of the literature classification system 1 is a computer system that is physically located on a single computer, or on multiple computers configured logically or physically, and may operate on a virtual computer built on multiple physical computing resources. The server 10 of the literature classification system 1 is preferably configured on the cloud, but may also be on-premises, configured on a specific computer (hardware).

[0034] The network N connecting the server 10 of the literature classification system 1 and the user terminal 20 could be the internet, a LAN (Local Area Network), a WAN (Wide Area Network), or a mobile phone network, but is not limited to these.

[0035] Furthermore, the user terminal 20 can be implemented in various forms, such as a PC (Personal Computer), tablet, smartphone, or XR (Cross Reality) goggles. Additionally, the user terminal 20 may be equipped with a bibliographic information DB 132, a thesaurus information DB 133, and a knowledge information DB 134, which can be accessed by the server 10 of the bibliographic classification system 1 via the network N.

[0036] Furthermore, data exchange between the server 10 and user terminal 20 of the literature classification system 1 may be carried out, for example, according to an API (Application Programming Interface) protocol. In that case, it is assumed that each device has already implemented the functions and configurations necessary to perform API request and response processing.

[0037] <Literature Classification Method: Main Flow> Next, the processing flow in the literature classification method of this embodiment will be explained along with each functional unit of the server 10 of the literature classification system 1. Figure 3 is a diagram showing an example of the literature classification method flow in this embodiment. Here, first, the server 10 distributes a predetermined input screen to the user terminal 20. The user views the input screen on their user terminal 20 and inputs search keywords for literature collection through its user interface. The user terminal 20 responds to the server 10 with these search keywords. These search keywords consist of one or more keywords. Also, the keywords may be single words or compound words.

[0038] In this embodiment, a user could be, for example, someone who reviews and creates a clinical trial plan for a drug. A drug trial is a study in which a new drug is administered to patients to investigate its therapeutic effect and safety. When planning a new such trial, the attributes of the subjects to be included in the trial are researched from past literature and designed accordingly. By referring to the types of patients targeted in past clinical trials for similar drugs or similar diseases, appropriate subjects are selected to confirm the efficacy and safety of the drug to be tested.

[0039] In this embodiment, when a user searches for literature to review for a clinical trial plan, the search keywords for literature collection entered on the input screen will correspond to items determined by the purpose and hypothesis of the clinical trial at the time of its implementation. For example, if the user wants to evaluate the effectiveness of a candidate drug for a specific indication in a clinical trial, the user will select the name of the indication or a name similar to the indication as the search keywords. The literature searched and collected by the server 10 in this embodiment may include not only papers in the field of medical biology, but also review reports (documents reporting the results of drug marketing reviews) and data registered in clinical trial registries that summarize clinical trial plans.

[0040] Meanwhile, the server 10 obtains the receiving terminal and the search target keyword via the user interface of the user terminal 20 (S10). At this time, the document download processing unit 1311 of the server 10 receives the clustering request from the user terminal 20. Next, the document download processing unit 1311 of the server 10 downloads the target document from a predetermined external database using the search target keyword entered by the user as the key (S11).

[0041] Such literature searches and downloads are performed, for example, on databases created and published by NCBI (National Center for Biotechnology Information). In this case, the literature download processing unit 1311 of server 10 performs searches and downloads from PubMed, the NCBI database, via an API (published by NCBI).

[0042] The document download processing unit 1311 stores the downloaded documents in the document information database 132 shown in Figure 4. The document information data stored in the document information database 132 is a record that includes the values ​​of each item, such as the document identifier document ID 1321, the title of the document 1322, and the summary of the document 1323 (abstract and main text), as shown in the figure. Of course, this data structure is just one example and is not limited to the form of this embodiment.

[0043] Next, the text extraction processing unit 1312 of server 10 extracts text to be used for clustering from the documents downloaded in S11 (S12). When extracting text, if the total length of the document is greater than or equal to a certain standard, it is preferable in terms of processing efficiency for the text extraction processing unit 1312 to extract only the summary or conclusion paragraphs of the document. On the other hand, if the total length of the document is shorter than the standard, the text extraction processing unit 1312 may extract the entire text of each document. Furthermore, if the downloaded documents are in a predetermined format (for example, PDF format), the text extraction processing unit 1312 converts the documents into text format for processing by activating and using the functions of the document reader (document viewing application) when extracting text.

[0044] Next, server 10 clusters the documents downloaded in S11, taking into account the text extracted in S12. As already mentioned, the documents to be processed here are obtained based on the search keywords specified by the user. The search keywords also indicate the topics that the user is interested in. If there are multiple search keywords, it is easier for the user to review the clustered documents if they are grouped by topics related to the search keywords. Therefore, server 10 performs the following processing on the feature vectors to group the documents by topics related to the search keywords before clustering them.

[0045] Here, the feature vector transformation processing unit 1315 of the server 10 transforms the text obtained in S12 into a vector (S13). In this case, the feature vector transformation processing unit 1315 appropriately executes a known vectorization method, such as Bag of Words, which performs vectorization according to the frequency of occurrence of words contained in the text. As such a known vectorization method, instead of using a method that uses word frequency, a distributed representation vectorization method such as Word2Vec or GloVe may be adopted.

[0046] The processing in the feature vector transformation processing unit 1315 is a process that performs vector transformation on the input as text, and is performed when vectorizing the text obtained by the text extraction process in S12, the text obtained by the target keyword text extraction process (S14), and the text obtained by the extended concept keyword text extraction process (S15) as input (the same applies hereinafter).

[0047] Next, the target keyword text extraction processing unit 1313 of server 10 performs a process to extract the term of interest specified by the user from each document in the set of documents to be clustered (obtained in S11) (S14). This process in the target keyword text extraction processing unit 1313 is similarly performed in the process of extracting the concept extension keywords obtained in the concept extension keyword identification process (S15) from each document in the set of documents to be clustered.

[0048] The processing in the target keyword text extraction processing unit 1313 will be explained based on the flowchart shown in Figure 5. In this case, the target keyword text extraction processing unit 1313 reads the keywords specified by the user (obtained in S10) as focus terms (S30). If multiple focus terms are specified, multiple focus terms will be targeted for extraction. In this flow, the process loops for the number of documents obtained by the document download processing unit 1311 (S31-S38). Also, if multiple focus terms (keywords) are specified, the process loops for the number of keywords (S32-S36).

[0049] First, the target keyword text extraction processing unit 1313 determines whether the above keyword appears in the document (S33). If the result of this determination is that the keyword is present in the document (S33: YES), the target keyword text extraction processing unit 1313 extracts the keyword as text (S34). If there are multiple keywords of interest in the same document or different keywords of interest, the target keyword text extraction processing unit 1313 separates them with spaces and combines them to form text (S35).

[0050] On the other hand, if the relevant keyword is not found in the literature (S33: NO), the target keyword text extraction processing unit 1313 executes the loop from S32 to S36 again, reads the next keyword, and determines whether the relevant keyword is found in the literature. The target keyword text extraction processing unit 1313 executes the loop from S32 to S36 for each keyword and saves the text containing each keyword read in S30, obtained for each literature, to the storage device 13 or memory 12 (S37), and terminates this flow.

[0051] Furthermore, the vectorization of text by the feature vector transformation processing unit 1315 is also performed on the text obtained by the processing by the target keyword text extraction processing unit 1313 (S14) (S18).

[0052] Next, the feature vector merging processing unit 1316 of server 10 merges the feature vectors obtained from the text of each document (obtained in S12) (obtained in S13) with the vectors obtained from the text obtained in the target keyword text extraction process (S14) (obtained in S18) (first feature merging process) to create a vector (first merged feature vector) (S19).

[0053] This feature vector merging process takes two vectors as input and outputs a combined vector. In this case, the feature vector merging unit 1316 performs the same process when merging a feature vector created from the text obtained in the target keyword text extraction process (S14) with a feature vector created from the extended concept keyword text extraction process (S16). The vector merging process itself is assumed to be performed using a known vector calculation method that follows the so-called vector associativity law.

[0054] Next, the clustering processing unit 1317 of server 10 performs clustering (first clustering process) (S20) using a combined feature vector obtained by combining the feature vector generated by the viewpoint of interest (the viewpoint specified by the above keyword) (obtained in S18) and the feature vector obtained from the original document text (obtained in S13). The clusters obtained by this clustering process are clusters grouped according to the viewpoint of interest of the user. Note that clustering itself is a classification process based on the similarity of vectors between documents.

[0055] Next, the clustering processing unit 1317 re-clusters the clusters formed in S20 into groups that are assumed to be of interest to the user (S21-S24). In this case, the clustering processing unit 1317 executes the conceptual extension keyword creation process in the conceptual extension keyword creation processing unit 1314 in order to set the groups that are assumed to be of interest to the user.

[0056] Therefore, the concept extension keyword creation processing unit 1314 of server 10 creates extended concept keywords for the focus words specified by the user from a perspective of interest, based on the thesaurus information DB 133 and knowledge information DB 134 (S15). The thesaurus information DB 133 is a database that defines the higher and lower levels of concepts between keywords, as shown in Figure 6. The knowledge information DB 134 is a database created from databases such as ClinicalTrials.gov, which is a clinical trial registry. The clinical trial registry stores information and results of clinical trials that have been conducted in the past. The information of such clinical trials includes target diseases, target drugs, subject conditions, and evaluation items that measured the effect, and the results of the clinical trials include evaluation items and their evaluation results. The knowledge information DB 134 is composed of information obtained from such clinical trial registries and includes values ​​for each item such as knowledge words, perspectives, detail of perspectives, items that determine the target population, treatment effect, and target population, as shown in Figure 7.

[0057] In the knowledge information DB134, knowledge words include terms related to target diseases and target drugs. Furthermore, among the events suggested by these terms, the values ​​of the following items, which were used as subject conditions in the same clinical trial, are stored in the column for items that determine the target population. These items include demographic variables (e.g., sex, age, race, height, weight, surface area, obesity level, etc.), variables related to constitution and genetics (e.g., drug allergies, preferences (drinking / smoking), disease-related gene types, subtypes, etc.), variables related to medical history (e.g., past medical history, treatment history for other diseases, underlying diseases, complications, etc.), and variables related to the current disease (e.g., diagnosis and disease type, duration of illness and number of days elapsed, age of first onset, treatment history for the original disease, severity at the start of the trial, etc.).

[0058] Furthermore, for example, if the subject criteria are set to "patients with HER2-positive status determined according to guidelines," then the "Criteria" would be "variables related to constitution and genetics," the "Criteria Details" would be its subtype, and the "Items that determine the target population" would be a keyword such as "HER2." These values ​​for "Criteria" and "Criteria Details" can be created and set in advance by an expert, or they can be classified and set using a machine learning classifier. Similarly, the keywords to be extracted can be extracted by an expert, or they can be recognized and extracted using natural language processing. In addition, "Therapeutic effect" stores the evaluation items and their evaluation results shown in the clinical trial results obtained from the clinical trial registry mentioned above. Finally, "Target population" stores the number of subjects who participated in the clinical trial.

[0059] The following describes the processing in the concept extension keyword creation processing unit 1314 based on the flowchart in Figure 8. Here, as an example, let's assume that the user specifies "Gastric Cancer" as the term of interest. In this case, the concept extension keyword creation processing unit 1314 searches the knowledge word column of the knowledge information data in Figure 7 based on this term of interest "Gastric Cancer" (S40).

[0060] If a matching keyword is found in this search (S41:YES), the concept extension keyword creation processing unit 1314 aggregates the knowledge information to which the keyword belongs based on the value in the "Category Details" column and calculates the average value of the treatment effect for each Category Details (S42). For example, among the records that contain "Gastric Cancer" as a "knowledge word," the average value of the "treatment effect" for records where the "Category Details" is "Past Medical History" is calculated as (15+15) / 2=15%. Similarly, the average value of the "treatment effect" for records related to other "Category Details" is calculated as (10+8+6) / 3=8%, and so on.

[0061] Alternatively, the concept extension keyword creation processing unit 1314 may aggregate the knowledge information to which the keyword belongs based on the value in the "Category Details" column and calculate the size of the population for each Category Details. This calculation of the size of the population may involve calculating the average or maximum value of the target population. For example, among the records that contain "Gastric Cancer" as a "knowledge word," the average value of the "Target Population" for records where the "Category Details" is "Past Medical History" is calculated as (300+300+300) / 3 = 300 people. Similarly, the average value of the "Target Population" for records related to another "Category Details" is calculated as (100+150+80) / 3 = 110 people, and so on.

[0062] The concept extension keyword generation processing unit 1314 outputs the "item that determines the target population" indicated by the record of the detail with the highest average value of the treatment effect or the average value of the target population calculated as described above (e.g., medical history) as a concept extension keyword (S43). By using knowledge information data generated based on clinical trial results data, it is possible to refine the topics of clusters that are effective when formulating clinical trial plans. In other words, by using knowledge information data to narrow down to keywords of concepts that users want more, it becomes possible to extract and present literature that shows the topics and concepts that users want as a result of reclassification with high accuracy.

[0063] On the other hand, if the search in S41 in the above flow does not find a matching keyword (S41:NO), the concept extension keyword creation processing unit 1314 searches the thesaurus information DB 133 for a higher-level concept (or lower-level concept) for the term of interest "Gastric Cancer" and obtains it as a concept extension keyword (S44). In this process, when obtaining a keyword for a higher-level concept, if the lower-level concept column 701 is "Gastric Cancer", the value in the higher-level concept column, "Neoplasm of stomach", can be obtained as a concept extension keyword. When obtaining a keyword for a lower-level concept of the term of interest, if the higher-level concept column 702 is "Gastric Cancer", the value in the lower-level concept column can be extracted to obtain a more detailed concept such as "Gastric carcinoma".

[0064] This process outputs items that determine the target population for detailed analysis segments with high therapeutic efficacy or large populations as conceptual extension keywords. When formulating clinical trial plans, it is sometimes effective to confirm the conditions for subjects with high therapeutic efficacy and large populations. This process automatically extracts detailed analysis segments with high therapeutic efficacy or large populations, allowing for the detailed refinement of cluster topics that are effective for clinical trial plans. Furthermore, each analysis segment includes at least four variables: (A) demographic variables, (B) variables related to constitution and genetics, (C) variables related to medical history, and (D) variables related to the current disease. These variables define the framework of the target population for drug indication, and since the detailed analysis segments are defined based on these variables, outputting items that determine the target population for these detailed analysis segments as conceptual extension keywords allows for the detailed refinement of cluster topics that are effective for clinical trial plans. In other words, by using knowledge information data to narrow down the keywords to the concepts that users want more, the reclassification results can be extracted with high accuracy to the topics and concepts that users want.

[0065] In this embodiment, when identifying a concept extension keyword, the first step is to search for a knowledge word in the knowledge information DB134 based on the word of interest (S40). If the keyword cannot be found (S41: NO), then step S44 is performed, which searches for a higher-level concept (or lower-level concept) for the word of interest in the thesaurus information DB133. However, in addition to this flow, step S44, which searches for a higher-level concept (or lower-level concept) for the word of interest in the thesaurus information DB133, may be performed before or in parallel with S40 and S41, regardless of the results of S40 and S41. Alternatively, in cases where the knowledge information DB134 has an insufficient amount of data, or where the knowledge information DB134 is not available or unusable, only step S44 may be performed.

[0066] Now, let's return to the explanation of the flow in Figure 3. The target keyword text extraction processing unit 1313 of Server 10 uses the concept extension keywords obtained up to this point to extract keywords from each document in the set of documents to be clustered and output them as text (S16). The feature vector transformation processing unit 1315 then creates a feature vector based on the text obtained in S16 (S17).

[0067] Here, the clustering processing unit 1317 of server 10 takes out each cluster that was clustered in step S20 and reclusters the literature group belonging to each cluster (S21-S24). By reclustering, it is possible to extract even more detailed clusters. This reclustering is repeated as many times as there are clusters.

[0068] In this re-clustering process, the clustering processing unit 1317 combines the feature vector created from the text of the literature belonging to the target cluster, the feature vector obtained from the text of the literature belonging to the target cluster using the word of interest as the key in the target keyword text extraction process, and the feature vector obtained using the concept extension keyword, using the feature vector combining processing unit 1316 (second feature combining process) to obtain a second combined feature vector (S22).

[0069] The clustering processing unit 1317 can obtain more detailed clusters by performing a re-clustering process (second clustering process) (S23) based on the combined feature vector obtained in S22. By performing re-clustering using a feature vector obtained by combining the feature vector generated from the viewpoint of interest with the feature vector obtained from the original literature text, it is possible to form clusters that are grouped by the viewpoint of interest.

[0070] Thus, by performing re-clustering using a combined feature vector that combines feature vectors extended with more detailed concepts from the perspective focused on by concept extension keywords, it is possible to obtain clustering results that drill down to more detailed concepts through the Nth clustering compared to the N-1th clustering result.

[0071] As described above, by performing re-clustering using keywords that extend concepts above or below the term of interest, it becomes possible to cluster similar but different perspectives. Therefore, it becomes possible to perform clustering that drills down to deeper perspectives on a set of literature obtained from a perspective of interest to the user, and to extract and present literature containing the topics and concepts that the user truly wants with high accuracy. In addition, when using concept-extending keywords, knowledge information data can be used to narrow down the keywords to those of concepts that the user is more likely to want, and use them for clustering. As a result, the results of re-clustering can be more accurately matched to the topics and concepts that the user desires.

[0072] As explained above, the literature classification system 1 in this embodiment allows for the refinement of cluster topics by repeating clustering multiple times, thereby enabling efficient identification and presentation of literature desired by the user. In general clustering, documents in a multidimensional space composed of words are statistically classified. Therefore, the clusters obtained in such clustering are determined solely from the perspective of the statistical behavior of words, and often include incomprehensible clusters. Consequently, even if the topics are refined, users may not be able to reach the information they truly desire. In this embodiment, by forming clusters with topics that broaden concepts through pre-created knowledge information about literature information from the user's perspective, it becomes easier for users to reach topics of interest even when it is difficult for them to devise keywords for conceptual expansion themselves. Clustering can be performed in advance, rather than after the user specifies the clusters, thus shortening the time required to obtain clustering results.

[0073] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail for the purpose of clearly illustrating the present invention, and the present invention is not necessarily limited to having all the described configurations. Furthermore, some of the configurations of one embodiment may be replaced with those of another embodiment. Furthermore, configurations of other embodiments may be added to the configuration of one embodiment. Furthermore, some of the configurations of each embodiment may be added, deleted, or replaced with those of other embodiments.

[0074] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, for example, by designing them as integrated circuits, or they may be implemented in software by having a processor interpret and execute programs that realize each function.

[0075] Information such as programs, tables, and files that implement each function can be stored in memory 12, a storage device 13 such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0076] Furthermore, the control lines and information lines shown are those deemed necessary for explanation purposes and do not necessarily represent all control lines and information lines required for implementation. In reality, it can be assumed that almost all components are interconnected.

[0077] Furthermore, the above explanations can be summarized as follows. The following summary may include supplementary explanations and explanations of modified versions of the above explanations. In the literature classification system 1 of this embodiment, the storage device further holds knowledge information that associates one or more knowledge words indicating a predetermined event with information on the perspective of the event, and the computing device, in the concept extension keyword identification process, matches the user-specified keyword with the knowledge information, identifies the information on the perspective that corresponds to the user-specified keyword from the knowledge information, selects a specific piece of knowledge information based on the characteristics indicated by the information on the perspective, and extracts the knowledge word defined in the selected knowledge information as the concept extension keyword.

[0078] According to this method, it becomes possible to identify information from a large amount of knowledge that shows a specific trend from a particular perspective (e.g., treatment effectiveness above a certain standard), and to select knowledge words that represent that knowledge as valid topics to use in clustering. In turn, it becomes possible to extract topics and concepts desired by the user with even greater accuracy from a group of literature classified according to the user's interests.

[0079] Furthermore, in the literature classification system 1 of this embodiment, the information of the aforementioned viewpoint may be any of the following: demographic variables, variables relating to constitution and genetics, variables relating to medical history, variables relating to the current disease, treatment effectiveness, and population size.

[0080] According to this, when selecting knowledge information based on the above-mentioned perspectives, it becomes possible to make selections based on various perspectives. Consequently, it becomes possible to extract topics and concepts desired by the user with even greater accuracy from a group of literature classified according to the user's interests.

[0081] Furthermore, in the literature classification system 1 of this embodiment, the computing device may, in the conceptual extension keyword identification process, identify information on the perspective from among the information on the perspective associated with the user-specified keyword, based on the information on the therapeutic effect, select a specific piece of knowledge information based on the characteristics indicated by the information on the therapeutic effect, and extract the knowledge word defined in the selected knowledge information as the conceptual extension keyword.

[0082] According to this method, when obtaining conceptual extension keywords, knowledge information can be selected from the perspective of therapeutic effect (e.g., the average therapeutic effect is relatively high), and these knowledge words can be identified as conceptual extension keywords. By using knowledge information generated based on clinical trial result data in this way, it becomes possible to refine the topics of clusters that are effective when formulating clinical trial plans. In turn, it becomes possible to extract topics and concepts desired by the user with even greater accuracy from a set of literature classified according to the user's interests.

[0083] Furthermore, in the literature classification system 1 of this embodiment, the computing device may, in the conceptual extension keyword identification process, identify information on the viewpoint associated with the user-specified keyword based on the information on the size of the population, select specific pieces of knowledge information based on the characteristics indicated by the information on the size of the population, and extract the knowledge words defined in the selected knowledge information as the conceptual extension keywords.

[0084] According to this method, when obtaining conceptual extension keywords, knowledge information can be selected from the perspective of the target population (e.g., the target population is relatively large), and these knowledge words can be identified as conceptual extension keywords. By using knowledge information generated based on clinical trial result data in this way, it becomes possible to refine the topics of clusters that are effective when formulating clinical trial plans. In turn, it becomes possible to extract topics and concepts desired by the user with even greater accuracy from a set of literature classified according to the user's interests. [Explanation of Symbols]

[0085] N Network 1. Literature Classification System 10. Server (Literature Classification Device) 11 Arithmetic unit 12 memory 13 Storage device 131 Literature Classification Processing Department 1311 Document Download Processing Unit 1312 Text Extraction Processing Unit 1313 Target Keyword Text Extraction Processing Unit 1314 Conceptual Extension Keyword Creation Processing Unit 1315 Feature vector transformation processing unit 1316 Feature Vector Combination Processing Unit 1317 Clustering Processing Unit 132 Literature information DB 133 Thesaurus Information Database 134 Knowledge information DB 14 Communication I / F 15 CPU (Processor) 20 User Terminals

Claims

1. A literature classification system for classifying a collection of literature, A storage device that holds knowledge information related to title information, A computing device that performs: a first feature merging process that merges the features of literature searched based on user-specified keywords with the features of text containing the keywords in the literature; a first clustering process that clusters the literature using a first merged feature vector created by the first feature merging process; a concept extension keyword identification process that matches the keywords with the knowledge information and identifies concept extension keywords; a second feature merging process that merges the features of literature included in at least one cluster generated by the first clustering process, the features of text containing the keywords in the literature, and the features of concept extension keywords in the cluster; and a second clustering process that clusters the literature using a second merged feature vector created by the second feature merging process. A literature classification system that includes this.

2. The aforementioned storage device holds thesaurus information for each word, The aforementioned computing device, in the conceptual extension keyword identification process, identifies conceptual extension keywords that correspond to concepts lower or higher than the keyword. The literature classification system according to claim 1.

3. The aforementioned storage device is Furthermore, it maintains knowledge information that associates one or more knowledge words representing a given event with information from the perspective of that event, The aforementioned computing device is In the aforementioned conceptual extension keyword identification process, the user-specified keyword is compared with the knowledge information, the information on the perspective corresponding to the user-specified keyword is identified from the knowledge information, a specific piece of knowledge information is selected based on the characteristics indicated by the information on that perspective, and the knowledge word defined in the selected knowledge information is extracted as the conceptual extension keyword. The literature classification system according to claim 1.

4. The information from the aforementioned perspectives includes information on any of the following: demographic variables, variables related to constitution and genetics, variables related to medical history, variables related to the current disease, treatment effectiveness, and population size. The literature classification system according to claim 3.

5. The aforementioned computing device is In the aforementioned conceptual extension keyword identification process, information on the perspective is identified from among the information on the perspective associated with the user-specified keyword, based on the information on the therapeutic effect, specific pieces of knowledge information are selected based on the characteristics indicated by the information on the therapeutic effect, and the knowledge words defined in the selected knowledge information are extracted as the conceptual extension keywords. The literature classification system according to claim 4.

6. The aforementioned computing device is In the aforementioned conceptual extension keyword identification process, information on the aforementioned viewpoint is identified from among the information on the aforementioned viewpoint associated with the user-specified keyword, based on the information on the size of the population, specific pieces of knowledge information are selected based on the characteristics indicated by the information on the size of the population, and the knowledge words defined in the selected knowledge information are extracted as the conceptual extension keywords. The literature classification system according to claim 4.

7. A literature classification device for classifying a collection of literature, A storage device that holds knowledge information related to title information, A computing device that performs: a first feature merging process that merges the features of literature searched based on user-specified keywords with the features of text containing the keywords in the literature; a first clustering process that clusters the literature using a first merged feature vector created by the first feature merging process; a concept extension keyword identification process that matches the keywords with the knowledge information and identifies concept extension keywords; a second feature merging process that merges the features of literature included in at least one cluster generated by the first clustering process, the features of text containing the keywords in the literature, and the features of concept extension keywords in the cluster; and a second clustering process that clusters the literature using a second merged feature vector created by the second feature merging process. A literature classification device that includes [this component].

8. A literature classification system that classifies a collection of literature, Knowledge information about bibliographic materials is stored in a storage device. A first feature merging process combines the feature quantities of literature searched based on user-specified keywords with the feature quantities of text in the literature that contains those keywords. A first clustering process is performed, which involves clustering the literature using the first combined feature vector created by the first feature merging process described above, A conceptual extension keyword identification process that identifies conceptual extension keywords by matching the aforementioned keywords with the aforementioned knowledge information, A second feature merging process is performed to combine, for at least one cluster generated by the first clustering process, features relating to the literature included in the cluster, features of the text containing the keyword in the literature, and features relating to the concept extension keyword. A second clustering process, which involves performing clustering on the literature using the second combined feature vector created by the second feature merging process described above. A method for classifying literature, characterized by performing the following actions.

Images