Item classification device, item classification system, and item classification method

Abstract

To provide an item classifying device, an item classifying system and an item classifying method which reduce costs of vector creation of an item.SOLUTION: Item classifying processing executed by an item classifying device: divides an item group of a proper reference sentence for each item; gives a tag being a character string for distinguishing meaning of words and phrases to the words and phrases of the item included in the proper reference sentence; sections the given tag into a narrower concept tag and a broader concept tag; generates a vector of the item including the narrower concept tag and the broader concept tag for each item; and causes a classification model to learn which uses the vector of the item and a classification category as an explanatory variable and an objective variable, respectively.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an item classification device, an item classification system, and an item classification method.

Background Art

[0002] In new drug development, pharmaceutical companies conduct clinical trials to obtain data on the efficacy and safety of new drugs. At that time, clinical trials with inappropriate recruitment conditions set cause the prolongation and delay of new drug development.

[0003] Therefore, pharmaceutical companies carefully plan the implementation method before conducting clinical trials. Planners are required to set eligible conditions for subjects (hereinafter referred to as "eligibility criteria") considering patient characteristics such as past history and age. At that time, planners refer to public clinical trial information and comprehensively investigate the protocols of clinical trials related to the target disease and drug.

[0004] Note that the eligibility criteria in this specification refer to all or any one or more of the criteria related to the selection of trial subjects such as selection criteria and exclusion criteria.

[0005] However, the above-mentioned public clinical trial information lists the eligibility criteria as a set of items in unstructured data and is not suitable for searching for the target information. The "items" mentioned here are sentences or words listed with symbols or numbers at the beginning or by line breaks. By classifying each of the listed items according to the type of criteria such as medical history and test values, planners can efficiently collect the target information.

[0006] Conventionally, as a method for classifying items, a supervised learning method of machine learning is used (see, for example, Non-Patent Document 1). In the example described in Non-Patent Document 1, each word included in an item is converted into a vector, and the vector of the item is expressed using the word vectors.

[0007] One example of a vector is the one-hot vector (hereinafter referred to as a vector), which represents an item's vector by creating as many dimensions as there are words in the item's vocabulary, with dimensions corresponding to words that appear in the item being "1" and dimensions corresponding to words that do not appear being "0".

[0008] The eligibility criteria include words with low frequency of occurrence (hereinafter referred to as "low-frequency words"), such as new disease names or rare drug names that do not exist in the training data. To adequately train a machine learning model on low-frequency words, the aforementioned method of creating vectors for the items requires a large amount of training data.

[0009] To address the above issues, for example, Patent Document 1 discloses a method for training a word embedding model that uses words of higher-level concepts in the ontology in addition to the words to be learned as training data. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Special Publication No. 2022-545052 [Non-patent literature]

[0011] [Non-Patent Document 1] Zong, H., Yang, J., Zhang, Z., Li, Z., & Zhang, X. (2021). Semantic categorization of Chinese eligibility criteria in clinical trials using machine learning methods. BMC medical informatics and decision making, 21(1), 128. [Non-Patent Document 2] Yuan, C., Ryan, PB, Ta, C., Guo, Y., Li, Z., Hardin, J., Makadia, R., Jin, P., Shang, N., Kang, T., & Weng, C. (2019). Criteria2Query: a natural language interface to clinical databases for cohort definition. Journal of the American Medical Informatics Association : JAMIA, 26(4), 294-305. [Overview of the Initiative] [Problems that the invention aims to solve]

[0012] However, creating item vectors using the method described in Patent Document 1 requires training a word embedding model with the superordinate concept of each low-frequency word appearing in the item as training data, which is costly in creating item vectors.

[0013] In practical terms, the cost of this vector generation presents challenges when changing classification categories based on customer requests and retraining the model, or when classifying new items, as the time required to convert items into vectors is significant.

[0014] The objective of this invention is to reduce the cost of creating item vectors in an item classification device. [Means for solving the problem]

[0015] An item classification device according to one aspect of the present invention comprises an item group division unit, an item tagging unit, an item vector generation unit, a classification model learning unit, and a classification model execution unit, and is an item classification device for classifying eligibility criteria sentences for clinical trials, wherein the item group division unit divides the item group of the eligibility criteria sentences into individual items, the item tagging unit assigns tags, which are strings of characters for distinguishing the meaning of the items, to the words or phrases of the items, the item vector generation unit divides the tags into subordinate concept tags, which are tags of subordinate concepts, and superior concept tags, which are superior concepts of the subordinate concept tags, and generates an item vector for each item that includes the subordinate concept tags and the superior concept tags, the classification model learning unit learns a classification model using the item vectors as explanatory variables and classification categories as the objective variable, and the classification model execution unit executes the classification model. [Effects of the Invention]

[0016] According to one aspect of the present invention, the cost of creating item vectors in an item classification device can be reduced. [Brief explanation of the drawing]

[0017] [Figure 1] This is a block diagram illustrating the schematic configuration of an item classification device according to an embodiment of the present invention. [Figure 2] This is a flowchart illustrating the item classification process performed by an item classification device according to an embodiment of the present invention. [Figure 3] This is a flowchart illustrating the item group division process performed by an item classification device according to an embodiment of the present invention. [Figure 4] This is a flowchart illustrating the item tagging process performed by an item classification device according to an embodiment of the present invention. [Figure 5] This is a flowchart illustrating the process of creating a vector representation of an item in the item vector generation process performed by the item classification device according to an embodiment of the present invention. [Figure 6] This is a flowchart illustrating the classification model learning process performed by an item classification device according to an embodiment of the present invention. [Figure 7] It is a flowchart for explaining the classification model execution process executed by the item classification device according to an embodiment of the present invention. [Figure 8] It is a diagram showing the data structure of the item group of eligibility criteria held by the item classification device according to an embodiment of the present invention. [Figure 9] It is a diagram showing the data structure of the item of eligibility criteria held by the item classification device according to an embodiment of the present invention. [Figure 10] It is a diagram showing the data structure of the tagged item held by the item classification device according to an embodiment of the present invention. [Figure 11] It is a diagram showing the data structure of the tag and identifier held by the item classification device according to an embodiment of the present invention. [Figure 12] It is a diagram showing the data structure of the hierarchical relationship between tags held by the item classification device according to an embodiment of the present invention. [Figure 13] It is a diagram showing the data structure of the vector of the item held by the item classification device according to an embodiment of the present invention. [Figure 14] It is a diagram showing the data structure of the item with classification category held by the item classification device according to an embodiment of the present invention. [Figure 15] It is a diagram showing the data structure of the classification result held by the item classification device according to an embodiment of the present invention. [Figure 16] It is an explanatory diagram showing an example of the display of the tag / superordinate concept selection screen by the item classification device according to an embodiment of the present invention. [Figure 17] It is an explanatory diagram showing an example of the display of the item narrowing-down condition input screen by the item classification device according to an embodiment of the present invention. [Figure 18] It is an explanatory diagram showing an example of the display of the item narrowing-down result screen by the item classification device according to an embodiment of the present invention.

Mode for Carrying Out the Invention

[0018] This invention relates to a technique for organizing and classifying eligibility (selection and exclusion) criteria items registered in a clinical trial database. In the embodiment, eligibility criteria items are classified into two categories: sentences related to "general condition" and sentences that are not related to "general condition". A classification category is a label assigned to a sentence that indicates whether or not the sentence falls under one of the types of criteria, such as medical history or laboratory values.

[0019] In this embodiment, in order to reduce the cost of creating item vectors, the words used to create item vectors are limited by the assignment of tags that indicate the attributes of the words, and item vectors are created using a higher-level concept of tags without requiring a word embedding model.

[0020] In this specification, "tags" refer to words or strings of characters that are assigned to keywords (hereinafter referred to as "phrases") from an item, and are used to distinguish the meaning of the assigned phrases.

[0021] First, to limit the words used in creating the item vectors, tags are assigned to the item phrases. For example, in the item "Prior treatment with leflunomide," the tag "Procedure" is assigned to the phrase "treatment," and the tag "Drug" is assigned to the phrase "leflunomide." Models such as the one in Non-Patent Document 2 can be used for this process.

[0022] Next, as a method that does not use a word embedding model, a vector with a number of dimensions equal to the number of tags and their superordinate concepts is prepared, and the tags and superordinate concepts are combined to create an item vector. In this way, it is possible to create an item vector in a short time without using a word embedding model.

[0023] The above embodiment is characterized in that, with respect to tags, which are strings attached to the words or phrases of an item to distinguish the meaning of the words or phrases, a hierarchical structure of tags indicating the hierarchical relationships between the tags is used to extract the higher-level concept of the tags in the hierarchical structure of the tags included in a certain item, and this higher-level concept is added to the vector relating to the tags of that item.

[0024] In this way, the classification model learns both low-frequency words and their higher-level concepts, improving the classification accuracy of items that contain many low-frequency words.

[0025] The following describes an example using drawings. [Examples]

[0026] Figure 1 is a block diagram showing the functional configuration of an item classification device according to an embodiment of the present invention.

[0027] As shown in Figure 1, the item classification device 100 includes a control unit 101, a memory 102, and a storage unit 103. The item classification system is then composed of the item classification device 100, a display unit 121, and an input unit 122.

[0028] The control unit 101 is a processor that executes processing according to a program stored in the memory 102. The memory 102 is a storage device that stores the program executed by the control unit 101 and data referenced by the control unit 101.

[0029] In the example shown in Figure 1, memory 102 stores the item group division unit 104, the item tagging unit 105, the item vector generation unit 106, the classification model learning unit 107, and the classification model execution unit 108. In other words, the processes performed by each of these units in the following description are actually executed by the control unit 101 according to the program stored in memory 102. The flow of the item classification process is shown in Figure 2.

[0030] The storage unit 103 of the item classification device 100 stores the group of eligibility criteria items 109, the eligibility criteria items 110, the tagged items 111, the tags and identifiers 112, the hierarchical relationships between tags and identifiers 113, the item vectors 114, the items with classification categories 115, the classification model 116, and the classification results 117.

[0031] The item group division unit 104 retrieves the item group 109 of the eligibility criteria from the storage unit 103, divides the item group into bullet points or numbered items, separates each item with a line break, and assigns an identifier to each item. As a result, the items 110 of the eligibility criteria are stored in the storage unit 103.

[0032] The item tagging unit 105 retrieves the eligibility criterion items 110 from the storage unit 103. The tag strings in Figure 11 are associated with the words contained in the eligibility criterion items, and the identifiers of the corresponding eligibility criterion items and the identifiers of the tags are stored in a table to collect the identifiers of each item and the tags attached to the words within each item.

[0033] For example, a model like that in Non-Patent Document 2 is used. As a result, the tagged item 111 is stored in the storage unit 103.

[0034] The item vector generation unit 106 obtains tagged items 111, tags and identifiers 112, and hierarchical relationships 113 between tag identifiers from the storage unit 103. All tags of items in tagged item 111 are converted in the order of tag identifier, identifier of its higher-level concept tag, and higher-level concept, and all tags and the higher-level concepts of all tags for all items are collected.

[0035] Next, a vector is created by expanding the items into columns representing the tags collected from all items and their higher-level concepts. For the created vector, the values ​​in the columns containing the tags and higher-level concepts obtained from each item in each item's row are set to "1", and the values ​​in the other columns in each item's row are set to "0". As a result, the item vector 114 is stored in the storage unit 103.

[0036] The classification model learning unit 107 obtains the eligibility criteria items 110, the item vector 114, and the items with classification categories 115 from the memory unit 103. Using the item vector 114 as the explanatory variable and the items with classification categories 115 as the target variable, it learns a classification model. As a result, the classification model 116 is stored in the memory unit 103.

[0037] The classification model execution unit 108 uses the item vector 114 and the classification model 116 from the storage unit 103 to classify the items. As a result, the classification result 117 is stored in the storage unit 103.

[0038] The group of eligibility criteria items 109 stores sets of items related to clinical trial eligibility criteria from a publicly available database on a website. For example, one such public database is the internet site where clinical trial protocols are registered (ClinicalTrials.gov).

[0039] From the public clinical trial database (ClinicalTrials.gov), groups of items corresponding to the eligibility criteria for a clinical trial are manually retrieved, and these groups are stored with assigned identifiers. An example of data for eligibility criteria groups is shown in Figure 8.

[0040] The eligibility criteria item 110 stores the eligibility criteria items created by the item group division unit 104 and the identifiers for each item. An example of the data for the eligibility criteria items is shown in Figure 9.

[0041] Tagged item 111 stores the identifier of the tag assigned to the item's phrase and the item's identifier, which were created during the item tagging process. An example of the data for tagged item 111 is shown in Figure 10.

[0042] The tags and identifiers 112 are manually obtained from an external website and stored as the type of tag and the tag identifier assigned by the item tag assignment unit 105. For example, from the website (Athena) that maintains the OMOP standardization vocabulary, the `concept_name` column of `concept.csv` in OMOP Common Data Model Vocabulary ver5.2.2 contains the tag type, and the `concept_id` column contains the tag identifier. An example of the tag and identifier data 112 is shown in Figure 11.

[0043] The hierarchical relationship 113 between tags and identifiers is determined by the identifiers of tags assigned by the item tag assignment unit. The identifiers of higher-level concepts and lower-level concepts are manually obtained from external websites and stored. For example, from Athena, a site that maintains the OMOP standardization vocabulary, the ancestor_concept_id column in concept_ancestor.csv of OMOP Common Data Model Vocabulary ver5.2.2 contains the identifier of the higher-level concept of the tag, and the descendant_concept_id column contains the identifier of the lower-level concept. An example of the hierarchical relationship between tags is shown in Figure 12.

[0044] The item vector 114 stores the vector created by the item vector generation unit. An example of the data for item vector 114 is shown in Figure 13.

[0045] The categorized item 115 manually assigns a classification category to each item in the eligibility criteria item 110, and stores the item identifier and classification category. An example of the data for the categorized item 115 is shown in Figure 14.

[0046] The classification model 116 stores the classification model created by the classification model learning unit 108. The classification result 117 stores the classification result obtained by the classification model execution unit. An example of the data for the classification result 117 is shown in Figure 15.

[0047] The display unit 121 displays three screens. The first is the tag / higher-level concept selection screen. It receives the item vector 114 from the item classification device 100 and displays the tags that characterize the item vector. In the display unit 122, the user selects and inputs tags or higher-level concepts that they want to exclude from the item vector 114, and the resulting vector with the exclusions is output to the display unit 121.

[0048] Furthermore, the user can delete tags and store them in item vector 114 by confirming the tags or higher-level concepts to be excluded. An example of the tag / higher-level concept selection screen is shown in Figure 16.

[0049] The second screen is for inputting the desired classification category, and the third screen is for the items after narrowing down to the desired classification category. The display unit 121 receives the classification results from the item classification device 100 and displays the items separated by classification category based on the classification results. The input unit 122 allows the user to input the desired classification category and outputs the results limited to the desired classification category to the display unit 121.

[0050] In the example of the display unit 121, Figure 17 shows the input screen for the desired classification category, and Figure 18 shows the screen after narrowing down to the desired classification category.

[0051] Next, the processing flow will be described. Figure 2 is a flowchart illustrating the item classification process performed by the item classification device according to an embodiment of the present invention. The item classification process in Figure 2 corresponds to the item classification unit 102 in Figure 1.

[0052] In step S201, the group of eligibility criteria items 109 is read from the storage unit 103, divided into individual items, and stored in the storage unit 103 as eligibility criteria items 110. The specific processing flow is shown in Figure 3.

[0053] In step S202, tags are assigned to the eligibility criteria items, and the tags assigned to each item's phrase and the identifier of each item are stored in the tagged item 111. The specific processing flow is shown in Figure 4.

[0054] In step S203, the tagged items 111, tags and identifiers 112, and the hierarchical relationships 113 between tags and identifiers are retrieved from the storage unit 103. A vector is created from the items and stored in the storage unit 103 as the item vector 114. The specific processing flow is shown in Figure 5.

[0055] In step S204, the item vector 114 and the items with classification categories 115 are obtained from the memory unit 103, a classification model is trained, and the classification model 116 is stored in the memory unit 103. The specific processing flow is shown in Figure 6.

[0056] In step S205, the item vector 114 and classification model 116 are obtained from the storage unit 103, item classification is performed, and the classification result 117 is stored in the storage unit 103. The specific processing flow is shown in Figure 7. After that, the item classification unit 102 terminates processing.

[0057] Figure 3 is a flowchart illustrating the item group division process performed by an item classification device according to an embodiment of the present invention. The item group division process in Figure 3 corresponds to the item group division unit 104 in Figure 1 and step S201 in Figure 2.

[0058] In step S301, the group of eligibility criteria items 109 is read from the memory unit 103.

[0059] In step S302, the group of eligibility criteria items are divided into bullet points or numbered items, with each item separated by a line break. For example, in the data example in Figure 8, the items "Patients must have a SWOG performance status of 0-2.", "Age 18-65 years.", and "Prior treatment with leflunomide." are obtained from the first group of items. The same process is performed for the other groups of items to collect all items.

[0060] In step S303, an item identifier is assigned to each item obtained. For example, the item identifier "0" is assigned to the item "Patients must have a SWOG performance status of 0-2.". The same process is performed for the other items to obtain all items and their identifiers.

[0061] In step S303, the items and item identifiers obtained in step S302 are stored in the storage unit 103 as the eligibility criteria item 110.

[0062] In step S304, it is determined whether there are any unprocessed item groups in the eligibility criteria item group 109. If there are unprocessed items, the item group division unit 104 returns to step S302 and repeats the above process using another unprocessed item group. On the other hand, if there are no unprocessed items, the item group division unit 104 terminates the process.

[0063] Figure 4 is a flowchart illustrating the item tagging process performed by an item classification device according to an embodiment of the present invention. The item tagging process in Figure 4 corresponds to the item tagging unit 105 in Figure 1 and step S202 in Figure 2.

[0064] In step S401, the eligibility criteria items 110 are read from the memory unit 103. In step S402, tags are assigned to the words of each item from the memory unit 103. For example, using the Criteira2Query model from Non-Patent Literature 2 for the eligibility criteria items, the tag "measurement" is assigned to the word "SWOG performance status" in the item "Patients must have a SWOG performance status of 0-2.", and the tag "ecog performance status panel" is assigned to the same word "SWOG performance status".

[0065] In step S403, the identifier of each item and the identifier of the tag attached to the phrase of each item are stored in the storage unit 103 as a tagged item 111. For example, the item "Patients must have a SWOG performance status of 0-2." is assigned the item identifier "0", so the item identifier "0" and the tag identifier "3", and the item identifier "0" and the tag identifier "2" are retrieved.

[0066] In step S404, it is determined whether there are any unprocessed items in the eligibility criteria item 110. If there are unprocessed items, the item tagging unit 105 returns to step S402 and repeats the above process using another unprocessed item. On the other hand, if there are no unprocessed items, the item tagging unit 105 terminates the process.

[0067] Figure 5 is a flowchart illustrating the vector generation process for items performed by an item classification device according to an embodiment of the present invention. The vector generation process in Figure 5 corresponds to the vector generation unit 106 in Figure 1 and step S203 in Figure 2.

[0068] In step S501, the tagged item 111 is read from the storage unit 103. In step S502, the tag and identifier 112 are read from the storage unit 103.

[0069] In step S503, the hierarchical relationships 113 between tag identifiers are read from the storage unit 103. In step S504, the identifiers of the higher-level concepts corresponding to the tag identifiers obtained in step S501 are obtained using the hierarchical relationships 113 between tags. For example, if the tag identifier "2" is obtained in step S501, the tag identifiers "0" and "1", which are higher-level concepts of tag identifier "2", are obtained in the data example in Figure 12. The same process is performed for all tag identifiers to obtain the identifiers of all higher-level concepts for all tags.

[0070] In step S505, the identifiers of each higher-level concept are converted to the higher-level concept using the tags and identifiers 112. For example, if the identifiers of the higher-level concepts "0" and "1" are obtained in step S504, then in the data example in Figure 11, the tag "clinical" corresponding to identifier "0" and the tag "order set" corresponding to identifier "1" are obtained.

[0071] In step S506, duplicate superordinate concepts are removed from all the superordinate concepts collected in step S505. For example, if there were three tags "order set" among the acquired superordinate concepts, one is kept and the other two are removed.

[0072] In step S507, using all the tags and their superordinate concepts obtained up to step S506, and the identifiers of the tagged items 111, a vector is generated in which all item identifiers are used as rows and the tags and superordinate concepts obtained from all items are used as columns, with all values ​​being 0.

[0073] In step S508, all tags assigned to a single item identifier are retrieved from the tagged item 111. For example, in the data example in Figure 10, tag identifiers "3" and "2" are retrieved for the item identifier "0".

[0074] In step S509, all identifiers of higher-level concepts for tag identifiers are obtained using the hierarchical relationships 113 between tags. For example, if identifiers "3" and "2" were obtained in step S508, the tag identifiers "0" and "1", which are higher-level concepts for identifier "2", are obtained from the data example in Figure 12.

[0075] In step S510, all identifiers of higher-level concepts are converted to higher-level concepts using tags and identifiers 112. For example, if the identifiers of higher-level concepts "0" and "1" are obtained in step S509, the tag "clinical" corresponding to identifier "0" and the tag "order set" corresponding to identifier "1" are obtained from the data example in Figure 7.

[0076] In step S511, the vector values ​​are set to 1 for the column of the tag that has been retrieved in the item row, and for the column of the higher-level concept in the item row. For example, in the data example in Figure 13, the values ​​of the columns for the tag "measurement", the tag "ecog performance status panel", the higher-level concept "clinical", and the higher-level concept "order set" in the row of the item identifier corresponding to the item "Patients must have a SWOG performance status of 0-2." are set to 1, and the values ​​of the other columns in the same row are set to 0.

[0077] In step S512, it is determined whether there are any unprocessed item identifiers in the tagged item data 111. If there are any unprocessed item identifiers, the item vector generation unit 106 returns to step S508 and repeats the above process using the identifier of another unprocessed item. On the other hand, if there are no unprocessed items, the process proceeds to step S513.

[0078] In step S513, the vector obtained in step S512 is stored in the storage unit 103 as the item vector 114, and the item vector generation unit 106 terminates its processing.

[0079] As described above, the item classification device 100 is used to obtain an item vector 114 from the group of eligibility criteria items 109, tags and identifiers 112, and the hierarchical relationships 113 between tags and identifiers.

[0080] The user can select tags or higher-level concepts to exclude from the item vector 114 using the display unit 121. By inputting the tags or higher-level concepts to be excluded via the input unit 122 and confirming the tags and higher-level concepts that will serve as explanatory variables for classification, the result of the excluded vector can be stored as the user-selected item vector 114. An example of feature selection using the display unit 121 is shown in Figure 16.

[0081] Figure 6 is a flowchart illustrating the classification model learning process performed by an item classification device according to an embodiment of the present invention. The classification model learning process in Figure 6 corresponds to the classification model learning unit 107 in Figure 1 and step S204 in Figure 2.

[0082] In step S601, the item vector 114 is read from the memory unit 103. In step S602, the item with classification category 115 is read from the memory unit 103.

[0083] In step S603, using the item vector 114 as the explanatory variable and the classification category as the dependent variable, a classification model is trained.

[0084] In step S604, the obtained trained classification model is stored in classification model 116. After that, the classification model training unit 107 terminates its processing.

[0085] Figure 7 is a flowchart illustrating the classification model execution process performed by an item classification device according to an embodiment of the present invention. The classification model execution process in Figure 7 corresponds to the classification model execution unit 108 in Figure 1 and step S205 in Figure 2.

[0086] In step S701, the item vector 114 is read from the memory unit 103. In step S702, the eligibility criterion item 116 is read from the memory unit 103.

[0087] In step S703, the item vector 114 from the storage unit 103 is applied to the classification model 116 to perform item classification. In step S704, the item identifier and the classification category of the classification result are stored in the classification result 117. After that, the classification model execution unit 108 terminates processing.

[0088] The classified items are displayed on the display unit 121, and based on user input obtained from the display unit 122, the items are narrowed down by classification category and displayed on the display unit 182. An example of the display of the item filtering condition input screen is shown in Figure 17. An example of the display of the item filtering result screen, which is narrowed down by the classification result, is shown in Figure 18.

[0089] Figure 8 shows an example of data for a group of eligibility criteria held by an item classification device according to an embodiment of the present invention.

[0090] Each item group 802 is assigned an identifier 801. This data corresponds to item group 109 of the eligibility criteria in Figure 1.

[0091] Figure 9 is an explanatory diagram illustrating an example of data for qualification criteria items held by an item classification device according to an embodiment of the present invention.

[0092] For item 902, the item identifier 901 is stored. This data corresponds to item 110 of the eligibility criteria in Figure 1.

[0093] Figure 10 is an explanatory diagram illustrating an example of data for tagged items held by an item classification device according to an embodiment of the present invention.

[0094] The data stores the item identifier 1001 and the tag identifier 1002 assigned to the item's phrase. This data corresponds to the tagged item 111 in Figure 1.

[0095] Figure 11 is an explanatory diagram illustrating an example of tag and identifier data held by an item classification device according to an embodiment of the present invention.

[0096] Tag 1102 is stored with the tag identifier 1101 assigned to it. This data corresponds to the tag and identifier 112 in Figure 1.

[0097] Figure 12 is an explanatory diagram illustrating an example of data showing the hierarchical relationships between tags held by an item classification device according to an embodiment of the present invention.

[0098] The tag identifier 1202 and the identifier 1201 of its higher-level concept are stored here. This data corresponds to the hierarchical relationship data 113 between tag identifiers in Figure 1.

[0099] Figure 13 is an explanatory diagram illustrating an example of data representing the vector representation of items held by an item classification device according to an embodiment of the present invention.

[0100] This data stores an identifier 1301 for each item, a vector column 1302 with the tag as the column name, and a vector column 1303 with the higher-level concept as the column name. This data corresponds to vector 114 of the items in Figure 1.

[0101] Figure 14 shows the data structure of items with classification categories held by an item classification device according to an embodiment of the present invention.

[0102] Each item is assigned an identifier 1501 and a classification category 1502 representing the patient characteristics to which the item belongs. This data corresponds to item 115 with classification categories in Figure 1.

[0103] Figure 15 shows the data structure of the classification results held by the item classification device according to an embodiment of the present invention.

[0104] Let 1501 be the identifier for each item, and 1502 be the classification category to which the item is classified by the classification model 116. This data corresponds to the classification result 117 in Figure 1.

[0105] Figure 16 is an explanatory diagram showing an example of the display of the tag / higher-level concept selection screen before the selection of excluded features by the item classification device according to an embodiment of the present invention.

[0106] The tag / higher concept selection screen 1600 includes a button 1601 for inputting an item vector 114, a button 1602 for loading the tag hierarchical structure, a display / selection screen 1603 for tags and their higher concepts, a button 1604 for loading a higher concept from a given tag, checkboxes 1605 for selecting tags and higher concepts, a button 1606 for displaying vectors based on the selected tags, selected vectors 1607 for excluded tags and higher concepts, a button 1608 for confirming features for each tag, and a button 1609 for confirming tags for all features.

[0107] The user first presses button 1601 to input item vector 114, which loads the tags that are a column of vector representations of items, and displays them on the tag and its higher-level concept display / selection screen 1603.

[0108] Next, by pressing button 1602 to load the tag hierarchy, the tags, identifiers 112, and the hierarchical relationships 113 between tags are loaded. Then, by pressing button 1604 to load the higher-level concepts from a given tag, all of the higher-level concepts of the tag are displayed on the tag and its higher-level concepts display / selection screen 1603.

[0109] The user can select tags and their parent concepts they wish to exclude from the displayed tags and their parent concepts using the checkboxes 1605 for selecting tags and parent concepts, and then press the button 1606 that displays a vector based on the selected tags to confirm the excluded tags and parent concepts as selected vector representations 1607.

[0110] Afterward, pressing button 1608 to confirm the features for each tag loads the next tag. Once all tags are finished, pressing button 1609 to confirm all feature tags stores the vector corresponding to the user's selection into vector 114 of the user-selected items.

[0111] Figure 17 is an explanatory diagram showing an example of the display of the condition input screen for item narrowing using an item classification device according to an embodiment of the present invention.

[0112] The filtering criteria input screen 1700 includes a list of classification items 1701 in checkbox format used for filtering, a filtering execution button 1702, and an example of items to be filtered 1703.

[0113] The user selects the desired classification category from the checkbox-style classification item list 1701 used for filtering, and then presses the filter execution button 1702 to narrow down the items.

[0114] Figure 18 is an explanatory diagram showing an example of the display of the output screen for the result of item narrowing by the item classification device according to an embodiment of the present invention.

[0115] The search results screen 1800 has the item filtering result 1801.

[0116] According to the above embodiment, by using higher-level conceptual tags, the classification accuracy can be improved for eligible criterion sentences that have only rare tags.

[0117] In the above embodiment, a method for classifying eligibility criteria for clinical trials was described, but the present invention is not limited thereto and can be applied to other item classification methods related to natural language processing. [Explanation of symbols]

[0118] 100 items classification device 101 Control Unit 102 memory 103 Storage section 104 Item group division part 105 Item Tagging Section 106-item vector generation unit 107 Classification Model Learning Unit 108 Classification Model Execution Unit 121 Display section 122 Input section

Claims

1. An item classification device for classifying eligibility criteria sentences for clinical trials, comprising an item group division unit, an item tagging unit, an item vector generation unit, a classification model learning unit, and a classification model execution unit, The item group division unit is, The group of items in the aforementioned eligibility criteria statement is divided into each item, The aforementioned item tagging unit, A tag, which is a string used to distinguish the meaning of the aforementioned items, is assigned to the words or phrases in the aforementioned items. The vector generation unit for the aforementioned item is: The aforementioned tags are divided into subordinate concept tags, which are tags of lower-level concepts, and superordinate concept tags, which are higher-level concepts of the aforementioned subordinate concept tags. For each of the above items, a vector of the item is generated that includes the subordinate concept tag and the superior concept tag. The aforementioned classification model learning unit, The classification model is trained using the vectors of the aforementioned items as explanatory variables and the classification categories as the dependent variable. The aforementioned classification model execution unit, Item classification device characterized by executing the aforementioned classification model.

2. The aforementioned item group division unit is, A identifier for each of the aforementioned items is assigned, The aforementioned item tagging unit, A tag identifier is assigned to each of the aforementioned tags, and the identifier of the item is associated with the identifier of the tag. The vector generation unit for the aforementioned item is: The identifier of the aforementioned tag is divided into the identifier of the subordinate concept tag and the identifier of the superior concept tag and made to correspond to them. The identifiers of the subordinate concept tags and the identifiers of the superior concept tags are converted to the subordinate concept tags and the superior concept tags, respectively. The item classification device according to claim 1, characterized in that it generates an item vector including the subordinate concept tag and the superior concept tag for each identifier of the item.

3. The vector generation unit for the aforementioned item is: The item classification device according to claim 1, characterized in that it generates a one-hot vector as the vector of the items, wherein the dimension corresponding to the tag that appears in the item is 1, and the dimension corresponding to the tag that does not appear in the item is 0.

4. The vector generation unit for the aforementioned item is: The item classification device according to claim 1, characterized in that the higher-level concept is added to the vector of items relating to the words that appear infrequently in the eligibility criteria sentence.

5. The item classification device according to claim 1, Display unit and It has an input section, The aforementioned display unit is The selected tag and the higher-level conceptual tag are displayed via the input unit, The item classification system according to claim 1, characterized in that it displays the vector of the item generated by the vector generation unit for the item.

6. The aforementioned display unit is The item classification system according to claim 5, characterized in that it displays the items of the eligibility criteria statement for each classification category narrowed down via the input unit.

7. A method for classifying eligibility criteria statements for a clinical trial, comprising the steps of item group division, item tagging, item vector generation, classification model learning, and classification model execution, In the item group division step, The group of items in the aforementioned eligibility criteria statement is divided into individual items, In the item tagging step, A tag, which is a string used to distinguish the meaning of the aforementioned items, is assigned to the words or phrases in the aforementioned items. In the vector generation step of the above item, The aforementioned tags are divided into subordinate concept tags, which are tags of lower-level concepts, and superordinate concept tags, which are higher-level concepts of the aforementioned subordinate concept tags. For each of the above items, a vector of the item is generated that includes the subordinate concept tag and the superior concept tag. In the aforementioned classification model learning step, The classification model is trained using the vectors of the aforementioned items as explanatory variables and the classification categories as the dependent variable. In the aforementioned classification model execution step, A method for classifying items, characterized by executing the aforementioned classification model.

8. In the item group division step, A identifier for each of the aforementioned items is assigned, In the item tagging step, A tag identifier is assigned to each of the aforementioned tags, and the identifier of the item is associated with the identifier of the tag. In the vector generation step of the above item, The identifier of the aforementioned tag is divided into the identifier of the subordinate concept tag and the identifier of the superior concept tag and made to correspond to them. The identifiers of the subordinate concept tags and the identifiers of the superior concept tags are converted to the subordinate concept tags and the superior concept tags, respectively. The item classification method according to claim 7, characterized in that a vector of the item including the subordinate concept tag and the superior concept tag is generated for each identifier of the item.

9. In the vector generation step of the above item, The item classification method according to claim 7, characterized in that a one-hot vector is generated as the vector of the items, with the dimension corresponding to the tag appearing in the item being 1 and the dimension corresponding to the tag not appearing in the item being 0.

10. In the vector generation step of the above item, The item classification method according to claim 7, characterized in that the higher-level concept is added to the vector of items relating to the words that appear infrequently in the eligibility criteria sentence.

11. A method for classifying items that classify eligibility criteria for clinical trials, A string tag is assigned to the words and phrases in the eligibility criteria statement in order to distinguish the meaning of the words and phrases. Using the semantic hierarchical structure of the aforementioned tags, the higher-level concepts of the tags are extracted. A method for classifying items, characterized by adding the aforementioned higher-level concept to the vector relating to the tags of the aforementioned items.

12. The vector relating to the aforementioned tag is, The item classification method according to claim 11, characterized in that the dimension corresponding to the tag appearing in the item is 1, and the dimension corresponding to the tag not appearing in the item is 0.

13. The item classification method according to claim 11, characterized in that the higher-level concept is added to the vector relating to the words and phrases that appear infrequently in the eligibility criteria sentence.

Images