Learning model generation method, program, information processing method, information processing device, and learning model data generation method

Abstract

To provide a method for creating a learning model and the like that can improve the accuracy of detection of an abnormality in a test value.SOLUTION: A method for creating a learning model causes a computer to execute processing of: acquiring a plurality of pieces of specimen data in a clinical test; based on the plurality of pieces of acquired specimen data, generating a first specimen data group including abnormal specimen data; and based on training data including the generated first specimen data group, a second specimen data group not including the abnormal specimen data, and classification information on the normality or abnormality of the specimen data groups, creating a learning model that outputs the classification information when receiving input of the specimen data groups.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a learning model generating method, a program, an information processing method, an information processing device, and a learning model data generating method. [Background technology]

[0002] In hospital laboratories, testing centers, and other testing facilities, a large number of clinical tests are constantly being performed, and clinical staff use the test results to determine the diagnosis and treatment of the patient's condition. Before deciding on a treatment plan based on the test results, the validity of the test results is verified, and if any abnormal values ​​are found, retests are performed to increase the reliability of the test.

[0003] A delta verification method has been disclosed in which a standard value is set for the difference between the previous value and the current value for each test item, and abnormal values ​​are extracted through data processing that compares the values ​​of samples from each patient with the previous values, thereby increasing the reliability of the test (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 5-151282 [Patent Document 2] Japanese Patent Application Publication No. 11-296605 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technologies described in Patent Documents 1 and 2, abnormalities are determined based on the difference in test values ​​in the test results of a single item, and fluctuations in other related test values, test intervals, or the chronological order of the tests are not taken into consideration, which raises concerns about reduced detection accuracy.

[0006] An object of the present disclosure is to provide a method for generating a learning model that can improve the accuracy of detecting abnormalities in test values. [Means for solving the problem]

[0007] A method for generating a learning model according to one embodiment of the present disclosure causes a computer to execute a process of acquiring a plurality of sample data from a clinical test, generating a first sample data group including abnormal sample data based on the acquired plurality of sample data, and generating a learning model that outputs classification information when the sample data group is input, based on training data including the generated first sample data group, a second sample data group not including the abnormal sample data, and classification information regarding whether the sample data group is normal or abnormal.

[0008] A program according to one aspect of the present disclosure causes a computer to execute a process of acquiring a specimen data group consisting of a plurality of specimen data arranged in chronological order, inputting the acquired specimen data group into a learning model that has been trained to output classification information regarding whether the last specimen data in chronological order included in the specimen data group is normal or abnormal when a specimen data group consisting of a plurality of specimen data arranged in chronological order is input, and outputting the classification information.

[0009] A method for generating learning model data according to one embodiment of the present disclosure includes a process of causing a computer to execute a process of acquiring a plurality of specimen data in a clinical test, generating a first specimen data group including abnormal specimen data based on the acquired plurality of specimen data, and storing the generated first specimen data group and a label indicating abnormality in association with a second specimen data group not including the abnormal specimen data and a label indicating normal as training data for a learning model in a memory unit. Effect of the Invention

[0010] According to the present disclosure, it is possible to improve the accuracy of detecting abnormalities in test values. [Brief description of the drawings]

[0011] [Figure 1]1 is a block diagram illustrating an example of the configuration of an information processing system. [Diagram 2] FIG. 13 is a diagram showing an example of the contents of a specimen DB. [Diagram 3] FIG. 13 is a diagram showing an example of the contents of a training DB. [Figure 4] FIG. 2 is an explanatory diagram showing an overview of a first learning model. [Diagram 5] 13 is a flowchart illustrating an example of a procedure for generating training data. [Figure 6] 11 is a flowchart showing an example of a procedure for generating a first learning model. [Figure 7] 13 is a flowchart illustrating an example of a procedure for detecting an abnormality. [Figure 8] FIG. 2 is a schematic diagram showing an example of a screen displayed on a display device. [Figure 9] FIG. 2 is a schematic diagram showing an example of a screen displayed on a display device. [Figure 10] FIG. 2 is a schematic diagram showing an example of a screen displayed on a display device. [Figure 11] 13 is a flowchart showing an example of a generation process procedure for a first learning model in the second embodiment. [Figure 12] FIG. 11 is an explanatory diagram showing an overview of a second learning model in the third embodiment. [Figure 13] 13 is a flowchart showing an example of a learning process procedure of a first learning model in the third embodiment. [Figure 14] An explanatory diagram showing an overview of a third learning model in the fourth embodiment. [Figure 15] 13 is a flowchart showing an example of a disease information acquisition processing procedure in the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present disclosure will be specifically described with reference to the drawings showing the embodiments thereof.

[0013] (First embodiment) 1 is a block diagram showing a configuration example of an information processing system. The information processing system includes an information processing device 1 and a testing facility device 2. The information processing device 1 and the testing facility device 2 are communicatively connected via a network N such as a LAN (Local Area Network) or the Internet.

[0014] The testing facility device 2 is installed in a testing facility such as a hospital testing room or testing center. The testing facility device 2 is, for example, a server computer, a personal computer, etc., and is connected to a display device 3, an input device 4, a testing device 5, etc. The display device 3 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display, etc. The input device 4 is an operation unit that accepts user operations, and is, for example, a pointing device such as a keyboard or a touch panel. The testing device 5 performs various testing (analysis) processes on the specimen and generates clinical test results (specimen data). The specimen data obtained by the testing device 5 is transmitted to the information processing device 1 via the testing facility device 2.

[0015] The information processing device 1 is, for example, a server computer, a personal computer, etc., and is capable of various information processing and information transmission and reception. The information processing device 1 acquires sample data obtained by a testing device 5 via a testing facility device 2. The information processing device 1 acquires classification information regarding normality or abnormality of the acquired sample data. The information processing device 1 functions as an abnormality detection device for the sample data. The information processing device 1 displays the generated classification information on a display device 3. The information processing device 1 may be a local server installed in the same facility as the testing facility device 2, or may be a cloud server communicatively connected to the testing facility device 2 via the Internet or the like.

[0016] Classification information is information indicating whether sample data obtained by a clinical test is normal or abnormal (presence or absence of abnormality). The sample data may include a plurality of test values ​​corresponding to a plurality of types of test items. Classification information includes normal and abnormal, and abnormalities are further classified into a plurality of types depending on the cause, such as sample mix-up and dilution error. Here, normal means that the test value of the sample data is not an abnormal value caused during the test, i.e., the clinical test result is appropriate. Abnormal means that the test value of the sample data is an abnormal value caused by the testing process (test error), i.e., the clinical test result is not appropriate.

[0017] The person in charge at the testing facility can understand the presence or absence of abnormalities in the sample data and their causes based on the classification information displayed on the display device 3. If the sample data is normal, it is reported to the clinic as correct sample data, and if the sample data is abnormal, retesting is performed in a manner appropriate to the cause of the abnormality, allowing for efficient and appropriate testing.

[0018] The test results in clinical testing vary due to various factors, including not only the physiological state of the patient, but also the test implementation conditions as described above, human error in the test, and abnormalities in the test reagents and testing equipment. Data handled daily in clinical testing generally yields test values ​​that are deeply related over time for the same patient. In addition, the range of possible test values ​​related to the human body is limited, although to a greater or lesser extent, and each test value is considered to be highly related to each other. Therefore, it is believed that there is a relationship between the difference in test values ​​of the same test and between multiple test items. Conventionally, the validity of test results has been determined using statistical methods such as delta validation.

[0019] However, since it is difficult to distinguish whether an abnormality in a test value is due to a patient's pathology, such as a panic value, or due to the testing process, such as a test error, using statistical inference alone, a retest is performed when a test value is determined to be abnormal to determine whether there is an abnormality in the testing process. If retests are performed frequently, it is possible to prevent abnormalities in the testing process from being overlooked, but on the other hand, there is a problem in that the retest requires additional costs and time. In this information processing system, efficient testing is achieved by providing classification information indicating the cause of the abnormality using a first learning model 113 described later.

[0020] The information processing device 1 includes a control unit 10, a storage unit 11, and a communication unit 12. The information processing device 1 may be a multi-computer including a plurality of computers, or may be a virtual machine virtually constructed by software.

[0021] The control unit 10 is a processor using one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), etc. The control unit 10 uses built-in memories such as ROMs (Read Only Memory) or RAMs (Random Access Memory) to control each component and execute processes.

[0022] The storage unit 11 is a non-volatile storage device such as a hard disk or a solid state drive (SSD). The storage unit 11 stores programs and data referenced by the control unit 10, including a program (program product) 1P. The control unit 10 reads and executes the program 1P to cause a general-purpose server computer to function as an information processing device specific to the present disclosure. The storage unit 11 may be composed of multiple storage devices, or may be an external storage device connected to the information processing device 1.

[0023] The program 1P stored in the storage unit 11 may be recorded in a computer-readable manner on a recording medium. The storage unit 11 stores the program 1P read from the recording medium 1A by a reading device (not shown). The program 1P may also be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 11. The program 1P may be deployed so as to be executed on a single computer or on a single site, or distributed across multiple sites and on multiple computers interconnected by a communication network.

[0024] The memory unit 11 also stores a sample DB (Data Base) 111, a training DB 112, and a plurality of first learning models 113. The sample DB 111 is a database that stores sample data related to samples in clinical tests. The training DB 112 is a database that stores training data (learning data) used to learn the first learning model 113. The first learning model 113 is a machine learning model that has learned the training data. The first learning model 113 is expected to be used as a program module that constitutes artificial intelligence software. The memory unit 11 may further store a second learning model 114 and a third learning model 115. The second learning model 114 and the third learning model 115 will be described in detail in other embodiments.

[0025] The communication unit 12 is a communication module for performing processing related to communication. The control unit 10 can transmit and receive information to and from the testing facility device 2 via the communication unit 12.

[0026] In this embodiment, the information processing device 1 is not limited to the above configuration, and may include, for example, a display unit that displays images, an input unit that accepts operational input, and the like.

[0027] The testing facility device 2 includes a control unit 20, a memory unit 21, a communication unit 22, and an input / output unit 23. The control unit 20 is a processor using one or more CPUs, GPUs, etc. The control unit 20 uses built-in memories such as ROM and RAM to control each component and execute processes.

[0028] The storage unit 21 is a non-volatile storage device such as a hard disk or SSD. Programs and data referenced by the control unit 20 are stored in the storage unit 21. The communication unit 22 is a communication module for performing processing related to communication. The control unit 20 can transmit and receive information to and from the information processing device 1 via the communication unit 22.

[0029] The input / output unit 23 is an input / output I / F (interface) for connecting an external device. The above-mentioned display device 3, input device 4, testing device 5, etc. are connected to the input / output unit 23. The control unit 20 causes the display device 3 to display classification information regarding whether the sample data is normal or abnormal via the input / output unit 23. The control unit 20 accepts information input to the input device 4 via the input / output unit 23. The control unit 20 accepts sample data output from the testing device 5 via the input / output unit 23.

[0030] FIG. 2 is a diagram showing an example of the contents of the specimen DB 111. The specimen data DB 111 stores, for example, a specimen ID, a testing facility name, patient attribute information, test date and time, and test values ​​for each of a plurality of specimen data in association with each other. The specimen ID is identification information for identifying a specimen. The testing facility name is information for identifying a testing facility that performed a test on the specimen. The patient attribute information is attribute information of a patient (subject) related to the specimen, and includes, for example, a patient ID, a patient name, an age, and a sex. Each patient can be uniquely identified by, for example, a patient ID and a patient name. The test date and time is the date and time when the test on the specimen was performed, or the date and time when blood was drawn for a blood test. The test value is a test value related to the specimen. The test value column stores each test value in association with each of a plurality of test items. The specimen data stored in the specimen DB 111 may further include information such as the type of department in charge at the testing facility (type of medical department at a hospital), a testing device ID, a lot number of a test reagent, a date of receipt of the specimen data, a test method, and a name of a person in charge of testing.

[0031] Every time the information processing device 1 receives specimen data from the testing facility device 2, it stores the received specimen data in the specimen DB 111. The specimen DB 111 may store specimen data for each patient in chronological order. The contents stored in the specimen DB 111 are not limited to the example shown in FIG.

[0032] 3 is a diagram showing an example of the contents of the training DB 112. The training DB 112 stores training data used for learning a first learning model 113 described below. The training DB 112 stores, for example, sample data group information and classification information related to the sample data group in association with a training data ID that identifies the training data.

[0033] The specimen data group information string includes the same or some of the items as the specimen data stored in the specimen DB 111, for example, the test date and time of the specimen data and test values ​​for multiple test items. The specimen data group information includes information on multiple specimen data constituting one specimen data group. A specimen data group is composed of specimen data in chronological order originating from the same patient. Specifically, a specimen data group is composed of a predetermined number of specimen data that are adjacent in chronological order among the specimen data obtained from the same patient. FIG. 3 shows an example in which one specimen data group is composed of four specimen data, but the number of specimen data constituting a specimen data group is not limited.

[0034] As will be described in detail later, the sample data group is divided into a sample data group that does not include abnormal sample data (abnormal sample data) resulting from a testing error, and a sample data group that includes the abnormal sample data. In the example shown in Fig. 3, training data ID001-1 and training data ID001-2 are sample data groups that do not include abnormal sample data, and training data ID001-3 is a sample data group that includes abnormal sample data. Training data ID001-3 is obtained by replacing all test values ​​in the sample data in the fourth row of training data ID001-2 with abnormal values.

[0035] The classification information column stores classification information regarding whether the sample data group is normal or abnormal. The classification information includes, for example, normal and abnormal. Abnormalities resulting from testing errors are classified into multiple types depending on their causes, such as mix-up, dilution error, unit misidentification, reagent error, and sample clogging in the testing device.

[0036] A mix-up is an abnormality in a test value caused by a mix-up of a sample (mix-up of all test values ​​related to the sample data) between different patients who underwent a test on the same reception date. A mix-up may include a mix-up of test values ​​for a specific test item between different patients who underwent a test on the same reception date. A dilution error is an abnormality caused by an error in the dilution ratio or dilution amount when diluting a sample with physiological saline, a buffer solution, a reagent, or the like. A unit misidentification is an abnormality caused by a misidentification of a unit related to a test value. A reagent error is an abnormality caused by an error in the reagent used in a test using a reagent, or a shortage or excess of a reagent. A specimen clogging is an abnormality caused by a specimen clogging inside a tube or the like in a test device used for the test. The classification information sequence may store normal or abnormal (presence or absence of abnormality) for each type of abnormality. The contents stored in the training DB 112 are not limited to the example shown in FIG. 3.

[0037] 4 is an explanatory diagram showing an overview of the first learning model 113. The first learning model 113 is a machine learning model that receives as input a sample data group consisting of a plurality of sample data in chronological order, and outputs classification information regarding normality or abnormality of the last (latest, i.e., most recent) sample data in the time series included in the sample data group. Specifically, the first learning model 113 receives as input a sample data group, and outputs a classification result of normality or abnormality (presence or absence of abnormality).

[0038] A plurality of first learning models 113 are stored for each type of abnormality. That is, a first first learning model 113 that receives a sample data group as input and outputs the presence or absence of mix-up, a second first learning model 113 that outputs the presence or absence of a dilution error, a third first learning model 113 that outputs the presence or absence of a unit misconception, a fourth first learning model 113 that outputs the presence or absence of a reagent mistake, and a fifth first learning model 113 that outputs the presence or absence of a sample clogging are stored. Since each first learning model 113 has the same configuration, the configuration of the first learning model 113 that outputs the presence or absence of an abnormality caused by mix-up will be described below.

[0039] The information processing device 1 performs machine learning to learn predetermined training data to generate in advance a first learning model 113. The information processing device 1 then inputs a sample data group acquired from the testing device 5 to the first learning model 113, and outputs classification information for the sample data group.

[0040] The first learning model 113 is constructed by, for example, XGBoost (eXtreme Gradient Boosting). XGBoost is an ensemble learning method that combines gradient boosting and random forest. The first learning model 113 is configured to perform boosting by constructing multiple decision trees (weak learners) and constructing a new decision tree using information from the previous decision tree. Specifically, a new decision tree is constructed with an error (gradient of a loss function) that could not be predicted by the previous decision tree as the objective variable. In each decision tree, input data is classified according to conditions on the way from the root to the branches, and when it reaches a terminal leaf node, the value given to the terminal leaf node is output as a predicted value.

[0041] The input of the first learning model 113 is a sample data group. The sample data group includes, for example, the current, previous, previous-before, and previous-before sample data related to the same patient in the same testing facility. The sample data includes the test date and time (the interval from the previous test date and time) and test values ​​corresponding to multiple test items. The multiple sample data are input to the first learning model 113 as sample data group data arranged in chronological order. The number of sample data included in the sample data group is not limited to four. The sample data group may include sample data to be determined for the presence or absence of abnormality (the last sample data in the chronological order) and at least one sample data acquired earlier than the sample data to be determined. Generally, in clinical practice, it is not past data but the most recent data that needs to be checked first as a test error or pathological abnormality, so in this method, it is assumed that the past data is correct, and whether the most recent (latest in the chronological order) data is correct is verified.

[0042] The output of the first learning model 113 is a classification result of the last sample data in the time series included in the sample data group as normal or abnormal. When the first learning model 113 receives a sample data group as input, it outputs the accuracy for each of the normal and abnormal classes. The first learning model 113 can output a class whose accuracy is equal to or greater than a threshold value as an output value.

[0043] The first learning model 113 may be a multi-class classification model, and may classify normality or multiple types of abnormality using one first learning model 113. In this case, the first learning model 113 receives as input a sample data group consisting of multiple sample data in chronological order, and outputs classification information indicating whether the last sample data in chronological order included in the sample data group is normal or multiple types of abnormality.

[0044] In the above, an example in which the first learning model 113 is XGBoost has been described, but the configuration of the first learning model 113 is not limited as long as it is possible to identify classification information regarding normality or abnormality of the last sample data in a time series included in the sample data group. The first learning model 113 may be, for example, a neural network such as Transformer, CNN (Convolution Neural Network), RNN (Recurrent Neural Network), or LSTM (Long Short Term Memory), or may use other learning algorithms such as a support vector machine, logistics regression, or random forest.

[0045] In a learning phase, which is a stage preceding an operation phase in which anomaly detection is performed, the information processing device 1 generates a first learning model 113 using training data and stores the generated first learning model 113 in the memory unit 11. Then, in the operation phase, anomalies in the sample data are detected using the stored first learning model 113.

[0046] 5 is a flowchart showing an example of a procedure for generating training data. The following process is executed by the control unit 10 in accordance with a program 1P stored in the storage unit 11 of the information processing device 1.

[0047] The control unit 10 of the information processing device 1 acquires multiple pieces of chronologically ordered sample data relating to the same patient based on information stored in the sample DB 111 (step S11). The control unit 10 extracts sample data relating to the same testing facility name and the same patient from the large amount of sample data stored in, for example, the sample DB 111, and arranges them in chronological order.

[0048] The control unit 10 extracts specimen data including test values ​​of test items with high co-occurrence frequencies from the acquired specimen data (step S12). Test items that are frequently performed at the same time, i.e., test items with high co-occurrence frequencies, can be extracted by using, for example, association analysis, market basket analysis, and co-occurrence cluster mining (see Kenichi Fukui et al., Artificial Intelligence, Vol. 30, No. 2, 2015). On the other hand, there are several methods for easily extracting test items with high co-occurrence frequencies, and they can be extracted, for example, as follows. First, a frequency table of test items that co-occur with a certain test item is created, and a brute-force frequency table between the two items is created for all test items. Next, in the frequency table, the co-occurrence frequency including all items in the set is confirmed for a test item set limited to the top test item, and test items with a co-occurrence frequency equal to or higher than a certain threshold are narrowed down, so that a test item group with high co-occurrence frequencies (multiple specimen data) can be extracted. This makes it possible to generate a specimen data group with few missing values.

[0049] The control unit 10 generates a sample data group that does not include abnormal sample data based on the multiple sample data with high co-occurrence frequency in chronological order that have been acquired (step S13). In detail, the control unit 10 generates a first sample data group consisting of four sample data including the newest sample data from the multiple sample data in chronological order that have been acquired. Similarly, the control unit 10 slides the reference sample data in order to generate multiple sample data groups, such as a second sample data group consisting of four sample data including the second newest sample data in chronological order, .... In this way, a sample data group that does not include abnormal sample data is generated using the test values ​​obtained in the clinical test as they are. The control unit 10 may generate sample data by extracting only predetermined management items necessary for the input elements to the first learning model 113.

[0050] The control unit 10 generates a specimen data group including abnormal specimen data by replacing a portion of the acquired specimen data with an abnormal value (step S14). In detail, the control unit 10 generates abnormal specimen data by replacing a portion of the last specimen data in time series among the specimen data included in the specimen data group with an abnormal value. The control unit 10 generates a specimen data group consisting of the generated abnormal specimen data and the remaining three specimen data that use the test values ​​obtained in the clinical test as they are.

[0051] A specific example of a method for generating abnormal sample data is shown below. The control unit 10 generates abnormal sample data resulting from a mix-up by replacing all test values ​​included in the sample data for a first patient at a specified test date and time with test values ​​for sample data for a second patient different from the first patient who was tested at the specified test date and time. The control unit 10 may replace only test values ​​for a specific test item included in the sample data for the first patient at the specified test date and time with test values ​​for the second patient.

[0052] The control unit 10 generates abnormal sample data caused by a dilution error (calculation error) by replacing the test values ​​for one or more test items included in the sample data with test values ​​calculated on the assumption that dilution was performed at a dilution ratio different from the specified one. The control unit 10 generates abnormal sample data caused by a unit misrecognition (calculation error) by replacing the test values ​​for one or more test items included in the sample data with test values ​​calculated on the assumption that a unit was misrecognized.

[0053] The control unit 10 generates abnormal specimen data caused by an incorrect reagent by replacing the test values ​​for one or more test items included in the specimen data with test values ​​calculated under the assumption that a reagent other than the specified one was used. The control unit 10 generates abnormal specimen data caused by specimen clogging by replacing the test values ​​for one or more test items included in the specimen data with test values ​​calculated under the assumption that a specimen clogging has occurred (e.g., zero, a fraction of the actual value, or a specific maximum value, etc.). In this way, the control unit 10 generates abnormal specimen data using actually obtained specimen data. Note that the method and type of abnormal specimen data generation are not limited to the above examples.

[0054] The control unit 10 generates training data, which is a data set in which classification information is labeled as a correct answer value for each of the generated sample data groups not including abnormal sample data and the generated sample data groups including abnormal sample data (step S15). Specifically, the control unit 10 generates training data in which a label indicating normal or abnormal for each type of abnormality is assigned to each sample data group. The training data may be data in which a label indicating normal or multiple types of abnormalities is assigned to each sample data group.

[0055] The control unit 10 stores the generated training data in the training DB 112 (step S16), and ends the series of processes. The control unit 10 generates a plurality of information groups based on a large amount of specimen data collected in the specimen DB 111, and accumulates the generated information groups in the training DB 112 as training data. Usually, when specimen data is determined to be abnormal in a clinical test, only normal specimen data after retesting is often ultimately recorded in the database, making it difficult to collect abnormal specimen data. According to the above-mentioned process, abnormal specimen data is virtually generated, thereby making it possible to efficiently accumulate training data. Note that, in order to objectively evaluate the accuracy of anomaly detection, it is preferable that the abnormal specimen data is generated taking into consideration the frequency of occurrence of abnormalities in an actual clinical test.

[0056] In the above-described processing, the specimen data may include, for example, a testing facility name, patient attribute information, a testing device ID, etc. In other words, the input elements to the first learning model 113 may include a testing facility name, patient attribute information, a testing device ID, etc. In this case, when generating a specimen data group, processing for classifying specimen data for the same testing facility or the same patient can be eliminated.

[0057] 6 is a flowchart showing an example of a process for generating the first learning model 113. The following process is executed by the control unit 10 according to the program 1P stored in the storage unit 11 of the information processing device 1 after, for example, the process of FIG.

[0058] The control unit 10 of the information processing device 1 acquires a set of training data extracted from an information group based on information stored in the training DB 112 (step S21). The control unit 10 uses the acquired training data to generate a first learning model 113 that outputs information indicating whether the last sample data in the time series included in the sample data group is normal or abnormal when a sample data group is input (step S22).

[0059] Specifically, the control unit 10 uses a sample data group included in the training data to construct a first decision tree. The control unit 10 applies input data to the first decision tree, and obtains a first residual between the input data and a predicted value included in a leaf node. The control unit 10 then uses the obtained first residual to construct a second decision tree by a similar procedure, and obtains a second residual between the first residual and a predicted value included in a leaf node. The second decision tree is constructed so as to achieve higher accuracy by combining the first decision tree and the second decision tree. The control unit 10 subsequently constructs decision trees sequentially by a similar procedure.

[0060] The control unit 10 adjusts parameters using, for example, a gradient descent method so as to optimize (minimize) the loss function in the first learning model 113, and sequentially learns and integrates decision trees. Note that, in order to prevent so-called overfitting (overlearning), when optimizing the loss function, the control unit 10 does not use all explanatory variables, but selects and uses the number of explanatory variables at a randomly determined ratio. The control unit 10 completes learning when the loss function satisfies a predetermined criterion.

[0061] The control unit 10 stores definition information related to the trained first learning model 113 in the storage unit 11 as the trained first learning model 113 (step S23), and ends the series of processes. Through the above-mentioned processes, the control unit 10 can construct the trained first learning model 113 capable of appropriately outputting classification information related to normality or abnormality for the sample data group. The control unit 10 executes the above-mentioned processes for each first learning model 113.

[0062] The information processing device 1 detects anomalies in the sample data using the first learning model 113 generated as described above. Hereinafter, a process procedure executed by the information processing device 1 in the operation phase will be described.

[0063] 7 is a flowchart showing an example of a processing procedure for detecting an anomaly. The following processing is executed by the control unit 10 in accordance with a program 1P stored in the storage unit 11 of the information processing device 1. The control unit 10 may perform the following processing in real time, for example, every time sample data is transmitted from the testing facility device 2, or may perform the processing at any timing based on the collected sample data.

[0064] The control unit 10 of the information processing device 1 acquires the specimen data from the testing facility device 2 (step S31), and stores the acquired specimen data in the specimen DB 111.

[0065] The control unit 10 acquires a sample data group including the acquired sample data based on the information stored in the sample DB 111 (step S32). The sample data group is, for example, sample data related to the same patient as the received sample data, and is obtained by chronologically arranging four sample data items adjacent to the received sample data in chronological order. The control unit 10 may extract sample data including test values ​​of test items that frequently co-occur with the current (latest) sample data from a plurality of sample data items acquired in the past, and acquire a sample data group including the sample data including the extracted test values ​​of the test items that frequently co-occur. The method of extracting the frequently co-occurring sample data may be the same as that used when generating the training data.

[0066] The control unit 10 inputs the acquired sample data group as input data to the first learning model 113 (step S33). The control unit 10 acquires the classification information output from the first learning model 113 (step S34).

[0067] The control unit 10 calculates the contribution of the test value corresponding to each test item in the sample data input to the first learning model 113, thereby identifying the test value that contributed to the output classification information (step S35). The method for calculating the contribution is not limited, but may be, for example, SHAP (SHapley Additive exPlanation), LIME (Local Interpretable Model-Agnostic Explanations), etc. The greater the contribution of the test value, the greater the influence on the classification of the test value.

[0068] Based on the calculated contribution degree, the control unit 10 identifies, for example, the test value whose contribution degree is equal to or greater than a predetermined value as the test value that contributed to the classification information. The control unit 10 may identify the test values ​​that contributed to the classification information by, for example, selecting a predetermined number of test values ​​in descending order of contribution degree. Note that, when an attention mechanism is used for the first learning model 113, the attention value may be used as the contribution degree. The control unit 10 executes the processes of steps S33 to S35 for each of the multiple first learning models 113. The control unit 10 may identify the test item related to the test value that contributed to the classification information.

[0069] The control unit 10 acquires the result of verifying the suitability of the sample data by another method (step S36). The control unit 10 may use, for example, the well-known delta verification method and occurrence record zone verification method as a method for verifying the suitability of the sample data.

[0070] The delta verification method compares the test value of a given test item contained in the current sample data acquired with the previous value of the same patient, and determines whether the current test value is within the upper and lower limit values ​​(e.g., ±2SD). If the current test value is outside the range of the upper and lower limit values, the current test value is determined to be abnormal. For details of the verification method using the delta verification method, see, for example, Japanese Patent Application Laid-Open No. 5-151282 and Japanese Patent Application Laid-Open No. 11-296605.

[0071] The occurrence record zone verification method finds the frequency distribution of test values ​​based on their occurrence record, sets the allowable range for previous value checks, i.e., the normal zone, non-linearly, and performs single-item checks, inter-item checks, and previous value checks. If the current test value is outside the normal zone, the current test value is determined to be abnormal. For details about the verification method using the occurrence record zone verification method, see, for example, Japanese Patent Publication No. 2828609.

[0072] The control unit 10 generates screen information including the acquired classification information and verification result (step S37). The control unit 10 causes the display device 3 to display the generated screen information (step S38), and ends a series of processes.

[0073] 8 to 10 are schematic diagrams showing an example of a screen displayed on the display device 3. FIG. 8 shows an example of a result list screen 31. The result list screen 31 displays a list of detection results including classification information by the first learning model 113 and verification results by the delta verification method and the occurrence history zone verification method for the specimen data to be displayed. The result list is, for example, in a table format, and the name of the abnormality detection method (e.g., the delta method meaning the delta verification method, the zone method meaning the occurrence history zone verification method, the machine learning method meaning the first learning model 113, etc.), the initial test value, and the latest test value in the case of retesting are displayed on the horizontal axis. In addition, each item of the specimen data (e.g., specimen ID, patient ID, test value, test item, etc.) is displayed on the vertical axis. Furthermore, multiple items indicating the type of abnormality are displayed below the machine learning method. The column of one or more test values ​​or test items in the method (item indicating the type of abnormality) in which the abnormality was detected shows the abnormality of the specimen data in a recognizable manner in a predetermined display mode. Specifically, a symbol indicating the presence or absence of an abnormality, a numerical value of the accuracy of the presence or absence of an abnormality predicted by the machine learning method, etc. are displayed.

[0074] The result list may further include detection results (degree of anomaly) by other anomaly detection methods such as Hotelling's theory, integrated results of anomaly determination (automatic estimation results based on the degree of anomaly of Hotelling's theory or the accuracy of prediction by machine learning methods), etc. The estimation method based on Hotelling's theory etc. will be described in detail in other embodiments.

[0075] The control unit 10 of the information processing device 1 performs processing such as coloring, framing, lighting / flashing, and displaying a symbol indicating the presence or absence of an abnormality on the test value or test item column of the abnormality type column corresponding to the classification information based on the classification information by the first learning model 113 and the identification result of the test value that contributed to the classification information. In this case, the control unit 10 may change the display mode of the abnormality depending on the classification information or the contribution degree, for example, by changing the density of the display color depending on the contribution degree of the test value or test item, or by changing the display color depending on the classification information (type of abnormality). Similarly, the control unit 10 performs processing such as coloring, framing, lighting / flashing, and displaying a symbol indicating the presence or absence of an abnormality on the test value or test item column of the method name column in which an abnormality was detected based on the verification results by the delta verification method and the occurrence history zone verification method. For the delta verification method and the occurrence history zone verification method, various processes may be performed on the test value or test item column used for verification.

[0076] The result list screen 31 may display time-series data of the test values ​​of the patient ID to be displayed. Furthermore, the result list screen 31 is not limited to displaying each sample data (sample ID), but may display a list of detection results for multiple sample data acquired on the same test date. This makes it easier to estimate the cause of the mix-up, for example, when a mix-up is detected.

[0077] When the control unit 10 receives a tap operation on any of the test values ​​while the list of results is displayed as shown in FIG. 8, it causes the display device 3 to display a first details screen 32 shown in FIG. 9. The first details screen 32 includes a graph showing the time series change of the selected test value. The vertical axis of the graph is the test value, and the horizontal axis is the test date. The control unit 10, for example, refers to the sample DB 111 and acquires the test values ​​in the sample data for the current test and a predetermined number of previous tests (the past four tests in the example of FIG. 9) for the same patient. The test values ​​correspond to the test items related to the test values ​​selected by the tap operation. The control unit 10 plots each of the acquired test values ​​on the graph.

[0078] The graph displays the normal variation range and the candidate abnormality range for the current test value. The control unit 10 estimates the normal variation range for the current test value based on, for example, the past four test values. The method of estimating the normal variation range is not limited, but the control unit 10 may estimate the normal variation range using a model trained by a learning algorithm such as Gaussian process regression. The control unit 10 performs processing such as different coloring on the area on the graph corresponding to the normal variation range and the area corresponding to the candidate abnormality range outside the normal range. By visualizing the normal variation range and the candidate abnormality range, the state of the current test value can be visually and easily grasped. Although FIG. 9 shows an example of the first detailed screen 32 including information on one test item, the first detailed screen 32 may display information on multiple test items simultaneously (in parallel).

[0079] In addition, when the control unit 10 receives a tap operation of the "delta method" or the "zone method" in a state where the result list is displayed as shown in FIG. 8, the control unit 10 causes the display device 3 to display a second detailed screen 32 shown in FIG. 10. The second detailed screen 32 includes a frequency distribution map of the test value by the delta verification method displayed in response to the selection of the "delta method" as shown in FIG. 10A, and a frequency distribution map of the test value by the occurrence history zone verification method displayed in response to the selection of the "zone method" as shown in FIG. 10B. In both frequency distribution maps, the normal range and the abnormal range of the current test value identified by the delta verification method and the occurrence history zone verification method are displayed in a distinguishable manner by a display mode such as different colors or boundary display. The control unit 10 generates a frequency distribution map based on the verification result. Furthermore, the control unit 10 plots the current test value at a corresponding position on the generated frequency distribution map.

[0080] According to this embodiment, the information processing device 1 can accurately detect abnormalities in the sample data using the first learning model 113. The first learning model 113 accurately classifies the sample data as normal or abnormal based on multiple test values ​​in a time series related to test items that frequently co-occur. Since the detection results obtained by various methods are integrated and displayed, the detection results can be easily compared. Since the information processing device 1 can classify multiple types of abnormalities, subsequent measures such as retesting can be efficiently taken by isolating the cause of the abnormality.

[0081] According to this embodiment, since it is possible to estimate abnormalities in the testing process, i.e., tests that are highly likely to be test errors, it is possible to efficiently increase the reliability of the test results (suppress overlooking test errors) while minimizing the frequency of retesting. Furthermore, while the need for retesting is generally determined by experienced clinical laboratory technicians, according to this embodiment, it is possible to automatically extract test subjects to be retested and automatically present the reason for retesting, so that retesting can be efficiently performed without relying on the experience of the tester. By identifying the cause of the abnormality, it is possible to determine whether or not to retest depending on the cause of the abnormality, making it possible to automate retesting.

[0082] In each of the above flowcharts, an example in which a series of processes is executed by the control unit 10 of the information processing device 1 has been described, but the processing entity of each process is not limited. The above processes may be executed in part or in whole by the control unit 20 of the testing facility device 2. They may be executed by the testing device 5. The first learning model 113 may be generated by the information processing device 1 and learned by the testing facility device 2. The first learning model 113 generated by the information processing device 1 may be installed in the testing facility device 2, and anomaly detection may be performed by the testing facility device 2. In addition, the classification information is not limited to being output to the display device 3 via the testing facility device 2. The control unit 10 may output the classification information to a device other than the display device 3 and display it. The information processing device 1 is not limited to being configured separately from the testing facility device 2, and may be included in the testing facility device 2.

[0083] Second embodiment In the second embodiment, actual abnormal specimen data obtained by a clinical test is used to train the first learning model 113. The following mainly describes the differences from the first embodiment, and the same reference numerals are used to designate the same components as in the first embodiment, and detailed descriptions thereof will be omitted.

[0084] The information processing device 1 of the second embodiment generates training data using actual abnormal sample data obtained by clinical testing. The information processing device 1 performs training of the first learning model 113 in the first embodiment as pre-training, and trains the first learning model 113 in two stages (fine tuning) using newly generated training data.

[0085] 11 is a flowchart showing an example of a generation process procedure of the first learning model 113 in the second embodiment. The following process is executed by the control unit 10 according to a program 1P stored in the storage unit 11 of the information processing device 1.

[0086] The control unit 10 of the information processing device 1 acquires abnormal specimen data including abnormal values ​​and the type of abnormality (cause of the abnormality) in association with each other from the testing facility device 2 (step S41). The acquired abnormal specimen data includes abnormal values ​​actually obtained in a clinical test.

[0087] The control unit 10 refers to the sample DB 111 to acquire one or more past sample data relating to the same patient as the acquired abnormal sample data, for example, the past three sample data, and generates a sample data group including the actual abnormal sample data and the past sample data (step S42).

[0088] The control unit 10 generates training data, which is a data set in which classification information indicating the type of abnormality is labeled as a correct answer value for the generated sample data group (step S43). The control unit 10 stores the generated training data in the training DB 112 (step S44).

[0089] The control unit 10 uses newly acquired training data to generate a first learning model 113 that outputs information indicating whether the last sample data in the time series included in the sample data group is normal or abnormal when the sample data group is input (step S45). In this case, the control unit 10 trains the first learning model 113 in advance using training data that does not include actual abnormal sample data from clinical tests, by the process described in the first embodiment. The control unit 10 then executes training of the first learning model 113 so that the parameters obtained by the advance training are set as initial values, and the parameters are optimized using the new training data.

[0090] The control unit 10 stores definition information related to the trained first learning model 113 in the storage unit 11 (step S46), and ends the series of processes.

[0091] According to this embodiment, the first learning model 113 is trained in two stages, thereby making it possible to efficiently execute the learning and improve the accuracy of the first learning model 113. For example, by pre-training the first learning model 113 using a group of sample data acquired from a plurality of testing facilities and fine-tuning using abnormal sample data actually obtained at each testing facility, it is possible to efficiently and effectively generate the first learning model 113 suitable for each testing facility even if a sufficient number of abnormal data cannot be prepared for each testing facility.

[0092] Third embodiment In the third embodiment, the first learning model 113 is learned by combining with an unsupervised learning model. In the following, differences from the first embodiment will be mainly described, and the same reference numerals will be used for configurations common to the first embodiment, and detailed description thereof will be omitted.

[0093] The information processing device 1 of the third embodiment stores a second learning model 114, which is an unsupervised learning model, in the storage unit 11. The information processing device 1 uses the classification result by the second learning model 114 to learn the first learning model 113.

[0094] 12 is an explanatory diagram showing an overview of the second learning model 114 in the third embodiment. The second learning model 114 is a machine learning model configured to input a sample data group and output information indicating the presence or absence of an abnormality (abnormal or normal) in the sample data group. The second learning model 114 outputs a classification result of the presence or absence of an abnormality in the sample data group, for example, by a method of LOF (Local Outlier Factor). The second learning model 114 calculates the degree of anomaly of the sample data group, and classifies the presence or absence of an abnormality by defining a case where the degree of anomaly is greater than a certain threshold as abnormal.

[0095] The information processing device 1 performs unsupervised learning using a large amount of previously collected sample data groups that do not include abnormal sample data, i.e., normal sample data groups. The second learning model 114 is trained to identify whether the degree of abnormality of a new sample data group is within a normal range, with the sample data groups that do not include abnormal sample data as normal data. The second learning model 114 identifies sample data groups other than those that do not include abnormal sample data, i.e., sample data groups that include abnormal sample data, as abnormal values. The second learning model 114 classifies the presence or absence of an abnormality using an outlier score (LOF) that evaluates the degree of deviation of the sample data group, which is input data, from the normal data. As a result, the second learning model 114 is constructed, which has been trained to be able to appropriately output information indicating the presence or absence of an abnormality when a sample data group is input.

[0096] The second learning model 114 may include a feature extractor that extracts features of the sample data group and the above-mentioned classification model, and may classify the presence or absence of an abnormality in the sample data group based on the features extracted by the feature extractor. The feature extractor is, for example, a convolutional neural network model that extracts features of the sample data group and outputs the extracted features. The CNN may use a trained model that has been trained in advance, for example, by transfer learning.

[0097] The configuration of the second learning model 114 is not limited to the above example, and may be any configuration that can identify the presence or absence of an abnormality in a sample data group using an unsupervised learning algorithm. The second learning model 114 may be configured using an anomaly detection method that defines the degree of anomaly, such as the k-nearest neighbor method, One-class SVM, or Hotelling's theory.

[0098] The input data to the second learning model 114 is not limited to a specimen data group including multiple specimen data, but may include at least the last specimen data in the specimen data group in time series, i.e., the most recent (most recent) specimen data.

[0099] 13 is a flowchart showing an example of a learning process procedure of the first learning model 113 in the third embodiment. The following process is executed by the control unit 10 according to a program 1P stored in the storage unit 11 of the information processing device 1. The storage unit 11 stores the first learning model 113 that has been previously learned.

[0100] The control unit 10 of the information processing device 1 acquires a sample data group (step S51). The acquired sample data group may be the same as the sample data group acquired in step S32 of FIG.

[0101] The control unit 10 inputs the acquired sample data group as input data to the second learning model 114 (step S52). The control unit 10 acquires the estimation result of the presence or absence of an abnormality output from the second learning model 114 (step S53).

[0102] The control unit 10 compares the acquired classification result by the second learning model 114 with the classification information by the first learning model 113 (step S54). The control unit 10 outputs the comparison result to the testing facility device 2 (step S55).

[0103] When the classification information by the first learning model 113 differs from the classification result by the second learning model 114, the control unit 10 acquires the classification result by the second learning model 114 as correction information (step S56). The control unit 10 re-learns the first learning model 113 using the correction information for the classification information, and updates the first learning model 113 (step S57). Specifically, the control unit 10 re-learns using the sample data group input to the first learning model 113 and the correction information for the classification information as training data, and updates the first learning model 113. The control unit 10 optimizes parameters so that the classification information output from the first learning model 113 approximates the corrected estimated information, and regenerates the first learning model 113.

[0104] The comparison result between the classification result by the second learning model 114 and the classification information by the first learning model 113 may be displayed using, for example, the result list screen 31 shown in FIG. 8. The control unit 10 acquires the abnormality degree for each test value (test item) in the second learning model 114 in addition to the classification information by the second learning model 114. The control unit 10 displays each acquired abnormality degree in the test value column of the anomaly detection method (Hotelling's theory in the example of FIG. 8) of the second learning model 114, and applies a display mode such as coloring to the test value column determined to be abnormal. In addition, the control unit 10 compares the classification result by the second learning model 114 with the classification information by the first learning model 113, and derives an integrated result of anomaly determination (normal or abnormal) based on, for example, the abnormality degree in the second learning model 114 and the accuracy of prediction by the first learning model 113. The control unit 10 displays the derived overall determination result in the abnormality determination column.

[0105] The information processing device 1 may be configured to construct a single integrated model by integrating the first learning model 113 and the second learning model 114, and to detect an abnormality in the sample data using the integrated model.

[0106] According to this embodiment, the first learning model 113 can be further optimized by combining the first learning model 113 and the second learning model 114. The first learning model 113 can suitably detect various anomalies that are different from past anomalies.

[0107] (Fourth embodiment) In the fourth embodiment, abnormalities caused by pathological conditions, diseases, and physiological conditions related to the condition of a subject (hereinafter referred to as "abnormal test values ​​caused by diseases") are detected. The following mainly describes the differences from the first embodiment, and the same reference numerals are used to designate the same components as the first embodiment, and detailed descriptions thereof will be omitted.

[0108] The information processing device 1 of the fourth embodiment stores a third learning model 115 for detecting abnormalities in test values ​​caused by a disease in the storage unit 11. The information processing device 1 uses the third learning model 115 to detect abnormalities in sample data caused by a patient's disease, which are different from abnormalities during testing due to testing errors.

[0109] FIG. 14 is an explanatory diagram showing an overview of the third learning model 115 in the fourth embodiment. The third learning model 115 is a machine learning model that receives sample data composed of test values ​​corresponding to a plurality of test items as input and outputs disease information regarding normality or abnormality of the sample data. The third learning model 115 is generated and learned in advance by deep learning using a neural network in the information processing device 1 or an external device. The learning algorithm may be a recurrent neural network (RNN) when time series data is acquired.

[0110] The third learning model 115 includes an input layer for inputting specimen data, an output layer for outputting disease information, and an intermediate layer (hidden layer) for extracting features. The intermediate layer has a plurality of nodes for extracting features of the input data, and passes the features extracted using various parameters to the output layer. When specimen data is input to the input layer, a calculation is performed in the intermediate layer, and disease information is output from the output layer.

[0111] The input of the third learning model 115 is specimen data including test values ​​corresponding to a plurality of test items. The specimen data may further include attribute information such as the age and sex of the patient.

[0112] The output of the third learning model 115 is disease information indicating whether the specimen data is normal or abnormal due to the patient's disease. The abnormality may be classified into a plurality of types according to the disease name (e.g., diabetes, hypertension, etc.). The output layer includes nodes corresponding to the set disease information (normal or each disease name), and outputs the accuracy for each disease information as a score. The information processing device 1 can set the disease information with the highest score, or the disease information with a score equal to or higher than a threshold, as the output value of the output layer. Note that the output layer may have one output node that outputs the most accurate disease risk, instead of having a plurality of output nodes that output the accuracy of each disease information.

[0113] The control unit 10 learns the third learning model 115 by collecting in advance, as training data, a group of information in which input information obtained from a large number of clinical tests performed in the past is given known test results (normal or disease name). The control unit 10 learns various parameters constituting the third learning model 115, for example, using an error backpropagation method, so as to output disease information corresponding to the specimen data. As a result, when specimen data is input, the third learning model 115 is constructed, which has been trained to be able to appropriately output disease information.

[0114] 15 is a flowchart showing an example of a disease information acquisition processing procedure in the fourth embodiment. The following processing is executed by the control unit 10 in accordance with a program 1P stored in the storage unit 11 of the information processing device 1.

[0115] The control unit 10 of the information processing device 1 identifies a sample data group that is classified as normal based on the classification information output from the first learning model 113, and extracts the latest (last in chronological order) sample data from the identified sample data group (step S61).

[0116] The control unit 10 inputs the acquired specimen data as input data to the third learning model 115 (step S62). The control unit 10 acquires disease information indicating normality or abnormality of the specimen data output from the third learning model 115 (step S63).

[0117] The control unit 10 outputs the acquired disease information to the testing facility device 2 (step S64), and ends the series of processes. The control unit 10 may generate screen information that displays the acquired disease information together with classification information and the like.

[0118] According to this embodiment, in addition to abnormalities resulting from testing errors during clinical testing, abnormalities related to diseases associated with the physiological state of the subject can be detected, providing more information. By adding disease information to abnormal values ​​in the sample data, the validity of the clinical test results can be more appropriately determined.

[0119] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. [Explanation of symbols]

[0120] 1. Information processing device 2. Testing facility equipment 3 Display device 4 Input Devices 5. Inspection equipment 10,20 Control section 11,21 Storage section 12,22 Communications Department 23 Input / output section 111 Sample DB 112 Training DB 113 First Learning Model 114 Second Learning Model 115 Third Learning Model 1P Program 1A Recording medium

Claims

1. Obtaining a plurality of pieces of sample data including test values ​​corresponding to a plurality of test items in a clinical test; generating a first sample data group including abnormal sample data by replacing at least one test value included in the acquired plurality of sample data with an abnormal value; A learning model is generated based on training data including the generated first specimen data group, the second specimen data group not including the abnormal specimen data, and classification information regarding normality or abnormality of the specimen data group, which outputs the classification information when the specimen data group is input. A method for generating a learning model that allows a computer to execute processing.

2. A first sample data group is generated by replacing a part of the sample data of a first patient at different test dates and times with the sample data of a second patient. A method for generating a learning model according to claim 1 .

3. A first group of sample data is generated by modifying the at least one test value according to a predetermined rule. A method for generating a learning model according to claim 1 or 2.

4. The predetermined rule includes changing the dilution ratio. The method for generating a learning model according to claim 3 .

5. The classification information includes normal and a plurality of types of abnormality including abnormality caused by sample mix-up. A method for generating a learning model according to any one of claims 1 to 4.

6. Obtaining abnormal specimen data obtained in the clinical test, The learning model is trained based on the third sample data group including abnormal sample data obtained in the acquired clinical test and the training data including a label indicating an abnormality in the sample data group. A method for generating a learning model according to any one of claims 1 to 5.

7. The learning model is trained using a result of classification of the sample data group as normal or abnormal by an unsupervised model that has been trained by unsupervised learning to classify the sample data group as normal or abnormal. A method for generating a learning model according to any one of claims 1 to 6.

8. Extracting the specimen data having a high co-occurrence frequency from the plurality of specimen data; The specimen data group is generated from the extracted specimen data with high co-occurrence frequency. A method for generating a learning model according to any one of claims 1 to 7.

9. Obtaining a specimen data group consisting of a plurality of specimen data arranged in chronological order; inputting the acquired specimen data group into a learning model that has been trained to output classification information regarding normality or abnormality of the last specimen data in the time series included in the specimen data group when a specimen data group consisting of a plurality of specimen data arranged in chronological order is input, and outputting the classification information; The learning model has been trained using training data including a first specimen data group including abnormal specimen data generated by replacing at least one test value included in a plurality of specimen data including test values ​​corresponding to a plurality of test items in a clinical test with an abnormal value, a second specimen data group not including the abnormal specimen data, and classification information regarding normality or abnormality of the specimen data group. A program that causes a computer to execute a process.

10. A specimen data group is obtained that is made up of a plurality of specimen data items having a high co-occurrence frequency that are extracted from the plurality of specimen data items. The program according to claim 9.

11. The classification information includes abnormalities due to sample mix-up. The program according to claim 9 or 10.

12. The classification information includes an abnormality caused by a calculation error when calculating any one of test values ​​corresponding to a plurality of test items constituting the sample data. The program according to any one of claims 9 to 11.

13. The learning model outputs the classification information including a plurality of types of abnormalities, including an abnormality caused by a mix-up of samples and an abnormality caused by a calculation error when calculating any one of the test values ​​corresponding to a plurality of test items constituting the sample data. The program according to any one of claims 9 to 12.

14. Obtaining a verification result regarding normality or abnormality of the sample data by a delta verification method or an occurrence record zone verification method; The obtained verification result is output in association with the classification information. The program according to any one of claims 9 to 13.

15. Among the test values ​​corresponding to the test items constituting the specimen data, the test values ​​or the test items having a high degree of contribution to the classification information are output in an identifiable manner. The program according to any one of claims 9 to 14.

16. A graph showing time series changes in the latest sample data and past sample data in a clinical test is output in association with an abnormality range of the latest sample data estimated based on the past sample data. The program according to any one of claims 9 to 15.

17. Extracting the latest sample data determined to be normal by the learning model; When specimen data consisting of test values ​​corresponding to a plurality of test items is input, the latest extracted specimen data is input to a second learning model that outputs disease information related to abnormalities in the test values ​​due to a disease, and the disease information is output. The program according to any one of claims 9 to 16.

18. Obtaining a specimen data group consisting of a plurality of specimen data arranged in chronological order; inputting the acquired specimen data group into a learning model that has been trained to output classification information regarding normality or abnormality of the last specimen data in chronological order when a specimen data group consisting of a plurality of specimen data arranged in chronological order is input, and outputting the classification information; The learning model has been trained using training data including a first specimen data group including abnormal specimen data generated by replacing at least one test value included in a plurality of specimen data including test values ​​corresponding to a plurality of test items in a clinical test with an abnormal value, a second specimen data group not including the abnormal specimen data, and classification information regarding normality or abnormality of the specimen data group. An information processing method for causing a computer to execute processing.

19. an acquisition unit that acquires a sample data group consisting of a plurality of sample data arranged in chronological order; an output unit that inputs the acquired sample data group into a learning model that has been trained to output classification information regarding normality or abnormality of the last sample data in the time series when a sample data group consisting of a plurality of sample data arranged in chronological order is input, and outputs the classification information; Equipped with The learning model has been trained using training data including a first specimen data group including abnormal specimen data generated by replacing at least one test value included in a plurality of specimen data including test values ​​corresponding to a plurality of test items in a clinical test with an abnormal value, a second specimen data group not including the abnormal specimen data, and classification information regarding normality or abnormality of the specimen data group. Information processing device.

20. Obtaining a plurality of pieces of sample data including test values ​​corresponding to a plurality of test items in a clinical test; generating a first sample data group including abnormal sample data by replacing at least one test value included in the acquired plurality of sample data with an abnormal value; The generated first specimen data group and the label indicating an abnormality are associated with the generated second specimen data group not including the abnormal specimen data and the label indicating a normality as training data for a learning model, and are stored in a storage unit. A method for generating learning model data that causes a computer to execute processing.

Images