Information processing program, information processing method, and information processing device

Abstract

To provide an information processing program, an information processing method, and an information processing device that output information regarding the adoption, replacement, or modification of objective variables obtained from a learning model.SOLUTION: An information processing program makes a computer execute processing that inputs a value of newly acquired explanatory variable into a learning model that has been trained based on training data containing a predetermined explanatory variable and multiple objective variables, obtains values of multiple objective variables from the learning model, calculates the similarity between the obtained values of the multiple objective variables and the values of the multiple objective variables included in the training data, and outputs information regarding the replacement or modification of the objective variable values obtained from the learning model based on the calculated similarity.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus that output information regarding values of target variables acquired from a learning model.

Background Art

[0002] In supervised learning in a multi-output machine learning model, effective learning can be expected from a structure in which the mutual outputs affect learning. However, in learning, in a case where the quality of label data and the progress of learning differ for each output, etc., sufficient accuracy may not be achieved in learning of a machine learning model that conforms to an actual environment in which each output is used in combination. In such a case, etc., there is a scene where it is desired to output some information about the output value of the machine learning model. Regarding the accuracy of the learning model, Patent Document 1 discloses a technique for generating a prediction model aimed at improving the prediction accuracy of a target variable.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, consideration has not been given to outputting information regarding replacement or correction of target variables acquired from a learning model.

[0005] The present invention has been made in view of such a situation. Its object is to provide an information processing program, an information processing method, and an information processing apparatus that output information regarding replacement or correction of target variables acquired from a learning model.

Means for Solving the Problems

[0006] An information processing program according to an aspect of the present application performs learning on a learning model that has been learned based on training data including a predetermined explanatory variable and a plurality of objective variables, inputs the value of the newly acquired explanatory variable into the learning model, acquires the values of the plurality of objective variables from the learning model, calculates the similarity between the acquired values of the plurality of objective variables and the values of the plurality of objective variables included in the training data, and causes a computer to perform a process of outputting information regarding replacement or correction of the value of the objective variable acquired from the learning model based on the calculated similarity.

Effect of the Invention

[0007] In one aspect of the present application, it becomes possible to output information regarding the adoption, replacement, or correction of the objective variable acquired from the learning model.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Mode for Carrying Out the Invention

[0009] (Embodiment 1) The following embodiments will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing a configuration example of an information processing system. The information processing system 100 includes a server 1, a data source 2, and a user terminal 3. The server 1, the data source 2, and the user terminal 3 are connected to each other communicably via a network N. In FIG. 1, one data source 2 and one user terminal 3 are shown, but there may be two or more of each.

[0010] The main function of Server 1 is to receive explanatory variables, estimate the corresponding target variables using a learning model, and output the estimated target variables. Server 1 is composed of a server computer, a workstation, a PC (Personal Computer), etc. Also, Server 1 may be composed of a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. Furthermore, the function of Server 1 may be realized by cloud services.

[0011] Figure 2 is a block diagram showing an example of the hardware configuration of Server 1. Server 1 includes a control unit 11, a main memory unit 12, an auxiliary storage unit 13, a communication unit 14, and a reading unit 15. Each component is connected by a bus B.

[0012] The control unit 11 has one or more arithmetic processing units such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit). The control unit 11 reads and executes the control program 1P (program, program product) stored in the auxiliary storage unit 13 to perform various information processing, control processing, etc., and realizes various functional units.

[0013] The main memory unit 12 is an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), a flash memory, etc. The main memory unit 12 mainly stores temporarily the data necessary for the control unit 11 to execute arithmetic processing.

[0014] The auxiliary storage unit 13 is a hard disk or an SSD (Solid State Drive), etc., and stores the control program 1P and various DBs (Databases) necessary for the control unit 11 to execute processing. The auxiliary storage unit 13 stores the learning DB 131 and the learning model M. The auxiliary storage unit 13 may be an external storage device that is separate from the server 1 and externally connected. The various DBs, etc. stored in the auxiliary storage unit 13 may be stored in a database server or cloud storage different from the server 1.

[0015] The communication unit 14 communicates with the data source 2 and the user terminal 3 via the network N. The control unit 11 may use the communication unit 14 to download the control program 1P from another computer via the network N, etc., and store it in the auxiliary storage unit 13.

[0016] The reading unit 15 reads a portable storage medium 1a including a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 15 and store it in the auxiliary storage unit 13. Also, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0017] The data source 2 is a device or the like that generates data serving as explanatory variables or data serving as the basis for explanatory variables. It is assumed that the device is being used by the user. The data generated by the data source 2 directly becomes the value of the explanatory variable. Or, the data obtained by performing predetermined processing on the data generated by the data source 2 becomes the value of the explanatory variable. The explanatory variable is a combination of multiple types of variables.

[0018] The user terminal 3 is a terminal used by the user. The user provides the explanatory variables based on the data generated by the data source 2 to the server 1 and obtains the predicted value of the target variable for the explanatory variables from the server 1. The user utilizes the obtained predicted value of the target variable in the business.

[0019] FIG. 3 is a block diagram showing an example of the hardware configuration of the user terminal 3. The user terminal 3 is composed of a notebook computer, a panel computer, a tablet computer, a smartphone, etc. The user terminal 3 includes a control unit 31, a main memory unit 32, an auxiliary storage unit 33, a communication unit 34, an input unit 35, and a display unit 36. Each component is connected by a bus B.

[0020] The control unit 31 has one or more arithmetic processing devices such as a CPU, an MPU, and a GPU. The control unit 31 provides various functions by reading and executing the control program 3P (program, program product) stored in the auxiliary storage unit 33.

[0021] The main memory unit 32 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 32 mainly stores temporarily the data necessary for the control unit 31 to execute arithmetic processing.

[0022] The auxiliary storage unit 33 is a hard disk or an SSD, etc., and stores various data necessary for the control unit 31 to execute processing. The auxiliary storage unit 33 may be an external storage device that is separate from the user terminal 3 and externally connected. Various DBs, etc. stored in the auxiliary storage unit 33 may be stored in a database server or cloud storage.

[0023] The communication unit 34 communicates with the server 1 via the network N. Also, the control unit 31 may use the communication unit 34 to download the control program 3P from another computer via the network N or the like and store it in the auxiliary storage unit 33.

[0024] The input unit 35 is a keyboard or a mouse. The display unit 36 includes a liquid crystal display panel or an organic EL (electro Luminescence) display panel, etc. The display unit 36 displays messages output by the server 1, etc. Also, the input unit 35 and the display unit 36 may be integrated to form a touch panel display. Note that the user terminal 3 may display on an external display device.

[0025] Next, the database used in the information processing system 100 will be described. FIG. 4 is an explanatory diagram showing an example of the training DB. The training DB 13T stores the training data of the learning model M. The training DB 13T is temporarily stored in the main memory unit 12 or the auxiliary storage unit 13 when the learning model M is generated or when the learning model M is relearned. The training DB 13T includes an explanatory variable column and a target variable column. The explanatory variable column stores the values of the explanatory variables. The explanatory variables consist of m values (m is an integer of 2 or more). The target variable column stores the values of the target variables corresponding to the explanatory variables. The target variables consist of n values (n is an integer of 2 or more).

[0026] FIG. 5 is an explanatory diagram showing an example of the learning DB 131. The learning DB 131 includes a target variable column. The target variable column stores the target variables that make up the training data used for the generation or relearning of the learning model.

[0027] Next, the learning model M will be described. The learning model M is, for example, a CNN (Convolution Neural Network). The learning model M includes an input layer, an intermediate layer, and an output layer. The input layer has a plurality of neurons that receive the input of the explanatory variables and passes the input explanatory variables to the intermediate layer. The intermediate layer has a plurality of neurons, extracts feature amounts from the explanatory variables, and passes them to the output layer. The output layer outputs the target variables. The learning model M is not limited to a CNN, and may be a learned model constructed by other learning algorithms such as neural networks other than CNNs, Bayesian networks, and decision trees.

[0028] FIG. 6 is an explanatory diagram regarding a method for generating the learning model M. In the generation process of the learning model M, the control unit 11 of the server 1 inputs the explanatory variables included in the training data into the learning model M. The control unit 11 acquires the target variable output by the learning model M. The control unit 11 optimizes parameters including the weights between neurons, the coefficients of the activation functions used in each neuron, etc., so that the target variable included in the training data and the target variable output by the learning model M approach each other, for example, using the error backpropagation method. The control unit 11 repeats the above process for all the training data. When the control unit 11 finishes the process for all the training data, it stores the optimized parameters in the auxiliary storage unit 13 as the definition of the learning model M and ends the generation process. The control unit 11 stores the target variable included in the training data in the learning DB 131.

[0029] FIG. 7 is a flowchart showing an example of the procedure of the generation process. The generation process is a process for generating the learning model M. The control unit 11 of the server 1 acquires the training data (step S1). The control unit 11 stores the acquired training data in the training DB 13T or the like. The control unit 11 selects one record of the training data to be processed (step S2). The control unit 11 performs learning of the learning model M (step S3). The learning is as described above. The control unit 11 stores the target variable included in the training data in the learning DB 131 (step S4). The control unit 11 determines whether there is unprocessed training data (step S5). If the control unit 11 determines that there is unprocessed training data (YES in step S5), it returns the process to step S2 and performs the process using the unprocessed training data. If the control unit 11 determines that there is no unprocessed training data (NO in step S5), it stores the optimized parameters in the auxiliary storage unit 13 (step S6) and ends the process.

[0030] Next, the estimation process of the target variable using the generated learning model M will be described. FIG. 8 is a flowchart showing an example of the procedure of the prediction process. The prediction process is a process executed when operating the learning model M. The control unit 11 acquires the explanatory variables used for prediction (step S11). The control unit 11 inputs the explanatory variables into the learning model M and acquires the estimated value of the target variable output by the learning model M (step S12). The control unit 11 substitutes a value obtained by adding a value α to a predetermined threshold as the initial value of the variable dmin (step S13). α is a positive constant for adjusting the initial value. When the initial value of dmin is used as the threshold, if all the similarities calculated later are greater than the threshold, the value of dmin will not be rewritten. Therefore, the minimum value of the similarity becomes the threshold, which causes inconvenience. α is, for example, a value obtained by multiplying the threshold by 0.1 or 0.01. The control unit 11 acquires one target variable stored in the learning DB 131 (step S14). The control unit 11 calculates the similarity between the acquired target variable and the estimated value output by the learning model M (step S15). The similarity is the Euclidean distance, the Mahalanobis distance, or the Hamming distance. Alternatively, the similarity may be the cosine similarity, the Jaccard coefficient, or the Dice coefficient. The similarity to be calculated may be selected according to the characteristics of the target variable. The control unit 11 determines whether the calculated similarity is smaller than the value of the variable dmin (step S16). If the control unit 11 determines that the calculated similarity is greater than or equal to the value of the variable dmin (NO in step S16), the process proceeds to step S19. If the control unit 11 determines that the calculated similarity is smaller than the value of the variable dmin (YES in step S16), the value of the calculated similarity is substituted into the variable dmin (step S17). The control unit 11 stores the target variable in a temporary storage area provided in the auxiliary storage unit 13 or the like (step S18). If the target variable is already stored, the control unit 11 overwrites it. The control unit 11 determines whether there is an unprocessed target variable (step S19). If the control unit 11 determines that there is an unprocessed target variable (YES in step S19), the process returns to step S14 and the process using the unprocessed target variable is repeated. If the control unit 11 determines that there is no unprocessed target variable (NO in step S19), it determines whether the value of the variable dmin is greater than the threshold (step S20).When the control unit 11 determines that the value of the variable dmin is greater than the threshold value (YES in step S20), it generates a message regarding the handling of the target variable (estimated value) obtained from the learning model M (step S21). The control unit 11 outputs the generated message (step S22) and ends the process.

[0031] The message to be output is, for example, as follows. The message includes the estimated value of the target variable obtained from the learning model M and the value of the target variable in the learning DB 131 that is most similar to the estimated value, and includes a sentence suggesting to correct to the value stored in the learning DB 131 instead of the estimated value or to correct with reference to the stored value. Also, the message does not include the estimated value of the target variable obtained from the learning model M, but only includes the value of the target variable in the learning DB 131 that is most similar to the estimated value, and includes a sentence stating that it is replaced with the value stored in the learning DB 131 instead of the estimated value. Note that symbols or numbers may be used to indicate the content (information) instead of a message. For example, ○ indicates the adoption of the estimated value, △ indicates that it has been replaced with a similar one, and × indicates that correction is necessary.

[0032] When the control unit 11 determines that the value of the variable dmin is less than or equal to the threshold value (NO in step S20), it outputs the value obtained from the learning model M as the estimated value of the target variable (step S23) and ends the process. In this case, a sentence stating that the value obtained from the learning model M has been adopted may be displayed together.

[0033] The present embodiment has the following effects. When the similarity between the estimated value of the target variable obtained from the learning model M and the target variable stored in the learning DB 131 is equal to or greater than a threshold value, it is proposed to replace the value stored in the learning DB 131 instead of the estimated value obtained from the learning model M. Alternatively, a value stored in the learning DB 131 that is similar to the estimated value of the target variable obtained from the learning model M is adopted. When generating the learning model M, if it is learned with a sufficient number of training data, it is considered that, in the operation stage, the estimated value of the target variable output by the learning model M is within the distribution of a plurality of target variables included in the training data or is at a position close to the distribution. Therefore, if the accuracy of the estimated value of the target variable obtained from the learning model M is high, the minimum value of the similarity between the estimated value and the target variable stored in the learning DB 131 is equal to or less than an appropriately set threshold value. Thus, it is reasonable to consider that the estimated value has low accuracy when the minimum value of the similarity between the estimated value and the target variable stored in the learning DB 131 exceeds the threshold value. From the above, in the present embodiment, even when the learning model M outputs an estimated value with low accuracy, it is possible not to use the estimated value.

[0034] (Embodiment 2) In the above-described Embodiment 1, in the operation stage, the calculation of the similarity between the estimated value of the target variable output by the learning model M and the target variable used in the training data stored in the learning DB 131 is crucial. However, since the target variable is a variable that combines a plurality of values, the amount of calculation of the similarity increases. Therefore, in the present embodiment, instead of storing the values of the target variables included in the training data as they are in the learning DB 131, normalization, dimensionality reduction, etc. are performed and stored. When calculating the similarity, the same normalization, dimensionality reduction, etc. are performed on the estimated value output by the learning model M.

[0035] FIG. 9 is an explanatory diagram showing an overview of the generation process and the prediction process. FIG. 9A shows an overview of the generation process (preparation). In the training data shown in FIG. 9A, x train is an explanatory variable, and y trainindicates the objective variable. Learning the learning model M using training data is learning the relationship between the objective variable x and the explanatory variable y, that is, the function y = f(x). This learning is the same as in the first embodiment, so a description thereof will be omitted.

[0036] In this embodiment, the objective variable stored in the learning DB 131 is y train 9A, the preprocessing is expressed by the function transform(), and y'=transform(y). The control unit 11 calculates the y train By preprocessing, y' train (However, y' train= transform(y train )) and convert it to y' train is stored in the learning DB 131. The function transform() is a function that normalizes values and reduces dimensions.

[0037] FIG. 9B shows an overview of the prediction process (operation). unknown is the explanatory variable. The control unit 11 unknown is input to the learning model M, and the predicted value of the objective variable y pred The control unit 11 performs preprocessing to obtain y' pred (=transform(y pred The control unit 11 obtains y' pred and all y' stored in the learning DB 131. train Calculate the distance between the two points and find the minimum value d of that distance. pred y modified The control unit 11 determines whether the minimum distance d is greater than a predetermined threshold value. If the control unit 11 determines that d is greater than the predetermined threshold value, the control unit 11 sets y pred y modified If the control unit 11 determines that d is equal to or smaller than a predetermined threshold, it proposes a correction to replace y pred Output.

[0038] Next, the information processing performed in this embodiment will be described. FIG. 10 is a flowchart showing another example of the generation process. The generation process is a process of generating the learning model M. The control unit 11 of the server 1 acquires the training data (x train , y train ) (step S31). The control unit 11 stores the acquired training data in the training DB 13T or the like. The control unit 11 selects one record of the training data to be processed (step S32). The control unit 11 performs learning of the learning model M (step S33). The learning is as described above. The control unit 11 performs preprocessing on the target variable (y train ) included in the training data (step S34). The control unit 11 stores the preprocessed target variable (y' train ) in the learning DB 131 (step S35). The control unit 11 determines whether there is unprocessed training data (step S36). If the control unit 11 determines that there is unprocessed training data (YES in step S36), the process returns to step S32, and processing using the unprocessed training data is performed. If the control unit 11 determines that there is no unprocessed training data (NO in step S36), the control unit 11 stores the parameters of the optimized learning model M in the auxiliary storage unit 13 (step S37), and ends the process. Note that the control unit 11 also stores the parameters of the preprocessing function transform() in the learning DB 131 so that y' train through y train can be restored.

[0039] FIGS. 11 and 12 are flowcharts showing another example of the prediction process. The prediction process is a process executed when the learning model M is operated. The control unit 11 acquires the explanatory variable (x unknown ) to be used for prediction (step S51). The control unit 11 inputs the explanatory variable to the learning model M and acquires the estimated value (y pred ) of the target variable output by the learning model M (step S52). The control unit 11 performs preprocessing on y pred to obtain y' predto obtain (step S53). The control unit 11 substitutes, as the initial value of the variable dmin, a value obtained by adding the value α to a predetermined threshold value (step S54). The role and value of α are as described above. The control unit 11 obtains one of the preprocessed target variables (y’ train ) stored in the learning DB 131 (step S55). The control unit 11 calculates the similarity between the obtained y’ train and y’ pred (step S56). The similarity is the same as in the first embodiment. The control unit 11 determines whether the calculated similarity is less than the value of the variable dmin (step S57). If the control unit 11 determines that the calculated similarity is greater than or equal to the value of the variable dmin (NO in step S57), the process proceeds to step S60. If the control unit 11 determines that the calculated similarity is less than the value of the variable dmin (YES in step S57), the control unit 11 substitutes the value of the calculated similarity into the variable dmin (step S58). The control unit 11 stores the preprocessed target variable (y’ train ) in a temporary storage area provided in the auxiliary storage unit 13 or the like (step S59). If the target variable is already stored, the control unit 11 overwrites it. The control unit 11 determines whether there is an unprocessed target variable (step S60). If the control unit 11 determines that there is an unprocessed target variable (YES in step S60), the process returns to step S55, and the process using the unprocessed target variable is repeated. If the control unit 11 determines that there is no unprocessed target variable (NO in step S60), the process proceeds to FIG. 12, and it is determined whether the value of the variable dmin is greater than the threshold value (step S61). If the control unit 11 determines that the value of the variable dmin is greater than the threshold value (YES in step S61), the preprocessed target variable (y’ train ) stored in the temporary storage area is retrieved, and the process reverse to the preprocessing is performed to obtain y modified (= y train ) (step S62). The control unit 11 generates a message regarding the handling of the target variable (estimated value) obtained from the learning model M (step S63). The control unit 11 outputs the generated message (step S64) and ends the process. The message is the same as in the first embodiment.

[0040] When the control unit 11 determines that the value of the variable dmin is less than or equal to the threshold value (NO in step S61), the value (y pred ) obtained from the learning model M is output as the estimated value of the target variable (step S65), and the process ends.

[0041] (Dimensionality reduction) In the above, normalization and dimensionality reduction were shown as examples of preprocessing. Below, dimensionality reduction will be supplemented. In the above-described embodiment, during operation, the similarity between the target variable output by the learning model M and the target variable used for learning stored in the learning DB 131 is calculated. Therefore, the larger the dimension of the target variable, the greater the storage capacity of the learning DB 131 and the greater the computational amount of similarity calculation. Therefore, dimensionality reduction is performed as preprocessing of the target variable.

[0042] Dimensionality reduction is a technique for converting high-dimensional data into low-dimensional data while preserving its original properties as much as possible and improving interpretability. There are various dimensionality reduction techniques. However, as described above, since it is necessary to perform the same processing as the dimensionality reduction performed in the learning stage (generation process) also in the operation stage (prediction process), it is necessary to use a technique that can store the parameters indicating the dimensionality reduction process. As a dimensionality reduction technique, hierarchical clustering (HCA: Hierarchical Cluster Analysis) is used. Since HCA forms clusters of similar data from the original properties, it is possible to evaluate the target population in two dimensions. As dimensionality reduction techniques other than HCA, parametric UMAP (Uniform Manifold Approximation and Projection), parametric t-SNE (t-distributed Stochastic Neighbor Embedding), or principal component analysis (PCA: Principal Component Analysis) may be used.

[0043] The reduction of memory capacity by dimensionality reduction will be explained using a virtual example. Assume that the data before dimensionality reduction has 15 types (15 dimensions) of feature values, and each value is an integer. If the data size of each value is 4 bytes, the data volume will be 4 bytes × 15 = 60 bytes per record. After performing dimensionality reduction to 2 dimensions and expressing each value as a decimal, if the data size of each value is 8 bytes, the data volume will be 8 bytes × 2 = 16 bytes per record. In this case, the reduction is approximately 75%. Also, if the number of dimensions decreases, the amount of calculation for similarity will also decrease.

[0044] (Embodiment 3) This embodiment relates to an embodiment in which the above-described embodiment is applied to the estimation of a gate region in flow cytometry. Flow cytometry (FCM) is a technique capable of measuring a plurality of feature values for each single cell. In flow cytometry, a suspension in which cells are suspended is prepared, and the suspension is flowed into a measuring device so that the cells flow in a line. Light is applied to each flowing cell, and indices such as the size of the cell, the complexity inside the cell, and the constituent substances of the cell can be obtained from the scattering and fluorescence conditions of the light.

[0045] In the medical field, flow cytometry is used, for example, in cellular immunity tests. In cellular immunity tests, a testing institution analyzes a plurality of index values obtained by flow cytometry and returns the analysis result to the requesting institution as a test result. One of the analysis techniques is gating. Gating is a technique for selecting only a specific population from the obtained data for analysis. In a scatter plot, gating is to surround the population to be analyzed with an ellipse or a polygon (referred to as a "gate").

[0046] Next, the estimation of the gate region using the learning model will be described. FIG. 13 is an explanatory diagram showing an example of measurement values obtained from a flow cytometer. In the table shown in FIG. 13, each column stores the measurement values for each cell for the measurement items. Each row stores the measurement values for each measurement item for one cell. In gating, two scatter plots are created. For example, for the first scatter plot, one axis is SSC and the other axis is FL3. For the second scatter plot, one axis is SSC and the other axis is FSC. SSC represents the measurement value of side scattered light (SSC: Side Scattered Light). Side scattered light is the light detected at an angle of 90° with respect to the optical axis of the laser beam. FL represents fluorescence (Fluorescence). The flow cytometer is equipped with a plurality of fluorescence detectors, and FL3 represents the detection result of the third fluorescence detector. FSC represents the measurement value of forward scattered light (FSC: Forward Scattered Light). It represents the value of the scattered light detected in front of the optical axis of the laser beam.

[0047] The learning model M used in this embodiment will be described. FIG. 14 is a block diagram showing the configuration of the learning model M. The learning model M is a regression model. The learning model M is generated by performing deep learning to learn the feature amounts of appropriate gates for scatter plot images. When a plurality of scatter plot images (scatter plot group) are input to the learning model M, the learning model M outputs gate parameters. The plurality of scatter plot images are a plurality of scatter plot images in which at least one axis item is different. For example, it is two scatter plot images consisting of a scatter plot image with the horizontal axis being SSC and the vertical axis being FL3, and a scatter plot image with the horizontal axis being SSC and the vertical axis being FSC. Three or more scatter plot images may be input. The learning model M is configured using, for example, CNN (Convolution Neural Network), Transformer, or U-net, etc. The learning model M may also be configured with a model constructed using other learning algorithms such as a Bayesian network or a decision tree.

[0048] The learning model M has a plurality of feature extractors, a combiner, and a plurality of predictors. The plurality of feature extractors respectively learn the feature amounts of each scatter plot image. The combiner combines the feature amounts output by each feature extractor. The plurality of predictors predict and output each item (center X coordinate, center Y coordinate, major axis, minor axis, tilt angle) of the gate parameter based on the feature amounts combined by the combiner. Instead of the scatter plot image, a set of measurement values that are the basis of the scatter plot may be input to the learning model M.

[0049] Each feature extractor includes an input layer and an intermediate layer. The input layer has a plurality of neurons that receive the input of the pixel values of each pixel included in the scatter plot image, and passes the input pixel values to the intermediate layer. The intermediate layer has a plurality of neurons, extracts the feature amounts from within the scatter plot image, and passes them to the output layer. For example, when the feature extractor is a CNN, the intermediate layer has a configuration in which a convolutional layer that convolves the pixel values of each pixel input from the input layer and a pooling layer that maps the pixel values convolved by the convolutional layer are alternately connected. The intermediate layer finally extracts the image feature amounts while compressing the pixel information. Instead of providing a feature extractor for each image to input the scatter plot image, a configuration may be adopted in which a plurality of scatter plot images are input to one feature extractor.

[0050] Next, the generation and use of the learning model M will be described. FIG. 15 is an explanatory diagram showing an overview of the generation process and the prediction process. FIG. 15A shows an overview of the generation process (preparation). In the training data shown in FIG. 15A, the FCM data is the explanatory variable x train is. The FCM data is a scatter plot created from the measurement values of flow cytometry. The gate parameter is the target variable y train is. Learning the learning model M with the training data means learning the relationship between the target variable x and the explanatory variable y, that is, the function y = f(x). Since this learning is the same as that in the second embodiment, the description thereof will be omitted.

[0051] Similar to the second embodiment, the target variable stored in the learning DB 131 is y included in the training data trainRather than simply storing it as it is, preprocessing is performed and then it is stored. In FIG. 15, the preprocessing is represented by the function transform(), and y’ = transform(y). The control unit 11 preprocesses the y included in the training data train to y’ through preprocessing train (where y’ train= is transform(y train )) and stores y’ train in the learning DB 131. The function transform() is a function that reduces the dimension of the value.

[0052] FIG. 15B shows an overview of the prediction process (operation). The unknown data x shown in FIG. 15B unknown is the explanatory variable. The control unit 11 inputs x unknown into the learning model M and obtains the predicted value y of the target variable pred . The control unit 11 performs preprocessing to obtain y’ pred (= transform(y pred )) The control unit 11 calculates the distance between y’ pred and all the y’ train stored in the learning DB 131, and obtains the minimum value d of the distances. The y pred indicating the minimum is set as y modified . The control unit 11 determines whether the minimum value d of the distance is greater than a predetermined threshold. If the control unit 11 determines that d is greater than the predetermined threshold, it proposes a correction to replace y pred with y modified . If the control unit 11 determines that d is less than or equal to the predetermined threshold, it outputs y pred .

[0053] In this embodiment, the procedures of the generation process and the prediction process are the same as those in Embodiment 2, so the description is omitted. That is, the procedure of the generation process is the same as that in FIG. 10. The procedure of the prediction process is the same as those in FIGS. 11 and 12.

[0054] Next, the method of determining the threshold used in the operation stage will be described. The determination of whether to adopt the predicted value y pred output by the learning model M is made using a threshold. Therefore, if the threshold is inappropriate, ypred Even though the accuracy of pred is not good, a situation may occur where the control unit 11 of the server 1 does not make a correction proposal. Therefore, setting the threshold value is important.

[0055] Regarding threshold determination, the following policy is adopted. Determine the threshold so as to detect all inappropriate gate settings. That is, determine the threshold to be the smallest value within a reasonable range. When determining the threshold, prepare inappropriate gate setting data (hereinafter referred to as "irregular data") in advance. The irregular data may be created by a person, or the irregularities output by the learning model M may be collected.

[0056] Figure 16 is a flowchart showing an example of the procedure for threshold determination. The control unit 11 of the server 1 sets an initial value for the threshold (step S81). The initial value is set to a sufficiently large value that has no possibility of being taken as the similarity. The control unit 11 reads one piece of irregular data (step S82). The control unit 11 performs preprocessing on the irregular data (step S83). The control unit 11 reads one y' train stored in the learning DB 131 (step S84). As described above, y' train is the data after preprocessing for the target variable included in the training data used when generating the learning model M. The control unit 11 calculates the similarity (step S85). The control unit 11 determines whether the threshold is greater than the calculated similarity (step S86). When the control unit 11 determines that the threshold is less than or equal to the calculated similarity (NO in step S86), the process proceeds to step S88. When the control unit 11 determines that the threshold is greater than the calculated similarity (YES in step S86), the threshold is replaced with the calculated similarity (step S87). The control unit 11 determines whether there is any unprocessed y' train (step S88). When the control unit 11 determines that there is unprocessed y' train (YES in step S88), the process returns to step S84 and performs the process using the unprocessed y' train When the control unit 11 determines that there is no unprocessed y' trainIf it is determined that there is none (NO in step S88), it is determined whether there is unprocessed irregular data (step S89). When the control unit 11 determines that there is unprocessed irregular data (YES in step S89), the process returns to step S82 to perform processing on the unprocessed irregular data. When the control unit 11 determines that there is no unprocessed irregular data (NO in step S89), it stores the threshold value in the auxiliary storage unit 13 (step S90) and ends the process.

[0057] FIG. 17 is an explanatory diagram showing an example of irregular data. In FIG. 17, three gates are set in the scatter plot. Among the three gates, gate 161 and gate 163 are appropriate, but gate 162 is inappropriate. The gate parameters of gate 162 are irregular data.

[0058] FIG. 18 is a graph plotting the pre-processed irregular data and y' train As described above, by dimensionality reduction, the processed irregular data and y' train can be represented by a two-dimensional graph. The square dots shown in FIG. 18 represent y' train . The star 181 is the pre-processed irregular data. When the similarity is the Euclidean distance, the distance = similarity to the y' train closest to the star 181 is set as the threshold value.

[0059] (Embodiment 4) In this embodiment, it relates to a form in which an alternative positive rate is added as an input to the learning model M. Conventionally, when setting a gate, an examiner checks the positive rate (the ratio of positive cells) within the gate and modifies the gate. Therefore, even when the gate is set by the learning model M, the positive rate is likely to be useful. However, since the positive rate is an index that can be calculated after the gate is set, it cannot be obtained before the gate is set. Therefore, an index that can be calculated even when the gate has not been set and is considered effective for gate setting, similar to the positive rate, is introduced. This index is called the alternative positive rate. The alternative positive rate is calculated as follows. The cell populations present in the sample have different thresholds for classifying positive and negative, respectively. Therefore, the cell population is divided into smaller groups, and thresholds are set within each of the divided groups. In this embodiment, in the FSC, SSC, and FL3 distributions of SEQ1, a three-dimensional automatic clustering method, k-means, is applied to create n smaller groups. n is a natural number. Here, n = 10. From FL1 and FL2 of each smaller group of SEQ1, a threshold indicating negativity is mechanically calculated. For example, a value that includes 90% of the cells within the smaller group is set as the threshold. Next, the number of cells for each fraction in each smaller group is determined. FIG. 19 is an explanatory diagram showing the number of cells for each fraction. In FIG. 19, the number of cells for each fraction is shown for each of the 10 smaller groups. Next, the sum of the number of cells for each fraction is determined, and the obtained sum is divided by the total number of cells to obtain a ratio. The ratio calculated for each fraction for each SEQ is defined as the alternative positive rate. Let the number of cells in each fraction within the smaller group be UL, UR, LR, and LL. UL is the number of cells in the upper left, the number of cells where FL1 is negative and FL2 is positive. UR is the number of cells in the upper right, the number of cells where FL1 is positive and FL2 is positive. LR is the number of cells in the lower right, the number of cells where FL1 is positive and FL2 is negative. LL is the number of cells in the lower left, the number of cells where FL1 is negative and FL2 is negative. When each smaller group is k (k = 1, 2,..., 10) and the total number of cells is N, the alternative positive rate (APR: Alternative Positive Rate) can be calculated by the following mathematical formula (1).

[0060] [Number]

[0061] Next, the generation and use of the learning model M in the present embodiment will be described. FIG. 20 is an explanatory diagram showing an overview of the generation process and the prediction process. Since part of the content shown in FIG. 20 is the same as that in FIG. 15, the following description will mainly focus on the different parts.

[0062] FIG. 20A shows an overview of the generation process (preparation). The training data includes FCM data (explanatory variable x train ) and the gate parameter (objective variable y train ). In the present embodiment, when learning the learning model M, the alternative positive rate APR train calculated from the explanatory variable x train and the explanatory variable x train is used as the input. In FIG. 20A, the calculation of the alternative positive rate is represented as the function APR = calculate(x). Similar to Embodiment 3, the objective variable stored in the learning DB 131 is not stored as y train included in the training data as it is, but is stored after preprocessing. In the present embodiment, prior to performing the preprocessing, the alternative positive rate APR train is combined with the objective variable y train . Although the alternative positive rate APR train is represented as a matrix, when combining it with the objective variable y train , it is converted into a matrix of one row with the same number of elements and then combined. For example, if the alternative positive rate APR train is a 10-row 4-column matrix, it becomes a 1-row 40-column matrix. The combined result of the alternative positive rate APR train and the objective variable y train is represented as y merge_train . Substitute y = y merge_train into the preprocessing function y' = transform(y) to obtain y' merge_train . The control unit 11 stores y' merge_train in the learning DB 131.

[0063] FIG. 20B shows an overview of the prediction process (operation). The control unit 11 inputs the unknown data x unknown to the learning model M and obtains the predicted value y pred of the objective variable. The control unit 11 uses the unknown data xunknown to the alternative positive rate APR pred is calculated. The control unit 11 calculates the predicted value y pred and the alternative positive rate APR pred and combines them to obtain y merge_pred . The control unit 11 performs preprocessing on y merge_pred to obtain y' merge_pred . The control unit 11 calculates the distance between y' merge_pred and all the y' merge_train stored in the learning DB131, and obtains the minimum value d of the distance. The y merge_pred indicating the minimum, excluding the alternative positive rate APR pred , is set as y modified . The control unit 11 determines whether the minimum value d of the distance is greater than a predetermined threshold. When the control unit 11 determines that d is greater than the predetermined threshold, it proposes a correction to replace y pred with y modified . When the control unit 11 determines that d is less than or equal to the predetermined threshold, it outputs y pred .

[0064] FIG. 21 is a flowchart showing another example of the generation process. The control unit 11 of the server 1 acquires training data (x train , y train ) (step S101). The control unit 11 stores the acquired training data in the training DB13T or the like. The control unit 11 selects one record of the training data to be processed (step S102). The control unit 11 calculates the alternative positive rate APR train from the explanatory variable x train (step S103). The control unit 11 performs learning of the learning model M (step S104). The control unit 11 combines the target variable y train and the alternative positive rate APR train (step S105). The control unit 11 performs preprocessing on the combined y merge_train to obtain y' merge_train (step S106). The control unit 11 uses y' merge_trainStore it in the learning database 131 (step S107). The control unit 11 determines whether there is any unprocessed training data (step S108). If the control unit 11 determines that there is unprocessed training data (YES in step S108), the process returns to step S102, and the process using the unprocessed training data is performed. If the control unit 11 determines that there is no unprocessed training data (NO in step S108), the parameters of the optimized learning model M are stored in the auxiliary storage unit 13 (step S109), and the process ends.

[0065] Figures 22 and 23 are flowcharts showing other procedure examples of the prediction process. The prediction process is a process executed when the learning model M is operated. The control unit 11 acquires the explanatory variable (x unknown ) to be used for prediction (step S121). The control unit 11 calculates the alternative positive rate APR unknown from x pred (step S122). The control unit 11 inputs the explanatory variable into the learning model M and acquires the estimated value (y pred ) of the target variable output by the learning model M (step S123). The control unit 11 combines the predicted value y pred and the alternative positive rate APR pred (step S124). The control unit 11 performs preprocessing on the combined y merge_pred to obtain y’ merge_pred (step S125). The control unit 11 substitutes, as the initial value of the variable dmin, a value obtained by adding the value α to a predetermined threshold value (step S126). The role and value of α are as described above. The control unit 11 acquires one preprocessed target variable (y’ merge_train ) stored in the learning database 131 (step S127). The control unit 11 compares the acquired y’ merge_train with y’ merge_predCalculate the similarity with (step S128). The similarity is the same as in Embodiment 1. The control unit 11 determines whether the calculated similarity is less than the value of the variable dmin (step S129). If the control unit 11 determines that the calculated similarity is greater than or equal to the value of the variable dmin (NO in step S129), the process proceeds to step S132. If the control unit 11 determines that the calculated similarity is less than the value of the variable dmin (YES in step S129), it substitutes the calculated similarity value into the variable dmin (step S130). The control unit 11 stores the pre-processed target variable (y’ merge_train ) in a temporary storage area provided in the auxiliary storage unit 13 or the like (step S131). If the target variable is already stored, the control unit 11 overwrites it. The control unit 11 determines whether there is an unprocessed target variable (step S132). If the control unit 11 determines that there is an unprocessed target variable (YES in step S132), it returns the process to step S127 and repeats the process using the unprocessed target variable. If the control unit 11 determines that there is no unprocessed target variable (NO in step S132), it proceeds to FIG. 23 and determines whether the value of the variable dmin is greater than the threshold value (step S133). If the control unit 11 determines that the value of the variable dmin is greater than the threshold value (YES in step S133), it retrieves the pre-processed target variable (y’ merge_train ) stored in the temporary storage area. The control unit 11 performs the reverse process of the pre-processing to obtain y merge_train . The control unit 11 obtains y merge_train excluding the alternative positive rate APR train to obtain y modified (= y train ) (step S134). The control unit 11 generates a message regarding the handling of the target variable (estimated value) obtained from the learning model M (step S135). The control unit 11 outputs the generated message (step S136) and ends the process. The message is the same as in Embodiment 1.

[0066] If the control unit 11 determines that the value of the variable dmin is less than or equal to the threshold value (NO in step S133), the value (y pred) is output as an estimated value of the target variable (step S137), and the process ends.

[0067] In the present embodiment, an alternative positive rate is included in the variables input to the learning model M. Thereby, an improvement in the accuracy of the gate information output by the learning model M is expected.

[0068] (Example of processing result) Regarding Embodiments 3 and 4, an example of a processing result is shown. FIGS. 24 and 25 are explanatory diagrams showing an example of a processing result. The diagrams shown in FIGS. 24 and 25 are both scatter diagrams with an elliptical gate superimposed. FIG. 24A shows a gate (estimated gate area) drawn based on the gate information output by the learning model M. FIG. 24B shows a gate drawn based on the target variable of the training data most similar to the gate information output by the learning model M. FIG. 24C shows a gate set by a skilled examiner. As is clear from comparing the three figures, it can be seen that the gate based on the target variable of the training data similar to it is closer to the gate set by the skilled examiner than the gate output by the learning model M. In the case of FIG. 24, FIG. 24B is displayed together with a message replaced with something similar.

[0069] Similar to FIG. 24, in FIG. 25, FIG. 25A shows a gate drawn based on the gate information output by the learning model M. FIG. 25B shows a gate drawn based on the target variable of the training data most similar to the gate information output by the learning model M. FIG. 25C shows a gate set by a skilled examiner. In FIG. 25, the gate of FIG. 25B is closer to the correct answer shown in FIG. 25C than the gate of FIG. 25A. In such a case, FIGS. 25A and 25B are displayed, and a message suggesting further correction of the gate is displayed. When FIG. 25A is shown, it is assumed that an examiner with little experience may not know how to make corrections, but if FIG. 25B is displayed, even an examiner with little experience can make appropriate corrections.

[0070] Regarding the correction of the prediction result using the learning model M, although the gating in flow cytometry has been described, it is not limited to this. It is widely applicable to the use of extracting a specific region included in an input image, surrounding it with a figure, and displaying it.

[0071] For example, it is applicable to a system for determining the growth condition of rice in paddy fields using satellite images. In a wide range of paddy fields, due to differences in environmental conditions such as transplanting time and soil, there are differences in the growth condition of rice. Rice is green around August when the ears come out and turns golden at the harvest time. Using the satellite image as an explanatory variable and the coordinates of a plurality of points indicating a figure (surrounding the area of paddy fields near the harvest time) as the objective variable, the learning model M is learned. During operation, when the satellite image is input to the learning model M, it is possible to obtain an output image in which the area of paddy fields near the harvest time is surrounded by a figure.

[0072] It is also applicable to multivariate analysis in marketing. When segmenting customers based on customer attributes, behavior history, etc., each customer is represented as a point in a multi-dimensional space. The set of points is the explanatory variable. A customer segment is a set of a plurality of points in close positions. The set is represented as a figure surrounding the customers that are elements. The coordinates of a plurality of points indicating the figure become the objective variable. This structure is similar to the gating in flow cytometry. During training, the learning model M is made to learn the segmentation of existing customers. During operation, the segmentation of new customers is performed. When implementing policies such as campaigns only for customers in a specific segment, it is possible to select appropriate customers by the method of correcting the prediction result of the learning model M described above.

[0073] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed this time should be considered as illustrative in all aspects and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims be included. Furthermore, in addition, the claims use a format (multi-claim format) in which claims that cite two or more other claims are described, but it is not limited to this. A format in which multi-claims (multi-multi-claims) that cite at least one multi-claim may be described.

Description of Reference Numerals

[0074] 100: Information processing system 1: Server 11: Control unit 12: Main memory unit 13: Auxiliary storage unit 131: Learning DB 13T: Training DB M: Learning model 14: Communication unit 15: Reading unit 1P: Control program 1a: Portable storage medium 1b: Semiconductor memory 2: Data source 3: User terminal 3P: Control program 31: Control unit 32: Main memory unit 33: Auxiliary storage unit 34: Communication unit 35: Input unit 36: Display unit B: Bus N: Network

Claims

1. Based on training data including predetermined explanatory variables and a plurality of objective variables, input the values of the newly acquired explanatory variables into the trained learning model, and acquire the values of the plurality of objective variables from the learning model. Calculate the similarity between the acquired values of the plurality of objective variables and the values of the plurality of objective variables included in the training data. Based on the calculated similarity, output information regarding replacement or correction of the values of the objective variables acquired from the learning model. An information processing program that causes a computer to perform the processing.

2. The similarity is determined based on the distance calculated from the values of the plurality of objective variables acquired from the learning model and the values of the plurality of objective variables included in the training data. The information processing program according to Claim 1.

3. The similarity is determined using the data obtained by reducing the dimension of the plurality of objective variables included in the training data and the data obtained by reducing the dimension of the plurality of objective variables acquired from the learning model. The information processing program according to Claim 1 or Claim 2.

4. Calculate the similarity for the values of the plurality of objective variables included in each of the plurality of training data and the values of the plurality of objective variables output by the learning model. When the minimum value of the calculated plurality of similarities is equal to or greater than a predetermined threshold, output information regarding correction of the objective variables acquired from the learning model. The information processing program according to Claim 1 or Claim 2.

5. The explanatory variables are a group of scatter plots including a plurality of scatter plots obtained from measurements of flow cytometry with different measurement items, and the plurality of objective variables are values for specifying gate regions corresponding to the group of scatter plots. The information processing program according to Claim 1 or Claim 2.

6. The similarity is calculated using the data obtained by combining the gate region included in the training data and the alternative positive rate corresponding to the gate region, and the data obtained by combining the estimated gate region based on the output of the learning model and the alternative positive rate corresponding to the estimated gate region. The information processing program according to Claim 5.

7. The similarity is calculated using the data obtained by reducing the dimension of the combined gate region and the alternative positive rate corresponding to the gate region, and the data obtained by reducing the dimension of the data obtained by combining the estimated gate region based on the output of the learning model and the alternative positive rate corresponding to the estimated gate region. The information processing program according to Claim 5.

8. Output the gate region with the minimum similarity and the estimated gate region based on the output of the learning model in a comparable manner. The information processing program according to claim 5.

9. Input the value of a newly acquired explanatory variable into a learning model trained based on training data including a predetermined explanatory variable and a plurality of objective variables, and obtain the values of the plurality of objective variables from the learning model. Calculate the similarity between the obtained values of the plurality of objective variables and the values of the plurality of objective variables included in the training data. Output information regarding replacement or correction of the objective variable obtained from the learning model based on the calculated similarity. An information processing method in which a computer performs the processing.

10. An information processing apparatus including a control unit, wherein the control unit inputs the value of a newly acquired explanatory variable into a learning model trained based on training data including a predetermined explanatory variable and a plurality of objective variables, and obtains the values of the plurality of objective variables from the learning model. Calculates the similarity between the obtained values of the plurality of objective variables and the values of the plurality of objective variables included in the training data. Outputs information regarding replacement or correction of the objective variable obtained from the learning model based on the calculated similarity. An information processing apparatus that performs the processing.

Images