Visualization device, visualization method, and program

Abstract

Visualize the reliability of prediction results based on the target data. [Solution] The visualization device comprises: an acquisition unit that acquires target data for predicting the target variable using a trained model; an anomaly calculation unit that calculates the anomaly of the target data based on the training data used to train the trained model; a determination unit that determines whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the training data; and a visualization unit that displays the distribution of anomalies of the target data belonging to the interpolation region and the distribution of anomalies of the target data belonging to the extrapolation region in correspondence.

Description

Technical Field

[0001] The present disclosure relates to a visualization device, a visualization method, and a program.

Background Art

[0002] Techniques for evaluating predicted values output by a learned model generated by machine learning are known. For example, in Patent Document 1, an attribute value is calculated between a plurality of data included in a learning dataset used for generating a learned model, a plurality of intervals for classifying the attribute value are determined from the frequency distribution of the attribute value, and it is determined into which of the plurality of intervals the attribute value calculated between the data to be predicted and the plurality of data is classified, thereby evaluating the appropriateness of the data to be predicted with respect to conflicting indicators. A prediction device is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, there is room for easily visualizing the reliability of prediction results. For example, since it is known that characteristic prediction in an extrapolation region is difficult, if it is possible to display whether the data to be predicted belongs to an interpolation region or an extrapolation region, it is easier to understand the reliability of the prediction result.

[0005] One aspect of the present disclosure aims to visualize the reliability of a prediction result based on target data.

Means for Solving the Problems

[0006] The present disclosure includes the following configuration.

[0007] <1> An acquisition unit that acquires target data for predicting the target variable using a pre-trained model, An anomaly calculation unit that calculates the anomaly of the target data based on the training data used to train the aforementioned trained model, A determination unit that determines whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the training data, A visualization unit that displays the distribution of abnormality of the target data belonging to the interpolation region and the distribution of abnormality of the target data belonging to the extrapolation region in correspondence. A visualization device equipped with the following features.

[0008] <2> The determination unit determines that the target data located inside the convex hull of the learning data belongs to the interpolation region, and determines that the target data located outside the convex hull belongs to the extrapolation region. the above <1> The visualization device described above.

[0009] <3> The anomaly score calculation unit calculates the minimum distance between the target data and the training data as the anomaly score. the above <1> or <2> The visualization device described above.

[0010] <4> The system further includes a reference value calculation unit that calculates a reference value based on the distribution of anomalies in the training data, The visualization unit divides and displays the distribution of the degree of abnormality of the target data based on the reference value. the above <1> from <3> A visualization device as described in any of the following.

[0011] <5> The reference value calculation unit calculates the reference value that divides the degree of abnormality of the learning data into two parts, The visualization unit displays the distribution of the anomaly of the target data in four quadrants, including a strong interpolation region, a weak interpolation region, a strong extrapolation region, and a weak extrapolation region. the above <4> The visualization device described above.

[0012] <6> It further includes a data splitting unit that splits the learning data into a plurality of partial data including combinations of different explanatory variables. The determination unit determines whether the target data belongs to the interpolation region or the extrapolation region for each of the plurality of partial data. The visualization unit displays, in association with each other, the distribution of the abnormality degrees of the target data belonging to the interpolation region and the distribution of the abnormality degrees of the target data belonging to the extrapolation region for each of the plurality of partial data. The visualization device according to any one of <1> to <5> above.

[0013] <7> A model generation unit that generates a plurality of learned models based on each of the plurality of partial data, A model selection unit that selects, from the plurality of learned models, the learned model based on the partial data to which the target data belongs to the interpolation region, A prediction unit that predicts the target variable by inputting the target data into the selected learned model, and further includes The visualization device according to <6> above.

[0014] <8> A computer A procedure for acquiring target data for predicting a target variable using a learned model, A procedure for calculating the abnormality degree of the target data based on the learning data used for learning the learned model, A procedure for determining whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the learning data, A procedure for displaying, in association with each other, the distribution of the abnormality degrees of the target data belonging to the interpolation region and the distribution of the abnormality degrees of the target data belonging to the extrapolation region, A visualization method for executing.

[0015] <9> To a computer A procedure for obtaining target data for predicting a target variable using a learned model, and a procedure for calculating the degree of abnormality of the target data based on the training data used for training the learned model; a procedure for determining whether the target data belongs to an interpolation region or an extrapolation region based on the distribution of the training data; a procedure for associating and displaying the distribution of the degree of abnormality of the target data belonging to the interpolation region and the distribution of the degree of abnormality of the target data belonging to the extrapolation region; A program for executing the above.

Advantages of the Invention

[0016] According to one aspect of the present disclosure, the reliability of a prediction result based on target data can be visualized.

Brief Description of the Drawings

[0017] [Figure 1] It is a block diagram showing an example of the overall configuration of a visualization system. [Figure 2] It is a block diagram showing an example of a computer. [Figure 3] It is a block diagram showing an example of the functional configuration of the visualization system according to the first embodiment. [Figure 4] It is a flowchart showing an example of visualization processing. [Figure 5] It is a flowchart showing an example of abnormality degree calculation processing of training data. [Figure 6] It is a diagram for explaining an example of abnormality degree calculation processing of training data. [Figure 7] It is a flowchart showing an example of reference value calculation processing. [Figure 8] It is a diagram for explaining an example of reference value calculation processing. [Figure 9] It is a flowchart showing an example of abnormality degree calculation processing of target data. [Figure 10] It is a diagram for explaining abnormality degree calculation processing of target data. [Figure 11]This is a diagram illustrating an example of region partitioning. [Figure 12] This figure shows an example of the visualization results. [Figure 13] This block diagram shows an example of the functional configuration of the visualization system according to the second embodiment. [Figure 14] This is a flowchart showing an example of a prediction process. [Modes for carrying out the invention]

[0018] Hereinafter, embodiments of this disclosure will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0019] [First Embodiment] The first embodiment of this disclosure is an example of an information processing system for visualizing the reliability of prediction results. Hereinafter, the information processing system according to this embodiment will be referred to as the "visualization system". In this embodiment, the visualization system may visualize the reliability of the prediction results output by a trained model generated by machine learning.

[0020] In recent years, in the development of new materials, attempts have been made to predict material properties using, for example, pre-trained predictive models generated by machine learning. This is because using such pre-trained predictive models can reduce the processes involved in material production and verification experiments of material properties, and is expected to improve the efficiency of new material development.

[0021] On the other hand, when predicting material properties using a pre-trained model generated by machine learning, it is known that the prediction accuracy decreases if the data to be predicted deviates significantly from the training data used in the machine learning process. In response to this, for example, Patent Document 1 proposes a method for extracting only highly reliable prediction results by defining the applicable distance of the pre-trained model (the distance of the data to be predicted that can achieve the desired prediction accuracy).

[0022] However, the prediction accuracy of a trained model does not depend solely on data distance. For example, the prediction accuracy of a trained model also depends on whether the data to be predicted exists in the interpolation region or the extrapolation region. The interpolation region is the data space in which the training data exists, while the extrapolation region is the data space in which the training data does not exist. It is known that predicting characteristics in the extrapolation region is difficult.

[0023] This embodiment aims to visualize the reliability of prediction results based on target data. To this end, this embodiment determines whether the target data for predicting the target variable using a trained model belongs to the interpolation region or the extrapolation region, and displays the distribution of anomalies of the target data belonging to the interpolation region and the distribution of anomalies of the target data belonging to the extrapolation region in correspondence.

[0024] In one aspect, according to this embodiment, the distribution of anomalies in the target data is displayed in correspondence between the interpolated and extrapolated regions, making it easy to understand the degree of prediction accuracy of the prediction results based on the target data. Therefore, according to this embodiment, the reliability of the prediction results based on the target data can be visualized.

[0025] <Overall Structure> The overall configuration of the visualization system according to this embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing an example of the overall configuration of the visualization system.

[0026] As shown in Figure 1, the visualization system 1000 includes a visualization device 10 and a terminal device 50. The visualization device 10 and the terminal device 50 are connected via a communication network N to enable data communication. The communication network N may be, for example, a LAN (Local Area Network), a VPN (Virtual Private Network), or the Internet.

[0027] The visualization device 10 is an example of an information processing device that visualizes the reliability of prediction results. The visualization device 10 may be a computer such as a personal computer, workstation, or server.

[0028] The visualization device 10 acquires the training data used to train the trained model. The visualization device 10 also acquires target data for which a predetermined target variable is predicted using the trained model. Based on the training data, the visualization device 10 visualizes the reliability of the prediction results based on the target data. The visualization device 10 transmits the visualization results of the prediction result reliability to the terminal device 50.

[0029] Terminal device 50 is an example of an information processing device operated by a user of the visualization system 1000. Terminal device 50 may be a computer such as a personal computer, smartphone, or tablet terminal.

[0030] The terminal device 50 receives the visualization results of the reliability of the prediction results from the visualization device 10. The terminal device 50 presents the visualization results of the reliability of the prediction results to the user.

[0031] The overall configuration of the visualization system 1000 shown in Figure 1 is just one example, and various system configurations are possible depending on the application and purpose. For example, one or more visualization devices 10 and terminal devices 50 may be included in the visualization system 1000. For example, the visualization device 10 may be implemented by multiple computers or as a cloud computing service. For example, the visualization system 1000 may be implemented by a standalone computer. The classification of devices such as the visualization device 10 and terminal device 50 shown in Figure 1 is just one example.

[0032] <Hardware Configuration> The hardware configuration of the visualization system 1000 will be explained with reference to Figure 2. The visualization device 10 and terminal device 50 are implemented, for example, by a computer. Figure 2 is a block diagram showing an example of the computer's hardware configuration.

[0033] As shown in Figure 2, the computer 500 includes a CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, HDD (Hard Disk Drive) 504, input device 505, display device 506, communication interface 507, and external interface 508. The CPU 501, ROM 502, and RAM 503 form what is known as a computer. Each piece of hardware in the computer 500 is interconnected via a bus line 509. The input device 505 and display device 506 may also be used by connecting them to the external interface 508.

[0034] The CPU 501 is a processing unit that reads programs and data from a storage device such as the ROM 502 or HDD 504 onto the RAM 503 and executes processing, thereby realizing the overall control and functions of the computer 500. The computer 500 may have a GPU (Graphics Processing Unit) in addition to or instead of the CPU 501.

[0035] ROM502 is an example of non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. ROM502 functions as the main memory, storing various programs and data necessary for the CPU501 to execute the programs installed on HDD504. Specifically, ROM502 stores boot programs such as BIOS (Basic Input Output System) and EFI (Extensible Firmware Interface) that are executed when the computer 500 starts up, as well as OS (Operating System) settings, network settings, and other data.

[0036] RAM503 is an example of volatile semiconductor memory (storage device) whose programs and data are erased when the power is turned off. RAM503 includes, for example, DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory). RAM503 provides a working area that is expanded when various programs installed on HDD504 are executed by CPU501.

[0037] HDD504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in HDD504 include the operating system (OS), which is the basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that computer 500 may use a storage device that uses flash memory as its storage medium (e.g., SSD: Solid State Drive) instead of HDD504.

[0038] The input device 505 includes a touch panel used by the user to input various signals, operation keys and buttons, a keyboard and mouse, and a microphone for inputting sound data such as voice.

[0039] The display device 506 consists of a display such as a liquid crystal or organic EL (Electro-Luminescence) that displays a screen, and a speaker that outputs sound data such as audio.

[0040] Communication I / F 507 is an interface that connects to a communication network and allows computer 500 to perform data communication.

[0041] External I / F 508 is an interface for external devices. Examples of external devices include the drive device 510.

[0042] The drive device 510 is a device for setting the recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 511 may also include semiconductor memory that records information electrically, such as ROMs and flash memory. This allows the computer 500 to read and / or write to the recording medium 511 via the external I / F 508.

[0043] The various programs to be installed on the HDD 504 are installed, for example, when the distributed recording medium 511 is set in a drive device 510 connected to an external I / F 508, and the various programs recorded on the recording medium 511 are read by the drive device 510. Alternatively, the various programs to be installed on the HDD 504 may be installed by downloading them via the communication I / F 507 from a communication network or another network different from the communication network.

[0044] <Functional Configuration> The functional configuration of the visualization system 1000 will be explained with reference to Figure 3. Figure 3 is a block diagram showing an example of the functional configuration of the visualization system according to the first embodiment.

[0045] As shown in Figure 3, the visualization device 10 includes a data storage unit 101, a model storage unit 102, a model generation unit 110, a prediction unit 120, an acquisition unit 130, anomaly calculation unit 140, a reference value calculation unit 150, a region division unit 160, a determination unit 170, and a visualization unit 180. The visualization device 10 functions as the data storage unit 101, model storage unit 102, model generation unit 110, prediction unit 120, acquisition unit 130, anomaly calculation unit 140, a reference value calculation unit 150, a region division unit 160, a determination unit 170, and a visualization unit 180 when a pre-installed visualization program is executed.

[0046] For example, the data storage unit 101 and the model storage unit 102 are implemented by the HDD 504 shown in Figure 2. For example, the model generation unit 110, prediction unit 120, acquisition unit 130, anomaly calculation unit 140, reference value calculation unit 150, area division unit 160, determination unit 170, and visualization unit 180 are implemented by a process in which a program loaded from the HDD 504 shown in Figure 2 onto the RAM 503 is executed by the CPU 501.

[0047] The data storage unit 101 stores training data. The training data is a dataset used to train the trained model. The training data may include the correct values ​​of one or more explanatory variables and one or more target variables. The explanatory variables and target variables may include any variables depending on the task to be performed by the trained model. For example, the explanatory variables may include descriptors related to matter. The target variables may include physical property values ​​of matter.

[0048] The data storage unit 101 stores the target data. The target data is electronic data used to predict the target variable using a trained model. The target data may include one or more explanatory variables. The explanatory variables included in the target data may include one or more explanatory variables included in the training data. The explanatory variables included in the target data may include explanatory variables not included in the training data, and may not include one or more explanatory variables included in the training data.

[0049] The model memory unit 102 stores the trained model. The trained model may be a machine learning model that takes explanatory variables as input and outputs a predicted value of the target variable. The machine learning model can be any type of machine learning model. Examples of machine learning models include linear regression, Gaussian process regression, neural networks, random forests, gradient boosting decision trees, etc. The trained model may also be generated by the model generation unit 110.

[0050] The model generation unit 110 generates a trained model. The model generation unit 110 may generate a trained model based on training data. The model generation unit 110 may generate a trained model based on training data read from the data storage unit 101. The model generation unit 110 may store the trained model in the model storage unit 102.

[0051] The model generation unit 110 may generate a trained model by learning the relationship between the explanatory variables and the target variable included in the training data. For example, the model generation unit 110 may learn the relationship between the explanatory variables and the target variable based on backpropagation. For example, the model generation unit 110 may update the parameters of the machine learning model so as to minimize the loss calculated based on the output from the machine learning model and the ground truth value of the target variable when the explanatory variables are input.

[0052] The prediction unit 120 predicts the target variable. The prediction unit 120 may predict the target variable based on a trained model. The prediction unit 120 may predict the target variable based on a trained model read from the model storage unit 102. The prediction unit 120 may predict the target variable based on target data read from the data storage unit 101. The prediction unit 120 may predict the target variable by inputting explanatory variables included in the target data into the trained model.

[0053] The acquisition unit 130 acquires training data. The acquisition unit 130 may read training data stored in the data storage unit 101. The acquisition unit 130 may receive training data from the terminal device 50. The acquisition unit 130 may acquire training data input to the visualization device 10 via the input device 505.

[0054] The acquisition unit 130 acquires the target data. The acquisition unit 130 may read the target data stored in the data storage unit 101. The acquisition unit 130 may receive the target data from the terminal device 50. The acquisition unit 130 may acquire the target data input to the visualization device 10 via the input device 505.

[0055] The anomaly score calculation unit 140 calculates the anomaly score of the training data. The anomaly score is an index that indicates the degree to which one data point deviates from other data points. In other words, the anomaly score takes a large value when one data point deviates significantly from other data points, and a small value when one data point does not deviate significantly from other data points. The anomaly score may be, for example, the distance between data points, the data density, or the similarity. The distance between data points may be, for example, the Euclidean distance. In this embodiment, an example in which the anomaly score is the distance between data points will be described.

[0056] The anomaly score calculation unit 140 may calculate the distance of each training data to each other training data. The anomaly score calculation unit 140 may also calculate the minimum distance to other training data as the anomaly score of the training data.

[0057] The anomaly score calculation unit 140 may perform preprocessing on the training data. The anomaly score calculation unit 140 may normalize the training data. For example, the anomaly score calculation unit 140 may normalize the training data so that it is between 0 and 1. The anomaly score calculation unit 140 may subtract the minimum value of the training data from each training data and divide by the absolute value of the difference between the minimum value and the maximum value of the training data.

[0058] The anomaly calculation unit 140 may remove variables with high correlations from the training data. For example, the anomaly calculation unit 140 may remove variables with a correlation coefficient of 1 or -1 from the training data.

[0059] The anomaly score calculation unit 140 may remove data with small variance from the training data. For example, the anomaly score calculation unit 140 may remove data with a variance of 0 from the training data.

[0060] The anomaly calculation unit 140 calculates the anomaly score of the target data. The anomaly calculation unit 140 may also calculate the distance of each target data to each training data. The anomaly calculation unit 140 may also calculate the minimum distance to the training data as the anomaly score of the target data.

[0061] The anomaly calculation unit 140 may perform preprocessing on the target data. The anomaly calculation unit 140 may normalize the target data. For example, the anomaly calculation unit 140 may normalize the target data using the same rules as the normalization performed on the training data. The anomaly calculation unit 140 may delete variables included in the target data. For example, the anomaly calculation unit 140 may delete variables that were deleted from the training data.

[0062] The reference value calculation unit 150 calculates a reference value. The reference value is a reference value used to divide the anomaly score of the training data into multiple intervals. The reference value calculation unit 150 may calculate one reference value that divides the anomaly score of the training data into two. The reference value calculation unit 150 may calculate three reference values ​​that divide the anomaly score of the training data into four.

[0063] The reference value calculation unit 150 may perform preprocessing on the anomaly score of the training data. The reference value calculation unit 150 may normalize the anomaly score of the training data. The reference value calculation unit 150 may normalize the anomaly score by the furthest distance. The furthest distance is the maximum distance in the data space, and is the distance between the origin of the data space and the furthest point. The furthest distance may also be the square root of the number of dimensions in the data space of the training data. For example, the furthest distance may be √N, where the number of dimensions of the training data is N. The reference value calculation unit 150 may divide the anomaly score by the furthest distance.

[0064] The reference value calculation unit 150 may calculate the reference value based on the distribution of anomalies in the training data. The reference value calculation unit 150 may also calculate the reference value based on statistical values ​​of the distribution of anomalies. The statistical values ​​of the distribution may include, as an example, the mean, median, mode, quartiles, percentiles, variance, standard deviation, etc.

[0065] For example, the reference value calculation unit 150 may calculate three reference values ​​that divide the degree of abnormality into four parts. For example, the first reference value may be the 75th percentile (third quartile), the second reference value may be the 75th percentile + 1.5 × interquartile range, and the third reference value may be the 75th percentile + 3 × interquartile range. The interquartile range is the absolute value of the difference between the 25th percentile and the 75th percentile.

[0066] For example, the reference value calculation unit 150 may calculate a single reference value that divides the degree of abnormality into two. As an example, the single reference value may be the 75th percentile.

[0067] The domain partitioning unit 160 partitions the data space. The domain partitioning unit 160 may partition the data space of the training data based on the training data. The domain partitioning unit 160 may partition the data space into an interpolation region and an extrapolation region. For example, the domain partitioning unit 160 may partition the data space of the training data into an interpolation region and an extrapolation region by calculating the convex hull of the training data. For example, the domain partitioning unit 160 may partition the data space of the training data into an interpolation region and an extrapolation region by Delaunay partitioning using the training data.

[0068] The determination unit 170 determines whether the target data belongs to the interpolation region or the extrapolation region. The determination unit 170 may also determine whether the target data belongs to the interpolation region or the extrapolation region based on region information indicating the interpolation region and the extrapolation region divided by the region division unit 160. The region information may also be information indicating the boundary between the interpolation region and the extrapolation region. As an example, the region information may be information indicating the convex hull of the training data. As another example, the region information may be information indicating the interface between the interpolation region and the extrapolation region generated by Delaunay tessellation.

[0069] The determination unit 170 may determine whether the target data resides in the interpolation region or the extrapolation region. For example, if the target data resides inside the convex hull calculated by the region division unit 160, the determination unit 170 may determine that the target data belongs to the interpolation region. On the other hand, if the target data resides outside the convex hull calculated by the region division unit 160, the determination unit 170 may determine that the target data belongs to the extrapolation region.

[0070] Hereafter, data belonging to the interpolation region will be referred to as "interpolated data," and data belonging to the extrapolation region will be referred to as "extrapolated data."

[0071] The visualization unit 180 displays the distribution of anomalies in the target data. The visualization unit 180 may also display the frequency distribution of anomalies in the target data. The visualization unit 180 may also display the distribution of anomalies in the target data using a graph with anomaly degree on the horizontal axis and frequency on the vertical axis.

[0072] The visualization unit 180 may display the distribution of anomalies of target data belonging to the interpolation region (i.e., interpolated data) and the distribution of anomalies of target data belonging to the extrapolation region (i.e., extrapolated data) in correspondence. The visualization unit 180 may display the distribution of interpolated data and the distribution of extrapolated data in a comparable manner. The visualization unit 180 may display the distribution of interpolated data and the distribution of extrapolated data simultaneously. The visualization unit 180 may display the distribution of interpolated data and the distribution of extrapolated data side by side in a predetermined direction.

[0073] The visualization unit 180 may display the distribution of anomalies in the training data along with the distribution of anomalies in the target data. For example, the visualization unit 180 may display the distribution of anomalies in the target data and the distribution of anomalies in the training data superimposed on each other.

[0074] The visualization unit 180 may divide the distribution of anomalies in the target data into multiple intervals and display it. The visualization unit 180 may divide the distribution of anomalies in the target data into multiple intervals based on reference values ​​calculated by the reference value calculation unit 150. The visualization unit 180 may divide the distribution of anomalies in the target data into two based on reference values ​​that divide the anomaly of the training data into two. For example, the visualization unit 180 may divide the distribution of anomalies in the interpolated data and the distribution of anomalies in the extrapolated data into two, and display them in a four-quadrant including a strong interpolation region, a weak interpolation region, a strong extrapolation region, and a weak extrapolation region.

[0075] The visualization unit 180 may assign ranks to the intervals divided based on the reference value. The ranks may also be the order obtained by sorting each of the multiple intervals in ascending or descending order based on the degree of abnormality. For example, the visualization unit 180 may sort each of the multiple intervals in ascending order of the degree of abnormality and assign a rank indicating the order from the beginning.

[0076] The visualization unit 180 may identify the rank of the target data. The visualization unit 180 may identify the rank of the target data based on the reference value calculated by the reference value calculation unit 150. The visualization unit 180 may identify the interval to which the target data belongs based on the reference value and identify the rank assigned to the interval to which the target data belongs.

[0077] <Processing Procedure> The visualization process performed by the visualization system 1000 will be explained with reference to Figures 4 to 12. The visualization process may also be performed by the visualization device 10. Figure 4 is a flowchart of an example of the visualization process.

[0078] The visualization process may be performed after the trained model has been generated. That is, the visualization device 10 may perform the visualization process after the model generation unit 110 has generated a trained model based on the training data stored in the data storage unit 101, and after the trained model has been stored in the model storage unit 102. However, the visualization process may be performed at any time. For example, the visualization process may be performed before the trained model has been generated.

[0079] In step S1, the acquisition unit 130 of the visualization device 10 reads the training data stored in the data storage unit 101. The acquisition unit 130 also reads the target data stored in the data storage unit 101.

[0080] The acquisition unit 130 sends the learning data and target data read from the data storage unit 101 to the anomaly calculation unit 140. The acquisition unit 130 also sends the learning data read from the data storage unit 101 to the region division unit 160.

[0081] In step S2, the abnormality calculation unit 140 of the visualization device 10 receives the training data and target data from the acquisition unit 130. The abnormality calculation unit 140 calculates the abnormality of the training data. The abnormality calculation unit 140 sends the abnormality of the training data to the reference value calculation unit 150.

[0082] The process of calculating the anomaly score of the training data (step S2 in Figure 4) will be explained in more detail with reference to Figures 5 and 6. Figure 5 is a flowchart showing an example of the process of calculating the anomaly score of the training data.

[0083] In step S11, the anomaly calculation unit 140 normalizes the training data. Specifically, the anomaly calculation unit 140 normalizes the training data so that it is between 0 and 1 (inclusive). In other words, the anomaly calculation unit 140 subtracts the minimum value of the training data from each training data and divides by the absolute value of the difference between the minimum value and the maximum value of the training data.

[0084] In step S12, the anomaly calculation unit 140 removes variables with high correlations from the training data. Specifically, the anomaly calculation unit 140 removes variables with a correlation coefficient of 1 or a correlation coefficient of -1 from the training data.

[0085] In step S13, the anomaly calculation unit 140 removes data with small variance from the training data. Specifically, the anomaly calculation unit 140 removes data with a variance of 0 from the training data.

[0086] In step S14, the anomaly calculation unit 140 calculates the distance of each training data to each other training data. The anomaly calculation unit 140 calculates the minimum distance of each training data to each other as the anomaly score for that training data.

[0087] Figure 6 illustrates an example of the anomaly score calculation process for training data. Figure 6 shows the process of calculating the anomaly score for each of the N training data. First, the anomaly score calculation unit 140 calculates the Euclidean distance for each of the N training data to each of the other training data. This generates an N × N distance matrix. The diagonal elements of this distance matrix are the distances between identical training data (i.e., 0) and are therefore not calculated. Then, the anomaly score calculation unit 140 selects the minimum value among the Euclidean distances for each of the N training data to each of the other training data. This allows the anomaly score calculation unit 140 to calculate the anomaly score for each of the training data.

[0088] Let's return to Figure 4 for explanation. In step S3, the reference value calculation unit 150 of the visualization device 10 receives the anomaly score of the training data from the anomaly score calculation unit 140. The reference value calculation unit 150 calculates a reference value based on the distribution of anomaly scores of the training data. The reference value calculation unit 150 sends the reference value to the visualization unit 180.

[0089] The reference value calculation process (step S3 in Figure 4) will be explained in more detail with reference to Figures 7 and 8. Figure 7 is a flowchart of an example of the reference value calculation process.

[0090] In step S21, the reference value calculation unit 150 calculates the furthest distance. Specifically, the reference value calculation unit 150 calculates the furthest distance as the square root of the number of dimensions in the data space of the training data.

[0091] In step S22, the reference value calculation unit 150 normalizes the anomaly score of the training data by the furthest distance. Specifically, the reference value calculation unit 150 divides the anomaly score of each training data by the furthest distance.

[0092] In step S23, the reference value calculation unit 150 calculates reference values. In this embodiment, the reference value calculation unit 150 calculates reference values ​​that divide the anomaly degree of the training data into four parts. Specifically, the reference value calculation unit 150 calculates four points: the 25th percentile, the 75th percentile, the 75th percentile + 1.5 × interquartile range, and the 75th percentile + 3 × interquartile range. The reference value calculation unit 150 sends the reference values ​​to the visualization unit 180.

[0093] In step S24, the visualization unit 180 receives a reference value from the reference value calculation unit 150. Based on the reference value, the visualization unit 180 divides the distribution of anomalies in the training data into multiple intervals. The visualization unit 180 assigns ranks to the intervals divided based on the reference value. Specifically, the visualization unit 180 divides the distribution of anomalies in the training data into multiple intervals, sorts each of the multiple intervals in ascending order of anomaly, and assigns ranks to indicate the order from the beginning.

[0094] Figure 8 is a diagram illustrating an example of the reference value calculation process. As shown in Figure 8, the reference value calculation unit 150 calculates the 25th percentile (q) based on the distribution of anomalies in the training data. 25 ), 75% quantile (q 75 (First reference value), 75th percentile + 1.5 × interquartile range (IQR = q 75 -q 25The visualization unit 180 calculates the second reference value and the 75th percentile + 3 × interquartile range (third reference value). Then, the visualization unit 180 assigns rank 1 to intervals below the first reference value. The visualization unit 180 also assigns rank 2 to intervals between the first reference value and the second reference value. The visualization unit 180 also assigns rank 3 to intervals between the second reference value and the third reference value. The visualization unit 180 also assigns rank 4 to intervals above the third reference value.

[0095] Let's return to Figure 4 for explanation. In step S4, the abnormality calculation unit 140 of the visualization device 10 calculates the abnormality of the target data received in step S2. The abnormality calculation unit 140 sends the abnormality of the target data to the visualization unit 180.

[0096] The process for calculating the anomaly severity of the target data (step S4 in Figure 4) will be explained in more detail with reference to Figures 9 and 10. Figure 9 is a flowchart showing an example of the process for calculating the anomaly severity of the target data.

[0097] In step S31, the anomaly calculation unit 140 normalizes the target data. Specifically, the anomaly calculation unit 140 normalizes the target data using the same rules as the normalization performed in step S11 (see Figure 5). Note that the normalized target data is normalized using the same rules as the training data, so it may not be between 0 and 1 (inclusive).

[0098] In step S32, the abnormality calculation unit 140 removes variables from the target data. Specifically, the abnormality calculation unit 140 removes the variables that were removed in step S12 (see Figure 5) from the target data.

[0099] In step S33, the anomaly calculation unit 140 calculates the distance for each target data to each training data. The anomaly calculation unit 140 calculates the minimum distance to the training data for each target data as the anomaly score for that target data. The anomaly calculation unit 140 sends the anomaly score for the target data to the visualization unit 180.

[0100] Figure 10 is a diagram illustrating the process of calculating the anomaly score of the target data. Figure 10 shows the process of calculating the anomaly score for each of the M target data. First, the anomaly score calculation unit 140 calculates the Euclidean distance for each of the M target data with each of the N training data. This generates an M × N distance matrix. Then, the anomaly score calculation unit 140 selects the minimum value among the Euclidean distances with the training data for each of the M target data. This allows the anomaly score calculation unit 140 to calculate the anomaly score for each of the target data.

[0101] Let's return to Figure 9 for explanation. In step S34, the visualization unit 180 receives the anomaly score of the target data from the anomaly score calculation unit 140. The visualization unit 180 normalizes the anomaly score of the target data by the furthest distance. Specifically, the visualization unit 180 divides the anomaly score of each target data by the furthest distance. The furthest distance may be the furthest distance calculated in step S21 (see Figure 7), or it may be newly calculated based on the training data.

[0102] In step S35, the visualization unit 180 identifies the rank of the target data. Based on a reference value, the visualization unit 180 identifies the interval to which the anomaly of the target data belongs and identifies the rank assigned to that interval. For example, if the anomaly of the target data lies between the 25th percentile and the 75th percentile in the distribution of anomaly of the training data, the visualization unit 180 identifies that the target data is rank 1.

[0103] Let's return to Figure 4 for explanation. In step S5, the region division unit 160 of the visualization device 10 receives training data from the acquisition unit 130. The region division unit 160 divides the data space of the training data based on the training data. Specifically, the region division unit 160 divides the data space of the training data into an interpolation region and an extrapolation region by calculating the convex hull of the training data. The region division unit 160 sends region information indicating the interpolation region and the extrapolation region to the determination unit 170.

[0104] Figure 11 is a diagram illustrating an example of the region partitioning process. As shown in Figure 11, the region partitioning unit 160 may calculate the convex hull 601 of the training data in the training data data space 600. As a result, the training data data space 600 is divided into an interpolation region 610 and an extrapolation region 620. The interpolation region 610 is the space inside the convex hull 601. The extrapolation region 620 is the space outside the convex hull 601.

[0105] Let's return to Figure 4 for explanation. In step S6, the determination unit 170 of the visualization device 10 receives region information from the region division unit 160. Based on the region information, the determination unit 170 determines whether the target data belongs to an interpolation region or an extrapolation region. Specifically, the determination unit 170 determines that the target data belongs to an interpolation region if it is located inside the convex hull. On the other hand, the determination unit 170 determines that the target data belongs to an extrapolation region if it is located outside the convex hull. The determination unit 170 sends the determination result for the target data to the visualization unit 180. The determination result for the target data may, for example, be information indicating whether each target data belongs to an interpolation region or an extrapolation region.

[0106] In step S7, the visualization unit 180 of the visualization device 10 receives the determination result of the target data from the determination unit 170. Based on the determination result of the target data, the visualization unit 180 divides the target data into interpolated data belonging to the interpolation region and extrapolated data belonging to the extrapolation region.

[0107] The visualization unit 180 generates visualization results. Specifically, the visualization unit 180 generates visualization results that show the distribution of anomalies in the interpolated data and the distribution of anomalies in the extrapolated data in correspondence. The visualization unit 180 transmits the visualization results to the terminal device 50. The terminal device 50 displays the visualization results received from the visualization device 10 on the display device 506 of the terminal device 50.

[0108] Figure 12 shows an example of the visualization results. As shown in Figure 12, the visualization result 700 may include the frequency distribution 710 of the extrapolated data and the frequency distribution 720 of the interpolated data. The frequency distribution 710 of the extrapolated data may also be the frequency distribution of the anomaly degree (minimum distance) between the extrapolated data and the training data. The frequency distribution 720 of the interpolated data may also be the frequency distribution of the anomaly degree (minimum distance) between the interpolated data and the training data. The horizontal axis of the frequency distributions 710 and 720 may be the normalized distance between data points.

[0109] The frequency distribution 710 of the extrapolated data and the frequency distribution 720 of the interpolated data may be joined vertically. Since both the frequency distribution 710 of the extrapolated data and the frequency distribution 720 of the interpolated data are normalized with the horizontal axis between 0 and 1, the distribution of anomalies in the extrapolated data and the distribution of anomalies in the interpolated data can be easily compared.

[0110] The frequency distribution 710 of the extrapolated data may represent a criterion value 711 that divides the anomaly score of the training data into two parts. The region 712 to the left of the criterion value 711 is a region that shows the distribution of target data with a weak degree of extrapolation (weak extrapolation region) because the anomaly score is smaller than the criterion value 711. The region 713 to the right of the criterion value 711 is a region that shows the distribution of target data with a strong degree of extrapolation (strong extrapolation region) because the anomaly score is greater than the criterion value 711.

[0111] The frequency distribution 720 of the interpolated data may represent a criterion value 721 that divides the anomaly score of the training data into two parts. The region 722 to the left of the criterion value 721 is a region that shows the distribution of target data with a strong degree of interpolation (strong interpolation region) because the anomaly score is smaller than the criterion value 721. The region 723 to the right of the criterion value 721 is a region that shows the distribution of target data with a weak degree of interpolation (weak interpolation region) because the anomaly score is greater than the criterion value 721.

[0112] As shown in the visualization result 700 in Figure 12, by displaying the distribution of anomalies in the interpolated data and the distribution of anomalies in the extrapolated data in correspondence, users of the visualization system 1000 can easily understand whether the target data tends to belong to the interpolation region or the extrapolation region. Since it is known that target data belonging to the extrapolation region tends to have lower prediction accuracy, for example, if the target data belongs to the extrapolation region and its anomaly is greater than the threshold value, it can be judged that the reliability of the prediction result based on the target data is low. On the other hand, for example, if the target data belongs to the interpolation region and its anomaly is greater than the threshold value, it can be judged that the reliability of the prediction result based on the target data is somewhat high.

[0113] <Effects of the First Embodiment> The visualization device 10 according to this embodiment acquires target data for predicting the target variable using a trained model, calculates the anomaly score of the target data based on the training data used to train the trained model, determines whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the training data, and displays the distribution of anomalies of the target data belonging to the interpolation region and the distribution of anomalies of the target data belonging to the extrapolation region in correspondence.

[0114] In one respect, according to this embodiment, the distribution of anomalies in the target data is displayed separately in the interpolation and extrapolation regions, making it easy to understand the degree of prediction accuracy of the prediction results based on the target data. Therefore, according to this embodiment, the reliability of the prediction results based on the target data can be visualized.

[0115] The visualization device 10 may determine that the target data located inside the convex hull of the training data belongs to the interpolation region. The visualization device 10 may also determine that the target data located outside the convex hull of the training data belongs to the extrapolation region. In one aspect, according to this embodiment, it is possible to accurately determine whether the target data belongs to the interpolation region or the extrapolation region based on the convex hull of the training data.

[0116] The visualization device 10 may calculate the minimum distance between the target data and the training data as the anomaly score. In one respect, according to this embodiment, the degree to which the target data deviates from the training data can be calculated with high accuracy based on the distance between the target data and the training data.

[0117] The visualization device 10 may calculate a reference value based on the distribution of anomalies in the training data. The visualization device 10 may also divide and display the distribution of anomalies in the target data based on the reference value. In one aspect, according to this embodiment, since the distribution of anomalies in the target data is divided, the anomalies in the target data can be displayed in an easy-to-understand manner.

[0118] The visualization device 10 may calculate a criterion value for dividing the anomaly score of the training data into two. The visualization device 10 may display the distribution of the anomaly score of the target data in a four-quadrant area including a strong interpolation region, a weak interpolation region, a strong extrapolation region, and a weak extrapolation region. In one aspect, according to this embodiment, the distribution of the anomaly score of the target data is displayed in a four-quadrant area based on the degree and strength of interpolation and extrapolation, making it easier to understand the reliability of the prediction results.

[0119] With the configuration described above, this embodiment makes it possible to quantify the degree of interpolation of the target data. For example, in this embodiment, the distribution of anomalies in the interpolated data and the distribution of anomalies in the extrapolated data are normalized and displayed by the maximum distance in the data space, so that the degree of interpolation in the interpolated region and the degree of extrapolation in the extrapolated region can be quantitatively compared.

[0120] Furthermore, the configuration described above allows for the visualization of the reliability of prediction results based on the target data in this embodiment. For example, in this embodiment, the degree of anomaly in the target data can be visualized using a four-quadrant system based on the degree and strength of interpolation and extrapolation, allowing for the evaluation of the positioning of experiments using that target data. For example, target data belonging to the strong extrapolation region is assumed to have low prediction accuracy, and can therefore be evaluated as a very challenging experiment. On the other hand, for example, target data belonging to the weak interpolation region is assumed to have a relatively high prediction accuracy, and can therefore be evaluated as a fill-in-the-blank experiment.

[0121] [Second Embodiment] The visualization system 1000 described in the first embodiment can be used to select a trained model suitable for prediction. In the second embodiment, a configuration is described in which a trained model suitable for predicting target data is selected from among multiple trained models with different explanatory variables.

[0122] The following describes the visualization system 1000 according to the second embodiment, focusing on the differences from the first embodiment. Unless otherwise specified, the visualization system 1000 according to the second embodiment may be configured in the same way as the first embodiment.

[0123] <Functional Configuration> The functional configuration of the visualization system 1000 according to the second embodiment will be described with reference to Figure 13. Figure 13 is a block diagram showing an example of the functional configuration of the visualization system according to the second embodiment.

[0124] As shown in Figure 13, the visualization device 10 includes a data storage unit 101, a model storage unit 102, a data partitioning unit 105, a model generation unit 110, a prediction unit 120, an acquisition unit 130, an anomaly calculation unit 140, a reference value calculation unit 150, a region partitioning unit 160, a determination unit 170, a model selection unit 175, and a visualization unit 180. In other words, the visualization device 10 according to the second embodiment differs from the first embodiment in that it further includes a data partitioning unit 105 and a model selection unit 175.

[0125] The data splitting unit 105 splits the training data. The data splitting unit 105 may split the training data read from the data storage unit 101. The data splitting unit 105 may split the training data into multiple subdata sets. The multiple subdata sets may contain combinations of explanatory variables that are different from each other. The multiple subdata sets may contain a common target variable. The multiple subdata sets may have all different explanatory variables, or some explanatory variables may be the same. The multiple subdata sets may contain the same number of explanatory variables, or they may contain different numbers of explanatory variables.

[0126] In this embodiment, the model generation unit 110 may generate multiple pre-trained models. The model generation unit 110 may generate multiple pre-trained models based on each of the multiple partial data. The model generation unit 110 may generate multiple pre-trained models based on each of the multiple partial data divided by the data division unit 105. The model generation unit 110 may store the multiple pre-trained models in the model storage unit 102.

[0127] In this embodiment, the domain partitioning unit 160 may partition the data space of each of the multiple subdata. The domain partitioning unit 160 may partition the data space of each of the multiple subdata into an interpolation region and an extrapolation region. The domain partitioning unit 160 may partition the data space of each of the multiple subdata into an interpolation region and an extrapolation region by calculating the convex hull of each of the multiple subdata. The domain partitioning unit 160 may partition the data space of each of the multiple subdata into an interpolation region and an extrapolation region by Delaunay partitioning using each of the multiple subdata.

[0128] In this embodiment, the determination unit 170 may determine for each of the multiple subdata whether the target data belongs to the interpolation region or the extrapolation region. Before making a determination for a certain subdata, the determination unit 170 may select the same explanatory variable from the target data as that subdata. For each of the multiple subdata, the determination unit 170 may determine that the target data belongs to the interpolation region for that subdata if the target data is located inside the convex hull of that subdata. On the other hand, for each of the multiple subdata, the determination unit 170 may determine that the target data belongs to the extrapolation region for that subdata if the target data is located outside the convex hull of that subdata.

[0129] In this embodiment, the visualization unit 180 displays the distribution of anomalies in the target data for each of the multiple partial data. The visualization unit 180 may also display the distribution of anomalies in the interpolated data and the distribution of anomalies in the extrapolated data in association for each of the multiple partial data. The visualization unit 180 may also display the distribution of anomalies in the target data for each of the multiple partial data by dividing it into multiple intervals. The visualization unit 180 may also display the distribution of anomalies in the target data for each of the multiple partial data in a four-quadrant including a strong interpolation region, a weak interpolation region, a strong extrapolation region, and a weak extrapolation region.

[0130] The model selection unit 175 selects a pre-trained model. The model selection unit 175 may select one pre-trained model from multiple pre-trained models. The model selection unit 175 may select one pre-trained model from multiple pre-trained models generated by the model generation unit 110. The model selection unit 175 may select a pre-trained model from among multiple pre-trained models that is based on partial data where the target data belongs to the interpolation region. The model selection unit 175 may select a pre-trained model for each of the target data that is based on partial data where the target data belongs to the interpolation region.

[0131] The model selection unit 175 may automatically select a pre-trained model based on the visualization results from the visualization unit 180. The model selection unit 175 may also select a pre-trained model selected by the user of the visualization system 1000. The user of the visualization system 1000 may decide which pre-trained model to select based on the visualization results.

[0132] In this embodiment, the prediction unit 120 may predict the target variable based on the trained model selected by the model selection unit 175. The prediction unit 120 may also predict the target variable by inputting the explanatory variables included in the target data into the trained model selected by the model selection unit 175. The prediction unit 120 may also predict the target variable by inputting the explanatory variables included in each target data into the trained model selected for each target data.

[0133] <Processing Procedure> The prediction process performed by the visualization system 1000 will be explained with reference to Figure 14. The prediction process may also be performed by the visualization device 10. Figure 14 is a flowchart of an example of the prediction process.

[0134] In step S41, the data partitioning unit 105 of the visualization device 10 reads the training data from the data storage unit 101. The data partitioning unit 105 partitions the training data so that each partition contains a different combination of explanatory variables. This generates multiple partial data sets, each containing a different combination of explanatory variables. The data partitioning unit 105 sends the multiple partial data sets to the model generation unit 110.

[0135] In step S42, the model generation unit 110 of the visualization device 10 receives multiple partial data from the data division unit 105. The model generation unit 110 generates multiple trained models based on each of the multiple partial data. The model generation unit 110 stores the multiple trained models in the model storage unit 102.

[0136] In step S43, the visualization device 10 performs visualization processing for each of the multiple partial data. The visualization processing performed in step S43 is the same as the visualization processing according to the first embodiment (see Figure 4), except that it uses the partial data generated in step S41 instead of the training data.

[0137] In step S44, the model selection unit 175 of the visualization device 10 selects a trained model to be used for prediction from among the multiple trained models generated in step S42. Specifically, for each target data, the model selection unit 175 selects a trained model based on the partial data to which the target data belongs in the interpolation region. The model selection unit 175 sends the trained model selection result to the prediction unit 120. The trained model selection result may include information indicating the trained model selected for each target data.

[0138] In step S45, the prediction unit 120 of the visualization device 10 receives the selection result of a trained model from the model selection unit 175. Based on the selection result of the trained model, the prediction unit 120 reads the selected trained model from the model storage unit 102. The prediction unit 120 inputs the explanatory variables included in the target data into the read trained model. The trained model predicts the target variable based on the input explanatory variables and outputs the predicted value. The prediction unit 120 obtains the predicted value output from the trained model.

[0139] The prediction unit 120 generates prediction results based on the target data. The prediction results based on the target data include the predicted values ​​output from the trained model. The prediction unit 120 transmits the prediction results based on the target data to the terminal device 50. The terminal device 50 displays the prediction results received from the visualization device 10 on the display device 506 of the terminal device 50.

[0140] <Effects of the second embodiment> In this embodiment, the visualization device 10 may divide the training data into a plurality of partial data sets containing combinations of variables that are different from each other. The visualization device 10 may determine whether the target data belongs to the interpolation region or the extrapolation region for each of the plurality of partial data sets. The visualization device 10 may also display, in association with each of the plurality of partial data sets, the distribution of anomalies of the target data belonging to the interpolation region and the distribution of anomalies of the target data belonging to the extrapolation region. In one aspect, according to this embodiment, since it is determined whether the target data belongs to the interpolation region or the extrapolation region for combinations of variables that are different from each other, it is possible to visualize combinations of variables that increase the reliability of the prediction results.

[0141] The visualization device 10 may generate multiple trained models for each of the multiple partial data points. The visualization device 10 may select a trained model from among the multiple trained models that is based on the partial data where the target data belongs to the interpolation region. The visualization device 10 may predict the target variable by inputting the target data into the selected trained model. In one aspect, according to this embodiment, the target variable can be predicted with high accuracy because it is predicted using a combination of variables that increases the reliability of the prediction result.

[0142] As configured as described above, this embodiment can present the user with selection criteria for a pre-trained model. For example, in this embodiment, the reliability of the prediction results for each combination of different explanatory variables of the target data can be visualized, thus providing criteria for selecting a pre-trained model with high prediction accuracy for that target data. For example, suppose that for a given set of target data, the first partial data is determined to belong to a strong extrapolation region, and the second partial data is determined to belong to a weak interpolation region. In this case, it is expected that prediction based on the second partial data, which belongs to the weak interpolation region, will result in higher prediction accuracy. Therefore, for this target data, predicting the target variable using a pre-trained model generated based on the second partial data will allow for more accurate prediction of the target variable.

[0143] [supplement] Each of the embodiments described above can be implemented by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as CPUs (Central Processing Units) or GPUs (Graphics Processing Units) implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each of the functions described above.

[0144] While embodiments of the present disclosure have been described in detail above, the embodiments disclosed herein are illustrative and not restrictive in all respects. The embodiments can be modified and improved in various ways without departing from the scope and spirit of the appended claims. The features described in the above embodiments can be combined in any way that is not inconsistent with other configurations. [Explanation of symbols]

[0145] 10: Visualization device 50: Terminal device 101: Data storage unit 102: Model Memory Unit 105: Data partitioning section 110: Model generation unit 120: Prediction Department 130: Acquisition Department 140: Abnormality calculation part 150: Reference Value Calculation Unit 160: Area division part 170: Judgment section 175: Model Selection Section 180: Visualization section 1000: Visualization System

Claims

1. An acquisition unit that acquires target data for predicting the target variable using a pre-trained model, An anomaly calculation unit that calculates the anomaly of the target data based on the training data used to train the aforementioned trained model, A determination unit that determines whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the training data, A visualization unit that displays the distribution of abnormality of the target data belonging to the interpolation region and the distribution of abnormality of the target data belonging to the extrapolation region in correspondence. A visualization device equipped with the following features.

2. The determination unit determines that the target data located inside the convex hull of the learning data belongs to the interpolation region, and determines that the target data located outside the convex hull belongs to the extrapolation region. The visualization device according to claim 1.

3. The anomaly score calculation unit calculates the minimum distance between the target data and the training data as the anomaly score. The visualization device according to claim 1.

4. The system further includes a reference value calculation unit that calculates a reference value based on the distribution of anomalies in the training data, The visualization unit divides and displays the distribution of the degree of abnormality of the target data based on the reference value. A visualization device according to any one of claims 1 to 3.

5. The reference value calculation unit calculates the reference value that divides the degree of abnormality of the learning data into two parts, The visualization unit displays the distribution of the anomaly of the target data in four quadrants, including a strong interpolation region, a weak interpolation region, a strong extrapolation region, and a weak extrapolation region. The visualization device according to claim 4.

6. The data splitting unit further comprises dividing the training data into multiple partial data sets containing different combinations of explanatory variables, The determination unit determines, for each of the plurality of partial data, whether the target data belongs to the interpolation region or the extrapolation region. The visualization unit displays, for each of the plurality of partial data, the distribution of the abnormality of the target data belonging to the interpolation region and the distribution of the abnormality of the target data belonging to the extrapolation region in correspondence. A visualization device according to any one of claims 1 to 3.

7. A model generation unit that generates multiple trained models based on each of the aforementioned multiple partial data sets, A model selection unit selects from among the plurality of trained models a trained model based on the partial data to which the target data belongs in the interpolation region, A prediction unit that predicts the target variable by inputting the target data into the selected pre-trained model, The visualization device according to claim 6, further comprising:

8. Computers The procedure for obtaining target data to predict the target variable using a pre-trained model, A procedure for calculating the anomaly score of the target data based on the training data used to train the aforementioned trained model, A procedure for determining whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the training data, A procedure for displaying the distribution of abnormality of the target data belonging to the interpolation region and the distribution of abnormality of the target data belonging to the extrapolation region in correspondence, A visualization method for performing this task.

9. On the computer, The procedure for obtaining target data to predict the target variable using a pre-trained model, A procedure for calculating the anomaly score of the target data based on the training data used to train the aforementioned trained model, A procedure for determining whether the target data belongs to the interpolation region or the extrapolation region based on the distribution of the training data, A procedure for displaying the distribution of abnormality of the target data belonging to the interpolation region and the distribution of abnormality of the target data belonging to the extrapolation region in correspondence, A program to execute.

Images