GENERATION METHOD, GENERATION PROGRAM, AND INFORMATION PROCESSING APPARATUS

Abstract

In the present invention, a computer is caused to execute processing including: generating first data on the basis of a first tree among a plurality of trees that hierarchically classifies a plurality of pieces of data belonging to a root node into nodes lower than the root node by a random threshold value, the first tree being one in which the number of pieces of data classified into leaf nodes among the plurality of nodes is equal to or lower than a reference value, the first data being generated in a domain of any leaf node of the plurality of the leaf nodes; and determining validity indicating that the first data is data out of distribution of the plurality of pieces of data on the basis of a first distance and a second distance, the first distance being from the root node to the leaf node where the first data has been generated, the second data being, in each of one or more second trees different from the first tree among the plurality of trees, from the root node of the second tree to a node of the second tree including the first data in the domain.

Description

[Technical field]

[0001] The present invention relates to a generation method, a generation program, and an information processing device. [Background technology]

[0002] Isolation Forest (iF) is known as one of the anomaly detection algorithms that detects abnormal data from multiple data.

[0003] iF is a method to obtain an index that indicates the "out-of-distribution likelihood" of each piece of data in the distribution of multiple data by using a tree (iT: Isolation Tree) that separates (splits) multiple pieces of data based on variables in the data. Out-of-distribution data is data that has a distribution that is different (e.g., deviates) from the distribution of multiple data (hereinafter, sometimes referred to as "out-of-distribution data"). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021-095101 Brochure Summary of the Invention [Problem to be solved by the invention]

[0005] Machine learning algorithms may be vulnerable to out-of-distribution data. For example, if out-of-distribution data is input to a machine learning model trained with a plurality of data from a given distribution, the machine learning model may output an inappropriate judgment result.

[0006] In order to suppress misjudgments by machine learning algorithms, it is conceivable to use artificially generated out-of-distribution data (hereinafter sometimes referred to as "pseudo out-of-distribution data") to train a machine learning model to recognize that the pseudo out-of-distribution data is "out-of-distribution data." For example, it is assumed that an index indicating the likelihood of out-of-distribution, which can be obtained from iF, is used to generate the pseudo out-of-distribution data.

[0007] However, in iF, the variables to be split in the data (which variables to split) and the splitting threshold are randomly determined, so the index indicating the out-of-distribution likelihood may differ for each tree even for the same data. Therefore, out-of-distribution data generated artificially based on the index indicating the out-of-distribution likelihood may become pseudo out-of-distribution data that is inappropriate (inappropriate) for training a machine learning model, which may reduce the efficiency of generating out-of-distribution data.

[0008] In one aspect, an object of the present invention is to improve the efficiency of generating out-of-distribution data. [Means for solving the problem]

[0009] In one aspect, the generating method may include a computer executing the following process. The process may generate first data in a domain of one of a plurality of leaf nodes based on a first tree among the plurality of trees, the first tree being a first tree in which a number of data items classified into a leaf node among the plurality of nodes is equal to or less than a reference value. The process may also determine the validity of the first data being data outside the distribution of the plurality of data items based on a first distance from the root node to a leaf node from which the first data item is generated, and one or more second distances from a root node of one or more second trees different from the first tree among the plurality of trees to a node of the second tree that includes the first data item in its domain. Effect of the Invention

[0010] In one aspect, the present invention can improve the efficiency of generating out-of-distribution data. [Brief description of the drawings]

[0011] [Figure 1] FIG. 13 is a diagram for explaining an example of out-of-distribution data. [Diagram 2] FIG. 13 is a diagram illustrating an example of data division using a tree. [Diagram 3] FIG. 3 is a diagram showing an example of a tree-format representation of the division shown in FIG. 2. [Figure 4] FIG. 13 is a diagram illustrating another example of data division using a tree. [Diagram 5] FIG. 2 is a block diagram showing an example of the hardware (HW) configuration of a computer that realizes the functions of a generating device according to an embodiment. [Figure 6] FIG. 2 is a block diagram illustrating an example of a functional configuration of a generating device according to an embodiment. [Figure 7] FIG. 13 is a diagram illustrating an example of a histogram of distribution density of pl (Path Length). [Figure 8] FIG. 13 is a diagram showing an example of derivation of depth d by estimating the distribution density of pl. [Figure 9] FIG. 13 is a diagram illustrating an example of out-of-distribution data generated using a data generation tree. [Figure 10] FIG. 13 is a diagram illustrating an example of out-of-distribution data in a data evaluation tree. [Figure 11] 11 is a flowchart illustrating an example of an operation of a generating device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or application of techniques not specified below. For example, this embodiment can be modified in various ways without departing from the spirit of the invention. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0013] [1] One embodiment [1-1] Explanation of out-of-distribution data Fig. 1 is a diagram for explaining an example of out-of-distribution data. As shown in Fig. 1, it is assumed that a machine learning model is trained to separate classes #1 and #2 at a discrimination boundary 101 using training data including a plurality of data having a first distribution and a plurality of data having a second distribution. Note that class #1 is a classification corresponding to data belonging to the first distribution, and class #2 is a classification corresponding to data belonging to the second distribution.

[0014] For example, when a machine learning algorithm is input with a sample from an area not included in the training data (out-of-distribution data 102), the data 102 is likely to be classified into class #1 with a high degree of confidence because the data 102 is on the class #1 side of the decision boundary 101.

[0015] If the data 102 is data for a cyber attack, the classification of the data 102 into class #1 corresponds to the attack being overlooked. In order to prevent such an attack from being overlooked, for example, it is conceivable to artificially generate out-of-distribution data such as the data 102 and train a machine learning model so that the machine learning model learns that the data 102 is "out-of-distribution data."

[0016] In one embodiment, the method of iF, which splits multiple data into individual data using a tree (iT), is used as a method for generating out-of-distribution data.

[0017] FIG. 2 is a diagram showing an example of data division using a tree, and FIG. 3 is a diagram showing an example of an expression in the tree format of the division shown in FIG.

[0018] The tree is a tree in which the variables to be split (which variables to split) and the splitting threshold are random. In one embodiment, as illustrated in Fig. 2, x1 and x2 (see the horizontal and vertical axes, respectively) are selected as variables from among multiple variables included in data 120 (see Fig. 3). In addition, as the threshold, a predetermined value (see reference numeral 111) for variable x1 or x2 is selected as shown by the dashed line.

[0019] For example, as shown in Fig. 3, iF selects a node, selects a variable and a threshold, and repeats the procedure of dividing the selected node into multiple (e.g., two) nodes according to the selected variables and thresholds until each node contains only one data (sample). A node means a box in the tree in which data that satisfies a condition is stored, and is a range bounded by a threshold 111 or an axis in Fig. 2, and a rectangular frame in Fig. 3. The initial state of a node may include all data 120 as shown in Fig. 3.

[0020] In this way, iF includes multiple trees that hierarchically classify multiple data belonging to a root node into nodes lower than the root node by a random threshold. Also, the number of data classified into the terminal (lowest) leaf node among the multiple nodes is equal to or less than a reference value. In one embodiment, the reference value is "1", but is not limited to this and may be "2" or more.

[0021] 3, iF selects a variable x2 and a threshold value of "0.4" and divides the node of all data 120 (root node) into a node including data that satisfies x2≦0.4 and a node including data that satisfies x2>0.4. Also, for the node including data that satisfies x2≦0.4, iF selects a variable x1 and a threshold value of "0.5" and divides the node into a node including data that satisfies x1≦0.5 and a node 112 including data that satisfies x1>0.5.

[0022] iF ends the division of node 112 because there is only one data (data 113; see FIG. 2) in node 112 that contains data that satisfies x1>“0.5”. Node 112 after division may be called a leaf node. iF continues to divide nodes that contain data that satisfies x1≦“0.5” using a similar procedure, dividing them into leaf nodes such as node 114. iF ends the process when it has divided each of all data 120 into a leaf node.

[0023] iF may include, for example, multiple machine learning models (hereinafter, may be referred to as "tree models") that generate a tree. The tree model is a model for generating a tree by repeating a process of selecting variables and a threshold and dividing data (nodes) until division of all input data is completed (reach a leaf node). In one embodiment, the multiple tree models included in iF all select the same combination of the same variables (for example, two variables x1 and x2). The number of variables may be three or more, or one.

[0024] In iF, the number of nodes (including the root node) passed from the root node to a leaf node, in other words, the distance from the root node to a leaf node, is called pl (Path Length) or "depth of a leaf node." In the example of Fig. 3, leaf node 112 has pl="2", and leaf node 114 has pl="4".

[0025] For example, in FIG. 2, reference numeral 112 denotes a relatively "sparse" region. In a "sparse" region, node division is completed relatively early (the number of divisions is small), resulting in shallow leaf nodes; in other words, pl is relatively small. On the other hand, reference numeral 114 denotes a relatively "dense" region. In a "dense" region, node division is completed relatively late (the number of divisions is large), resulting in deep leaf nodes; in other words, pl is relatively large.

[0026] In this way, pl can be regarded as an example of an index that indicates the likelihood that each piece of data is outside the distribution of all data 120.

[0027] Here, let us consider a case where a computer that generates pseudo out-of-distribution data, for example a server, generates pseudo out-of-distribution data using iF. As a method for generating pseudo out-of-distribution data, for example, the following steps (i) to (iii) are assumed.

[0028] (i) The server selects a node depth d for generating pseudo-out-of-distribution data. (ii) The server randomly selects a tree in iF and one leaf node of depth d. (iii) The server generates data (samples) uniformly randomly from the region (domain) of the selected leaf nodes.

[0029] As described above, nodes with small pl, in other words, nodes at shallow positions on the tree, are "sparse" regions, so data generated uniformly at random in nodes with small depth d will be samples that are likely to be outside the distribution (highly likely to be outside the distribution). On the other hand, nodes with large pl, in other words, nodes at deep positions on the tree, are "dense" regions, so data generated uniformly at random in nodes with large depth d will be samples that are likely to be within the distribution (highly likely to be within the distribution).

[0030] Therefore, if the server can select an appropriate depth d (in other words, a predetermined criterion) in step (i), it can generate appropriate out-of-distribution data in steps (ii) and (iii).

[0031] In addition, appropriate out-of-distribution data may mean, for example, out-of-distribution data that may be erroneously classified into a class of in-distribution data by a machine learning model trained without using out-of-distribution data (using in-distribution data). The generated out-of-distribution data is used to train the machine learning model. This makes it possible to train the machine learning model to determine (classify) the out-of-distribution data as out-of-distribution data using a data set in which it is difficult to determine whether the data is in-distribution or out-of-distribution.

[0032] However, while iF can obtain the pl of individual nodes, it is not expected to specify an appropriate depth d. For example, if the depth d is too large, the generated samples will be within the distribution. On the other hand, if the depth d is too small, the generated samples will be out of the distribution, but it is clear that they are out of the distribution, and they will be inappropriate samples for training machine learning models.

[0033] As mentioned above, iF uses multiple tree models with random variable and threshold selection, so that pseudo-outlier data generated based on one tree may be inappropriate (not likely to be outlier) for other trees.

[0034] 4 is a diagram showing another example of dividing data by a tree. As shown in FIG 4, in the tree, data is divided by a threshold 131 and an axis.

[0035] In the example of Fig. 4, node (area) 132 separated by threshold 131 and axis has pl="3", and node (area) 133 has pl="4". According to the relationship between the value of pl and the likelihood of being out of distribution, a sample generated in node 132 with pl="3" should have a greater index of likelihood of being out of distribution than a sample generated in node 133 with pl="4". However, as illustrated in Fig. 4, due to the randomness of the division, a reversal in the relationship between the value of pl and the likelihood of being out of distribution may occur.

[0036] Thus, in the proposed approach, it may be difficult to select an appropriate depth d based on the tree and / or to determine whether the generated samples are valid.

[0037] Therefore, in one embodiment, an example of a method for generating appropriate pseudo out-of-distribution data by selecting an appropriate depth d and enabling judgment of the validity of the pseudo out-of-distribution data generated at the node of the selected depth d will be described. Note that, although both methods of selecting an appropriate depth d and judging the validity of the pseudo out-of-distribution data will be described below, it is possible to generate pseudo out-of-distribution data more appropriate than the above-mentioned expected method by implementing either one of them.

[0038] [1-2] Configuration example of one embodiment Hereinafter, a generating device 1 (see FIG. 6) that generates pseudo out-of-distribution data will be described as an example.

[0039] [1-2-1] Hardware configuration example The generating device 1 according to an embodiment may be various servers, for example, a virtual server (VM; Virtual Machine) or a physical server. The functions of the generating device 1 may be realized by one computer or by two or more computers. Furthermore, at least a part of the functions of the generating device 1 may be realized using HW (Hardware) resources and NW (Network) resources provided by a cloud environment.

[0040] 5 is a block diagram showing an example of a hardware (HW) configuration of a computer 10 that realizes the functions of the generating device 1 according to an embodiment. When multiple computers are used as HW resources that realize the functions of the generating device 1, each computer may have the HW configuration shown in FIG.

[0041] As shown in FIG. 5, the computer 10 may include, as a HW configuration, a processor 10a, a memory 10b, a storage unit 10c, an IF (Interface) unit 10d, an IO (Input / Output) unit 10e, and a reading unit 10f, for example.

[0042] The processor 10a is an example of a processing unit that performs various controls and calculations. The processor 10a may be connected to each block in the computer 10 via a bus 10i so that they can communicate with each other. The processor 10a may be a multiprocessor including a plurality of processors, a multicore processor having a plurality of processor cores, or a configuration having a plurality of multicore processors.

[0043] Examples of the processor 10a include integrated circuits (ICs) such as a CPU, MPU, GPU, APU, DSP, ASIC, and FPGA. Note that a combination of two or more of these integrated circuits may be used as the processor 10a. CPU is an abbreviation for Central Processing Unit, and MPU is an abbreviation for Micro Processing Unit. GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0044] For example, when the generating device 1 executes a machine learning process in addition to the generating process according to an embodiment, the processor 10a may be a combination of a processing device such as a CPU that executes the generating process and an accelerator that executes the machine learning process. Examples of the accelerator include the above-mentioned GPU, APU, DSP, ASIC, or FPGA.

[0045] The memory 10b is an example of HW that stores various data, programs, and other information. Examples of the memory 10b include one or both of a volatile memory such as a dynamic random access memory (DRAM) and a non-volatile memory such as a persistent memory (PM).

[0046] The storage unit 10c is an example of HW that stores various data, programs, and other information. Examples of the storage unit 10c include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of the nonvolatile memory include a flash memory, a storage class memory (SCM), and a read only memory (ROM).

[0047] The storage unit 10c may store a program 10g (generation program) that realizes all or part of the various functions of the computer 10.

[0048] For example, the processor 10a of the generating device 1 can realize the function of the generating device 1 (control unit 19 illustrated in FIG. 6) described later by expanding the program 10g stored in the storage unit 10c into the memory 10b and executing it.

[0049] The IF unit 10d is an example of a communication IF that controls connection and communication between various networks including a network between the generating device 1 and a device not shown in the figure. Examples of the device include a computer such as a user terminal or a server that provides data to the generating device 1, and a computer such as a server that uses data output from the generating device 1 (for example, performs machine learning processing based on the data).

[0050] For example, the IF unit 10d may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may be compatible with one or both of wireless and wired communication methods.

[0051] The program 10g may be downloaded to the computer 10 from a network via the communication IF and stored in the storage unit 10c.

[0052] The IO unit 10e may include one or both of an input device and an output device. Examples of the input device include a keyboard, a mouse, and a touch panel. Examples of the output device include a monitor, a projector, and a printer. The IO unit 10e may also include a touch panel in which the input device and the display device are integrated.

[0053] The reading unit 10f is an example of a reader that reads out data or program information recorded on the recording medium 10h. The reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reading unit 10f include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. The recording medium 10h may store a program 10g, and the reading unit 10f may read the program 10g from the recording medium 10h and store it in the storage unit 10c.

[0054] Examples of the recording medium 10h include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of the magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of the flash memory include semiconductor memories such as USB memories and SD cards.

[0055] The above-described HW configuration of the computer 10 is an example. Therefore, the HW in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, or integrated in any combination, or buses may be added or deleted, etc.

[0056] [1-2-2] Example of functional configuration 6 is a block diagram showing an example of a functional configuration of a generating device 1 according to an embodiment. The generating device 1 is an example of an information processing device or a computer that generates (pseudo) out-of-distribution data.

[0057] For example, the generating device 1 may employ various methods for acquiring an index indicating the likelihood of a plurality of data being out of distribution to generate out-of-distribution data to be used for training a machine learning model. In one embodiment, the above-mentioned iF (Isolation Forest) may be used as an example of a method for acquiring an index indicating the likelihood of a plurality of data being out of distribution.

[0058] 6, the generating device 1 may exemplarily include a memory unit 11, an acquiring unit 12, a statistical information generating unit 13, a parameter determining unit 14, a sample generating unit 15, a validity determining unit 16, and an output unit 17. The generating device 1 may also include a machine learning unit 18a, and may further include an inference processing unit 18b. The acquiring unit 12, the statistical information generating unit 13, the parameter determining unit 14, the sample generating unit 15, the validity determining unit 16, and the output unit 17 (as well as the machine learning unit 18a and the inference processing unit 18b) are an example of a control unit 19.

[0059] The memory unit 11 is an example of a storage area, and stores various data used by the generation device 1. The memory unit 11 may be realized, for example, by a storage area included in one or both of the memory 10b and the storage unit 10c illustrated in FIG.

[0060] 6, the memory unit 11 may be capable of storing, for example, a plurality of data 11a, a plurality of tree models 11b, statistical information 11c, parameters 11d, and out-of-distribution data 11e. In addition, when the generating device 1 includes a machine learning unit 18a, the memory unit 11 may be capable of storing a machine learning model 11f. In addition, when the generating device 1 includes an inference processing unit 18b, the memory unit 11 may be capable of storing an inference result 11g. The memory unit 11 may store these pieces of information in various formats, such as a DB (Database), a table, or an array.

[0061] The acquiring unit 12 acquires various information used by the generating device 1. For example, the acquiring unit 12 may receive a request to generate out-of-distribution data from a user terminal or the like.

[0062] Furthermore, the acquisition unit 12 may acquire an iF including a plurality of data 11a and a plurality of tree models 11b from a data providing device (not shown), and store the iF in the memory unit 11. The plurality of data 11a may be included in the generation request.

[0063] The generation request may include a request to generate out-of-distribution data of the plurality of data 11a. The generation request may also specify, for example, the number (predetermined number) of out-of-distribution data to be generated.

[0064] The data 11a is data including values ​​of multiple attributes, and is, for example, an example of training data used for training a machine learning model. Each of the multiple attributes is, for example, a variable x1, x2, . . . , x m (m is an integer equal to or greater than 2). In one embodiment, the plurality of data 11a does not include data outside the distribution.

[0065] The tree model 11b is a trained machine learning model for generating a tree, for example, an iT (Isolation Tree). The tree model 11b generates a tree by, for example, selecting variables and a threshold value, and repeating a process of dividing a plurality of input data (nodes) until division of all input data is completed (reach a leaf node).

[0066] In one embodiment, each of the multiple tree models 11b is trained to select a variable to be compared with a threshold from among two variables x1 and x2 as a variable for identifying an attribute included in the data 11a, while the multiple tree models 11b are trained to select a variable to be compared with a threshold from among two variables x1 and x2 as a variable for identifying an attribute included in the data 11a.

[0067] The statistical information generating unit 13 generates statistical information 11c of an index indicating the likelihood of the data 11a being outside the distribution in a predetermined tree in which a plurality of data 11a are divided into individual data 11a (nodes), and stores the information in the memory unit 11. As an index indicating the likelihood of the data 11a being outside the distribution, for example, pl (Path Length) may be used. The predetermined tree is an example of a first tree.

[0068] A first example of the statistical information 11c is an average value c(n) (n is an integer equal to or greater than 1) of pI in a predetermined tree. For example, the statistical information generating unit 13 may calculate the average value c(n) of pI using each pI of the data 11a (nodes) included in the predetermined tree according to the following formula (1). c(n) = 2H(n-1) - (2(n-1) / n) (1)

[0069] In the above formula (1), n ​​is the number of unique samples, e.g., the total number of data 11a included in a given tree. H(i) (i is an argument, i=n-1) is the sum of the natural logarithm ln(i) and Euler's constant (e.g., 0.5772156649).

[0070] A second example of the statistical information 11c is a histogram of the distribution density of pl in a given tree.

[0071] Fig. 7 is a diagram showing an example of a histogram of the distribution density of pl. As shown in Fig. 7, the statistical information generating unit 13 may calculate (generate) a histogram by tallying up the occurrence frequency of the pl value of a plurality of data 11a included in a predetermined tree. Examples of the occurrence frequency of the pl value include the number of data 11a having a common pl value, or the ratio of data 11a having a common pl value to the total number of data 11a.

[0072] As a modified example of the second example of the statistical information 11c, the statistical information generating unit 13 may use the average value of the pl values ​​(average value at iF) of each of the data 11a in the multiple trees, instead of the pl values ​​of the multiple data 11a included in a specific tree. In this case, the statistical information generating unit 13 may calculate a histogram using the average value of pl calculated for each of the data 11a. For example, the above formula (1) may be used as a method for calculating the average value of pl for each of the data 11a.

[0073] The parameter determination unit 14 determines a parameter 11d based on the statistical information 11c, and stores the parameter 11d in the memory unit 11. An example of the parameter 11d is the depth d (first distance) of a node included in the tree.

[0074] For example, the parameter determination unit 14 may determine, as the depth d, one of the pl of the multiple data 11a included in a predetermined tree, based on the statistical information 11c.

[0075] As a first example, when the statistical information 11c is an average value c(n), the parameter determining section 14 may calculate the depth d according to the following formula (2). d = int(αc(n)) (2)

[0076] In the above formula (2), α is a coefficient between "0" and "1". For example, α may be "0.5". Also, int() is a function that outputs the maximum integer that does not exceed the argument.

[0077] Thus, in the first example, the parameter determining unit 14 derives an appropriate depth d by using an index that is easy for a user to understand, that is, α times the average numerical value of pl (rounded down to an integer).

[0078] As a second example, when the statistical information 11c is a histogram, the parameter determining section 14 may calculate the depth d by using the cumulative point of the density of the histogram.

[0079] 8 is a diagram showing an example of derivation of the depth d by estimating the distribution density of pl. As shown in Fig. 8, the parameter determination unit 14 may select the value of pl at the cumulative β% point in the histogram of the distribution density of pl (see Fig. 7) as the depth d. β is between "0" and "100". fixed As an example, β may be set to "20".

[0080] The value of pl at the cumulative β% point may mean, for example, the value of pl when the cumulative value of the occurrence frequency of pl reaches β%. In the example of Fig. 8, the parameter determination unit 14 accumulates the occurrence frequency of pl in ascending order of pl value, and selects "3", which is the value of pl when the cumulative value reaches β%, as the depth d. Note that the parameter determination unit 14 may accumulate the occurrence frequency of pl in descending order of pl value, and select the value of pl when the cumulative value reaches β%.

[0081] Thus, in the second example, the parameter determining unit 14 derives an appropriate depth d by using an index that is easy for a user to understand, that is, selection of pl at the cumulative β% point by estimating the distribution density of pl.

[0082] Based on the parameters 11d determined by the parameter determination unit 14, the sample generation unit 15 generates samples in the domain of the node according to the parameters 11d.

[0083] For example, the sample generating unit 15 may select a node according to the depth d indicated by the parameter 11d from a predetermined (first) tree, and identify the domain of the selected node. The domain may be, for example, the range of variables of a node (node ​​located at depth d) whose value of pl matches the depth d. As an example, the domain of the selected node is a range determined by the lower and upper limits of the variable x1 of the node, and the lower and upper limits of the variable x2. The lower and upper limits of each variable can be identified by a threshold value selected when dividing the multiple data 11a by the tree model 11b.

[0084] Then, the sample generating unit 15 generates out-of-distribution data (first data) as samples uniformly and randomly within the range of the domain identified from the node. As an example, the sample generating unit 15 may generate, as out-of-distribution data, a combination of the values ​​of the variables x1 and x2, where the variable x1 is equal to or greater than the lower limit and equal to or less than the upper limit, and the variable x2 is equal to or greater than the lower limit and equal to or less than the upper limit.

[0085] The validity determination unit 16 determines whether the out-of-distribution data generated by the sample generation unit 15 using a predetermined tree is valid out-of-distribution data.

[0086] For example, the validity determination unit 16 may evaluate the validity of the out-of-distribution data based on one or more (second) trees among the multiple trees included in iF. The one or more trees are trees other than the predetermined (first) tree used to generate the out-of-distribution data. Hereinafter, the first tree may be referred to as a "data generation tree," and each of the one or more second trees may be referred to as a "data evaluation tree."

[0087] FIG. 9 is a diagram showing an example of out-of-distribution data generated using a data generation tree, and FIG. 10 is a diagram showing an example of out-of-distribution data in a data evaluation tree.

[0088] Hereinafter, it is assumed that the sample generation unit 15 generates out-of-distribution data 143 at a node 142 corresponding to a depth d in a data generation tree including multiple leaf nodes separated by a threshold 141 and an axis, as illustrated in FIG. 9.

[0089] The validity determining unit 16 determines, for example, at what depth (pl; second distance) in the data evaluation tree the value of the out-of-distribution data 143 (the values ​​of variables x1 and x2) corresponds to a leaf node.

[0090] As an example, the validity determining unit 16 may compare the average value of pI of the leaf node to which the out-of-distribution data 143 belongs in the data evaluation tree with the depth d of the parameter 11d.

[0091] 10, the data evaluation tree includes a number of leaf nodes separated by a threshold 151 and an axis. Due to the randomness of the division of the data 11a in each tree, the domain of each leaf node of the data evaluation tree is different from the domain of each leaf node of the data generation tree.

[0092] Therefore, when out-of-distribution data 143 is mapped to the data evaluation tree, the out-of-distribution data 143 belongs to node 152 in a domain different from node 142 on the data generation tree.

[0093] For example, the validity determination unit 16 obtains the pl of the node 152 to which the out-of-distribution data 143 belongs by comparing the value of the out-of-distribution data 143 with the domain of each node included in the data evaluation tree. For example, the validity determination unit 16 may obtain the pl of the node 152 to which the out-of-distribution data 143 belongs for each of one or more data evaluation trees.

[0094] Then, the validity determination unit 16, for example, calculates an average value of one or more pls acquired from one or more data evaluation trees, and compares the calculated average value with the depth d. For example, the validity determination unit 16 may determine whether or not the difference between the calculated average value and the depth d is equal to or greater than a threshold value γ. Note that the calculation method for the average value of one or more pls may use, for example, the above formula (1). The threshold value γ is an integer equal to or greater than "1", and may be, for example, a value such as "3".

[0095] When the difference between the calculated average value and the depth d is equal to or greater than a threshold value γ, the out-of-distribution data 143 generated from the data generation tree based on the depth d can be said to be data at an inappropriate depth in other trees. For example, when the depth d is greater than the average value by γ or more, the out-of-distribution data 143 is highly likely to be in-distribution in one or more data evaluation trees. Also, when the depth d is smaller than the average value by γ or more, the out-of-distribution data 143 is out-of-distribution in one or more data evaluation trees, but it is clear that it is out-of-distribution, and is an inappropriate sample for training a machine learning model.

[0096] Therefore, when the difference between the calculated average value and the depth d is equal to or greater than the threshold value γ, the validity determination unit 16 may determine that the out-of-distribution data generated by the sample generation unit 15 is inappropriate (not valid). On the other hand, when the difference between the calculated average value and the depth d is less than the threshold value γ, the validity determination unit 16 may determine that the out-of-distribution data generated by the sample generation unit 15 is appropriate (valid) out-of-distribution data, and may store it in the memory unit 11 as out-of-distribution data 11e.

[0097] When it is determined that the out-of-distribution data is inappropriate data, the validity determination unit 16 may, for example, instruct the sample generation unit 15 to generate new (other) out-of-distribution data. In response to the generation instruction from the validity determination unit 16, the sample generation unit 15 may identify a node located at depth d in the data generation tree, and generate uniformly random out-of-distribution data (second data) in the domain of the node. The validity determination unit 16 may evaluate the validity described above every time out-of-distribution data is generated by the sample generation unit 15.

[0098] If there are multiple nodes located at the depth d indicated by parameter 11d in the data generation tree, the sample generation unit 15 may select a node that generates the second data in the domain other than the node that generated the first data in the domain.

[0099] If there are multiple nodes located at the depth d indicated by the parameter 11d in the data generation tree, the sample generation unit 15 may generate out-of-distribution data in the domains of each of the multiple nodes. In this case, the validity determination unit 16 may evaluate the validity of each of the multiple out-of-distribution data generated by the sample generation unit 15.

[0100] The output unit 17 outputs output data. An example of the output data is out-of-distribution data 11e that is determined to be appropriate (valid) by the validity determination unit 16. The output data may include a plurality of data 11a in addition to the out-of-distribution data 11e. Furthermore, the output data may include one or both of a machine learning model 11f and an inference result 11g, which will be described later.

[0101] In "outputting" the output data, the output unit 17 may, for example, transmit (provide) the output data to another computer (not shown), or may store the output data in the memory unit 11 and manage it so that it can be acquired from the generating device 1 or another computer. Alternatively, in "outputting" the output data, the output unit 17 may output information indicating the output data to a screen of an output device such as the generating device 1, or may output the output data in various other modes.

[0102] As described above, the generation device 1 may include the machine learning unit 18a, and may further include the inference processing unit 18b.

[0103] In the machine learning phase, the machine learning unit 18a executes a machine learning process to train the machine learning model 11f based on training data including the plurality of data 11a and the out-of-distribution data 11e. The machine learning model 11f may be a Neural Network (NN) model including machine-learned parameters. The machine learning process may be realized by various known methods. For example, the machine learning unit 18a can make the machine learning model 11f learn that the out-of-distribution data 11e is "out-of-distribution data" by using the pseudo-generated out-of-distribution data 11e for training the machine learning model 11f.

[0104] In the inference phase, the inference processing unit 18b performs inference processing using the machine learning model 11f trained based on the out-of-distribution data 11e. For example, the inference processing unit 18b inputs target data (not shown) for the inference processing to the machine learning model 11f, and stores the inference result 11g output from the machine learning model 11f in the memory unit 11.

[0105] [1-3] Example of operation Next, an operation example of the generating device 1 according to an embodiment will be described. Fig. 11 is a flowchart illustrating an operation example of the generating device 1 according to an embodiment. It is assumed that the generating device 1 holds trained iF (multiple tree models 11b) in the memory unit 11 in advance.

[0106] 11, the acquisition unit 12 of the generating device 1 receives a request to generate the out-of-distribution data 11e. The acquisition unit 12 also acquires a plurality of data 11a (step S1).

[0107] The statistical information generating unit 13 inputs a data set of a plurality of data 11a into each of a plurality of tree models 11b, and generates a plurality of trees (iT) (step S2).

[0108] The statistical information generating unit 13 selects one of the generated trees as a data generation tree, and generates statistical information 11c based on the data generation tree (step S3).

[0109] The parameter determination unit 14 calculates the depth d of the node in the data generation tree as a parameter 11d based on the statistical information 11c (step S4).

[0110] The sample generating unit 15 uniformly randomly generates samples (out-of-distribution data 11e) in the domain of the node at depth d in the data generation tree (step S5).

[0111] The validity judgment unit 16 obtains the depth pl of the node to which the sample generated in step S5 belongs in each of one or more data evaluation trees other than the data generation tree among the multiple trees, and calculates the average value of the depth pl between the trees (step S6).

[0112] The validity determining unit 16 determines whether or not the absolute value of the difference between the depth d and the calculated average value of pl is equal to or greater than a threshold value γ (step S7).

[0113] If the absolute value of the difference is equal to or greater than the threshold value γ (YES in step S7), the validity determination unit 16 discards the sample generated in step S5 (step S8), and the process proceeds to step S5. In this case, in step S5, the sample generation unit 15 uniformly randomly generates a sample (out-of-distribution data 11e) in the domain of the node at depth d in the data generation tree. If there are multiple nodes at depth d in the data generation tree, the sample generation unit 15 may select a node different from the node for which a sample was previously generated.

[0114] In step S7, if the absolute value of the difference is less than the threshold value γ (NO in step S7), the validity determining unit 16 stores the sample generated in step S5 in the memory unit 11 as out-of-distribution data 11e (step S9).

[0115] Then, the validity determination unit 16 determines whether the number of out-of-distribution data 11e reaches a predetermined number, for example, the number of data to be generated designated in the generation request (step S10). If the number of out-of-distribution data 11e does not reach the predetermined number (NO in step S10), the process proceeds to step S5.

[0116] When the number of out-of-distribution data 11e reaches the predetermined number (YES in step S10), the process proceeds to step S11.

[0117] In step S11, the output unit 17 outputs the generated out-of-distribution data 11e, and the process ends.

[0118] [1-4] Effects of one embodiment In the generating device 1 according to an embodiment, the control unit 19 generates a sample (out-of-distribution data) in the domain of any one of a plurality of leaf nodes based on the data generation tree. The control unit 19 also determines the validity of the sample being out-of-distribution data of the plurality of data 11a based on the pl (depth d) of the sample in the data generation tree and one or more pl of the node including the sample in its domain in each of one or more data evaluation trees.

[0119] This allows the generating device 1 to ensure that a sample generated at depth d in the data generation tree is likely to be data at depth d in the data evaluation tree. This improves the efficiency of generating out-of-distribution data by the generating device 1.

[0120] Furthermore, in the generating device 1 according to an embodiment, the control unit 19 selects the depth d from among the distances based on the statistical information 11c of the distances (p1) from the root node to the leaf nodes of the data generation tree.

[0121] This allows the generating device 1 to derive an appropriate depth d of the node used to generate the out-of-distribution data 11e, thereby improving the efficiency with which the generating device 1 generates the out-of-distribution data.

[0122] Furthermore, in one embodiment, iF may be used to generate the out-of-distribution data 11e. iF is applicable to various types of tabular data. In addition, the tree-based algorithm does not require data pre-processing, such as encoding and normalization of nominal variables. Therefore, the type of data to which the out-of-distribution data 11e generation method can be applied can be expanded compared to other density estimation methods such as kernel density estimation.

[0123] [2] Other The technology according to the embodiment described above can be modified and changed as follows.

[0124] For example, the acquisition unit 12, statistical information generation unit 13, parameter determination unit 14, sample generation unit 15, validity judgment unit 16 and output unit 17 (as well as the machine learning unit 18a and inference processing unit 18b) provided in the generation device 1 shown in Figure 6 may be merged in any combination or may be divided separately.

[0125] 6 may be a configuration (system) in which a plurality of devices cooperate with each other via a network to realize each processing function. As an example, the memory unit 11 may be a DB server, the acquisition unit 12 and the output unit 17 may be a Web server or an application server, and the statistical information generation unit 13, the parameter determination unit 14, the sample generation unit 15, the validity determination unit 16, the machine learning unit 18a, and the inference processing unit 18b may be an application server, etc. In this case, the DB server, the application server, and the web server may cooperate with each other via a network to realize each processing function of the generation device 1. [Explanation of symbols]

[0126] 1 generator 10. Computers 11 Memory section 11a Data 11b Tree model 11c Statistics 11d Parameters 11e Out-of-distribution data 11f Machine learning models 11g inference results 12 Acquisition Department 13 Statistical information generation section 14 Parameter determination section 15 Sample Generation Section 16 Validity judgment part 17 Output section 18a Machine Learning Department 18b Inference processing unit 19 Control section

Claims

1. A method for generating a first data item in a domain of one of a plurality of leaf nodes based on a first tree among a plurality of trees that hierarchically classify a plurality of data items belonging to a root node into nodes lower than the root node by a random threshold value, the first data item being generated in a domain of one of the plurality of leaf nodes based on a first tree among the plurality of trees in which the number of data items classified into a leaf node among the plurality of nodes is equal to or less than a reference value; determining whether the first data is outside the distribution of the plurality of data based on a first distance from the root node to a leaf node from which the first data was generated, and one or more second distances from a root node of one or more second trees different from the first tree among the plurality of trees to a node of the second tree that includes the first data in its domain; A generating method in which the processing is performed by a computer.

2. The process of determining the validity includes a process of determining that it is valid for the first data to be data outside the distribution when a difference between the first distance and an average value of the one or more second distances is less than a threshold, and determining that it is not valid for the first data to be data outside the distribution when the difference is equal to or greater than a threshold. The method of claim 1 .

3. When it is determined in the process of determining the validity that the first data is not valid because it is data outside the distribution, second data different from the first data is generated in a domain of a leaf node according to the first distance of the first tree; determining whether the second data is outside the distribution of the plurality of data; The method according to claim 1 or 2, wherein the processing is carried out by the computer.

4. selecting the first distance from the plurality of distances based on statistical information of a plurality of distances from a root node of the first tree to the plurality of leaf nodes of the first tree; The generating method according to any one of claims 1 to 3, wherein the processing is executed by the computer.

5. the statistical information being an average value of the plurality of distances in the first tree; The process of selecting the first distance includes a process of calculating the first distance based on an average value of the plurality of distances. The method of claim 4.

6. the statistical information is a histogram of distribution density of the plurality of distances in the first tree; the process of selecting the first distance includes a process of determining, as the first distance, a distance corresponding to an appearance frequency when a cumulative value of an appearance frequency of the distance in the histogram reaches a predetermined value. The method of claim 4.

7. A method for generating a first data item in a domain of one of a plurality of leaf nodes based on a first tree among a plurality of trees that hierarchically classify a plurality of data items belonging to a root node into nodes lower than the root node by a random threshold value, the first data item being generated in a domain of one of the plurality of leaf nodes based on a first tree among the plurality of trees in which the number of data items classified into a leaf node among the plurality of nodes is equal to or less than a reference value; determining whether the first data is outside the distribution of the plurality of data based on a first distance from the root node to a leaf node from which the first data was generated, and one or more second distances from a root node of one or more second trees different from the first tree among the plurality of trees to a node of the second tree that includes the first data in its domain; A generating program that causes a computer to execute processing.

8. A method for generating a first data item in a domain of one of a plurality of leaf nodes based on a first tree among a plurality of trees that hierarchically classify a plurality of data items belonging to a root node into nodes lower than the root node by a random threshold value, the first data item being generated in a domain of one of the plurality of leaf nodes based on a first tree among the plurality of trees in which the number of data items classified into a leaf node among the plurality of nodes is equal to or less than a reference value; determining whether the first data is outside the distribution of the plurality of data based on a first distance from the root node to a leaf node from which the first data was generated, and one or more second distances from a root node of one or more second trees different from the first tree among the plurality of trees to a node of the second tree that includes the first data in its domain; An information processing device comprising a control unit.

Images