Processing unit

Abstract

To overcome a situation in which it is difficult to execute a process for an appropriate risk evaluation.SOLUTION: A processing device 700 includes: an obtaining unit 721 that obtains, from a decision tree which is an already-learnt model and which includes multiple nodes, score information representing a value in accordance with the number of data falling in the node among multiple pieces of training data utilized for training of the decision tree; and an identifying unit 722 that identifies a range that a value of an unknown attribute which is a part of multiple attributes contained in the training data can take on the basis of the score information obtained by the obtaining unit 721.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] This invention relates to a processing apparatus, a processing method, and a program. [Background technology]

[0002] Techniques are known for assessing risks such as the risk of information leakage from machine learning models, including the risk of training data being estimated from the learning model.

[0003] For example, Patent Document 1 describes a system including a processor and a memory device. According to Patent Document 1, the memory device has statistical data of the training data of a first learning model and evaluation data for evaluating the response of the learning model to the first learning model. The processor generates pseudo-data consisting of the same target variable and explanatory variables as the training data based on the statistical data, and trains a second learning model using the pseudo-data. Subsequently, the processor compares the response result of the first learning model to the evaluation data with the response result of the second learning model to the evaluation data, and evaluates the risk of information leakage from the first learning model based on the comparison result.

[0004] Furthermore, as a related technology, for example, there is Non-Patent Document 1. Non-Patent Document 1 describes a technique used to estimate the value of an unknown attribute, which is an explanatory variable to be inferred. For example, according to Non-Patent Document 1, the unknown attribute is fixed to a certain value, the proportion of the decision tree training data that is assigned to the same partition region as the target data is calculated, and the marginal probability is evaluated using the calculated proportion as a weight to estimate the most likely attribute value. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-007311 [Non-patent literature]

[0006] [Non-Patent Document 1] Matthew Fredrikson et al., Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures, October 2015 [Summary of the Invention] [Problems to be Solved by the Invention]

[0007] In the case of the technology described in Patent Document 1, in order to perform risk assessment, statistical data of the training data used when training the first learning model is required. Therefore, it is impossible to perform an assessment when there is no statistical data. Also, in the case of the technology described in Non-Patent Document 1, since probabilistic attribute estimation is performed in the average case, there is a risk that appropriate evaluation cannot be performed depending on the output status of the decision tree. For example, as described above, there has been a problem that it may be difficult to perform processing for appropriate risk assessment.

[0008] Therefore, an object of the present invention is to provide a processing device, a processing method, and a program that solve the above-described problems. [Means for Solving the Problems]

[0009] A processing device according to one embodiment of the present disclosure for achieving such an object an acquisition unit that acquires score information indicating a value corresponding to the number of data that has fallen to a node among a plurality of training data used during training of a decision tree, which is a learned model composed of a plurality of nodes; a specifying unit that specifies a range in which values of unknown attributes, which are a part of a plurality of attributes included in the training data, can be taken based on the score information acquired by the acquisition unit; has such a configuration.

[0010] In another aspect of the present disclosure, a processing method is such that an information processing apparatus obtains score information indicating a value corresponding to the number of data items that have fallen to a node among a plurality of training data items used during training of a decision tree, which is a learned model and consists of a plurality of nodes, and based on the obtained score information, specifies a range within which a value of an unknown attribute, which is a part of a plurality of attributes included in the training data, can be taken. It has such a configuration.

[0011] In another aspect of the present disclosure, a program causes an information processing apparatus to obtain score information indicating a value corresponding to the number of data items that have fallen to a node among a plurality of training data items used during training of a decision tree, which is a learned model and consists of a plurality of nodes, and based on the obtained score information, specify a range within which a value of an unknown attribute, which is a part of a plurality of attributes included in the training data, can be taken. It is a program for realizing the processing.

Advantages of the Invention

[0012] According to each of the configurations described above, the above-described problems can be solved.

Brief Description of the Drawings

[0013] [Figure 1] It is a diagram showing a configuration example of a risk assessment system in a first embodiment of the present disclosure. [Figure 2] It is a block diagram showing a configuration example of a model storage device. [Figure 3] It is a diagram showing an example of a decision tree. [Figure 4] It is a block diagram showing a configuration example of a risk assessment device. [Figure 5] It is a diagram showing an example of prior information. [Figure 6] It is a diagram for explaining an operation example of a specifying unit. [Figure 7]This flowchart shows an example of how a risk assessment device works. [Figure 8] This flowchart shows a detailed example of the operation in step S104. [Figure 9] This figure shows another example of prior information. [Figure 10] This block diagram shows another example configuration of a risk assessment device. [Figure 11] This figure shows an example configuration of a risk assessment system in a second embodiment of the present disclosure. [Figure 12] This block diagram shows an example configuration of a model storage device. [Figure 13] This is a block diagram showing an example configuration of a risk assessment device. [Figure 14] This is a diagram illustrating an example of processing in a specific area. [Figure 15] This flowchart shows an example of how a risk assessment device works. [Figure 16] This block diagram shows an example of the configuration of the processing apparatus in the third embodiment of this disclosure. [Figure 17] This is a block diagram showing an example of the configuration of a processing unit. [Modes for carrying out the invention]

[0014] [First Embodiment] A first embodiment of this disclosure will be described with reference to Figures 1 to 10. Figure 1 is a diagram showing an example configuration of the risk assessment system 100. Figure 2 is a block diagram showing an example configuration of the model storage device 200. Figure 3 is a diagram showing an example of a decision tree. Figure 4 is a block diagram showing an example configuration of the risk assessment device 300. Figure 5 is a diagram showing an example of prior information 341. Figure 6 is a diagram illustrating an example of the operation of the identification unit 354. Figure 7 is a flowchart showing an example of the operation of the risk assessment device 300. Figure 8 is a flowchart showing a detailed example of the operation of step S104. Figure 9 is a diagram showing another example of prior information 341. Figure 10 is a block diagram showing another example configuration of the risk assessment device 300.

[0015] In a first embodiment of this disclosure, a risk assessment system 100 is described that can perform risk assessment by identifying the range of possible values ​​for unknown attributes when some of the attributes constituting the training data used to train a decision tree 241, which is a learning model, are unknown. For example, the risk assessment system 100 obtains the confidence score output by the decision tree 241 as score information that indicates a value corresponding to the number of training data points that have fallen into the nodes constituting the decision tree 241. Then, based on the obtained confidence score, the risk assessment system 100 identifies the range of possible values ​​for the unknown attribute.

[0016] For example, in this embodiment, the risk assessment system 100 uses the attributes (x1, x) that constitute the training data. 2、 ..., x d ) some of the attribute values ​​(x 2、 ..., x d ) is known, and the unknown attribute x1 has k values ​​(v 11 ...v 1k It is known that the unknown attribute x1 can take any of the following. In such a case, for example, the risk assessment system 100 determines that the unknown attribute x1 is (v 11 , ……, v 1k Assuming that the value can take one of the following values, candidate data corresponding to each value is created. The risk assessment system 100 then inputs each of the created candidate data into the decision tree 241, which is a learning model, and obtains a confidence score, which is the output from the decision tree 241 corresponding to the input.

[0017] The confidence score, for example, refers to the information that decision tree 241 outputs in response to inputs such as candidate data. For instance, the confidence score indicates the proportion of data points for each label that fell into a leaf node, corresponding to the leaf node identified in response to the input, within the training data.

[0018] Figure 1 shows an example configuration of the risk assessment system 100 in this embodiment. Referring to Figure 1, the risk assessment system 100 includes, for example, a risk assessment device 300 and a model storage device 200. As shown in Figure 1, the risk assessment device 300 and the model storage device 200 are connected to each other so that they can communicate with each other, for example, via a network.

[0019] The model storage device 200 is an information processing device that stores a decision tree 241, which is a learning model trained using training data. Figure 2 shows an example configuration of the model storage device 200. For example, referring to Figure 2, the model storage device 200 has a storage unit 240 in which the decision tree 241 is stored, as well as a receiving unit 210, an inference unit 220, and an output unit 230. For example, the model storage device 200 has a computing device such as a CPU (Central Processing Unit) and a memory device. The model storage device 200 can realize each of the above processing units by having the computing device execute a program stored in the memory device. Note that the model storage device 200 may have a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof instead of the CPU mentioned above.

[0020] As shown in Figure 2, the memory unit 240 stores a decision tree 241 that has been pre-trained using multiple training data sets containing multiple attributes and labels. The decision tree 241 may be trained within the model storage device 200 or outside the model storage device 200. In this embodiment, the labels are, for example, categorical variables that take discrete values.

[0021] Figure 3 shows an example of the decision tree 241. As shown in Figure 3, the decision tree 241 is composed of a plurality of nodes 241-1, 241-2, 241-3, 241-4, 241-5, 241-6, 241-7, 241-8, 241-9. Here, among the nodes constituting the decision tree 241, the nodes existing at the ends such as nodes 241-2, 241-6, 241-7, 241-8, 241-9 are called leaf nodes. In the decision tree 241, depending on the value of the attribute of the input data, the input of the target will fall into one of the plurality of leaf nodes. Also, among the nodes constituting the decision tree 241, the node 241-1 that refers to the entire first data is called the root node.

[0022] For example, as illustrated in Figure 3, the nodes 241-1, 241-3, 241-4, 241-5 other than the leaf nodes constituting the decision tree 241 have branching conditions used when distributing the input data such as candidate data. For example, the branching condition can indicate a condition such as whether the value of a certain attribute is greater than or equal to a predetermined value. The branching condition is adjusted during learning using the training data. Also, each node constituting the decision tree 241 has a score value indicating the ratio of the number of data for each label distributed to that node among the training data. For example, in the case illustrated in Figure 3, the node 241-4 has a score value of [0, 33, 3]. This indicates that during the training of the decision tree 241, 0 training data with label 1, 33 training data with label 2, and 3 training data with label 3 were respectively distributed to the node 241-4. For example, the decision tree 241 can output the score value of the leaf node where the input data such as candidate data has fallen as a confidence score. Note that each node constituting the decision tree 241 may have information other than that illustrated in Figure 3.

[0023] The receiving unit 210 receives candidate data from the risk assessment device 300. For example, the receiving unit 210 is "v 11 , x 2、 …, x d " or "v 12 , x 2、 …, xd The receiving unit 210 receives candidate data that includes values ​​of attributes known to the risk assessment device 300, as well as candidate attributes that are unknown to the device. For example, the receiving unit 210 receives a number of candidate data from the risk assessment device 300 corresponding to the number of candidate attributes that are unknown to the risk assessment device 300. The receiving unit 210 may also receive information other than those exemplified above, such as identification information, along with the candidate data.

[0024] The inference unit 220 inputs each candidate data received by the receiving unit 210 into the decision tree 241, which is a learning model. As a result of this input, the inference unit 220 obtains a confidence score, which is the inference result corresponding to each candidate data. In other words, the inference unit 220 obtains the score value of the leaf node corresponding to the candidate data as the confidence score by inputting the candidate data into the decision tree 241.

[0025] The output unit 230 transmits the confidence score acquired by the inference unit 220 to the risk assessment device 300. For example, the output unit 230 may transmit the confidence score to the risk assessment device 300 along with candidate data identification information, etc., so that it can determine which candidate data the confidence score was inferred from.

[0026] For example, as described above, the model storage device 200 has a decision tree 241, which is a learned model trained using training data. When the model storage device 200 receives candidate data from the risk assessment device 300, it performs inference using the decision tree 241 based on the received candidate data to obtain a confidence score corresponding to the candidate data. The model storage device 200 then transmits the obtained confidence score as score information to the risk assessment device 300.

[0027] The risk assessment device 300 is an information processing device that identifies the range of possible unknown attributes based on the confidence score, which is score information acquired from the model storage device 200. Furthermore, the risk assessment device 300 can perform risk assessments, such as the possibility of privacy leakage, based on the identified results.

[0028] Figure 4 shows an example configuration of the risk assessment device 300. Referring to Figure 4, the risk assessment device 300 has, for example, an operation input unit 310, a screen display unit 320, a communication I / F unit 330, a storage unit 340, and an arithmetic processing unit 350 as its main components.

[0029] Figure 4 illustrates an example where the functions of the risk assessment device 300 are realized using a single information processing device. However, the risk assessment device 300 may be realized using multiple information processing devices, for example, by being implemented on the cloud. For example, the functions of the risk assessment device 300 may be realized by two information processing devices: one processing device having the functions of a candidate data creation unit 351, a candidate data transmission unit 352, an inference result acquisition unit 353, and a identification unit 354, and an evaluation device having the functions of an evaluation unit 355 and an output unit 356. Furthermore, the risk assessment device 300 may not include some of the configurations exemplified above, such as not having an operation input unit or a screen display unit, and may have configurations other than those exemplified above.

[0030] The operation input unit 310 consists of an operation input device such as a keyboard or mouse. The operation input unit 310 detects the operation of the operator operating the risk assessment device 300 and outputs it to the calculation processing unit 350.

[0031] The screen display unit 320 consists of a screen display device such as an LCD (Liquid Crystal Display). The screen display unit 320 can display various information stored in the storage unit 340 on the screen in response to instructions from the arithmetic processing unit 350.

[0032] The communication interface unit 330 consists of data communication circuits and the like. The communication interface unit 330 performs data communication with external devices such as the model storage device 200, which are connected via a communication line.

[0033] The storage unit 340 is a storage device such as a hard disk or memory. The storage unit 340 stores processing information and programs 343 necessary for various processes in the arithmetic processing unit 350. The programs 343 are read into the arithmetic processing unit 350 and executed to realize various processing functions. The programs 343 are pre-read from external devices or recording media via data input / output functions such as the communication I / F unit 330 and stored in the storage unit 340. The main information stored in the storage unit 340 includes, for example, prior information 341 and inference result information 342.

[0034] The prior information 341 includes information already known about the training data used when training the decision tree 241 stored in the model storage device 200. For example, the prior information 341 is acquired in advance by methods such as obtaining it from an external device via the communication I / F unit 330 or inputting it using the operation input unit 310, and is stored in the storage unit 340.

[0035] Figure 5 shows an example of prior information 341. Referring to Figure 5, prior information 341 includes partial training data information and unknown attribute information. For example, as shown in Figure 5, prior information 341 may include multiple pieces of information that associate partial training data information and unknown attribute information.

[0036] Here, the partial training data information shows the values ​​of known attributes and their corresponding labels in the training data used to train decision tree 241, where some attributes are unknown. For example, in Figure 5, the attributes (x2, ..., x d This example illustrates the case where the label y and attribute x1 are known, but attribute x1 is unknown. Furthermore, the unknown attribute information indicates information about the value of the unknown attribute. For example, in Figure 5, the unknown attribute x1 has k values ​​(v 11 ...v 1kThis indicates that it takes one of the following options.

[0037] The inference result information 342 includes information indicating the confidence score, which is score information obtained from the model storage device 200. For example, the inference result information 342 may include information indicating the confidence score according to the number of candidates in the unknown attribute. For example, the inference result information 342 is generated and updated in response to the inference result acquisition unit 353, described later, acquiring the confidence score from the model storage device 200.

[0038] The arithmetic processing unit 350 includes an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 350 reads and executes a program 343 from the storage unit 340, thereby realizing various processing functions by cooperating with the hardware and the program 343. Examples of the main processing functions realized by the arithmetic processing unit 350 include a candidate data creation unit 351, a candidate data transmission unit 352, an inference result acquisition unit 353, a identification unit 354, an evaluation unit 355, and an output unit 356. As mentioned above, the arithmetic processing unit 350 may have a GPU or the like instead of a CPU.

[0039] The candidate data creation unit 351 creates candidate data based on the prior information 341. For example, the candidate data creation unit 351 creates candidate data corresponding to the number of candidates indicated by the unknown attribute information. The candidate data creation unit 351 may create candidate data at any time.

[0040] Specifically, for example, as prior information 341, partial training data information (x2, ..., x d ,y) is stored, and the unknown attribute x1 is (v 11 ...v 1k Assume that the value of the unknown attribute x1 is one of the values ​​of (v 11 ...v 1k (v 11 ...v 1k ) Creates candidate data corresponding to each. In other words, the candidate data creation unit 351 creates (v11 , x2, ..., x d ), ..., (v 1k , x2, ..., x d This creates candidate data.

[0041] The candidate data transmission unit 352 transmits the candidate data created by the candidate data creation unit 351 to the model storage device 200. The candidate data transmission unit 352 may also transmit candidate data identification information, such as partial training data information used during candidate data creation, along with the candidate data.

[0042] The inference result acquisition unit 353 receives and acquires a confidence score from the model storage device 200 as the result of inference based on candidate data. For example, the inference result acquisition unit 353 may acquire the confidence score from the model storage device 200 along with identification information, etc., so that the candidate data that was the subject of inference can be identified. The inference result acquisition unit 353 also stores the received confidence score as inference result information 342 in the storage unit 340. The inference result acquisition unit 353 may also store the confidence score in the storage unit 340 along with the identification information of the corresponding candidate data, etc.

[0043] The identification unit 354 identifies the range of possible unknown attributes based on the confidence score, which is score information. For example, the identification unit 354 identifies candidate unknown attribute x1 (v 11 ...v 1k By excluding candidate values ​​from the confidence score that are unlikely to have actually constituted the training data, or are deemed unlikely to have done so, the range of possible outcomes for the unknown attribute is identified.

[0044] For example, the identification unit 354 checks the value in the confidence score that corresponds to the label of the candidate data. Then, if the value corresponding to the label of the candidate data is below a predetermined threshold, the identification unit 354 excludes the candidate value corresponding to the candidate data from the unknown attribute candidates.

[0045] For example, as illustrated in Figure 6, a certain candidate data (v 11, x2, ..., x d Suppose the label of the candidate data is y1. Also, suppose the confidence score corresponding to the candidate data is [0, 32, 0], and the value corresponding to label y1 is 0. In this case, according to the confidence score, it can be determined that the number of training data with label 1 that fell into the corresponding leaf node during the training of the decision tree 241 was 0. Therefore, the identification unit 354 determines, based on the confidence score, that there is no possibility that the above candidate data actually constituted training data, and excludes the candidate value of the unknown attribute included in the candidate data from the candidates.

[0046] Also, for example, as illustrated in Figure 6, a certain candidate data (v 1a , x2, ..., x d Suppose the label of the candidate data is y1. Also, suppose the confidence score corresponding to the candidate data is [37, 0, 0], and the value corresponding to label y1 is 37. In this case, according to the confidence score, it can be determined that the number of training data with label 1 that fell into the corresponding leaf node during the training of the decision tree 241 was 37. Therefore, the identification unit 354 determines, based on the confidence score, that the candidate data may have actually constituted training data, and does not exclude the candidate value of the unknown attribute included in the candidate data from the candidates.

[0047] For example, as described above, the identification unit 354 refers to the confidence score and checks whether the value corresponding to the label of the candidate data is below a threshold, thereby determining whether to exclude the candidate value corresponding to the candidate data from the candidates. For example, by performing the above determination for each candidate data, the identification unit 354 can identify the range of unknown attributes that can be taken based on the confidence score. Note that the threshold mentioned above can be set arbitrarily. For example, if the threshold is set to 0, the candidate data can be excluded from the candidates only if it does not definitely constitute training data.

[0048] The evaluation unit 355 can assess risks such as privacy leaks based on the results identified by the identification unit 354. For example, the evaluation unit 355 can assess the risk in accordance with the number and proportion of candidate values ​​excluded by the identification unit 354. As an example, the evaluation unit 355 can assess that the risk of privacy leaks is higher if the number and proportion of candidate values ​​excluded by the identification unit 354 are large. In some cases, as a result of the identification unit 354 excluding candidate values ​​from the candidates, the possible values ​​for the unknown attribute may be narrowed down to a single value. In such cases, the evaluation unit 355 may determine that the risk of privacy leaks is extremely high based on the results identified by the identification unit 354. The evaluation unit 355 may also perform the above assessment when, as a result of the identification unit 354 excluding candidate values ​​from the candidates, the range of possible values ​​for the unknown attribute falls below a predetermined range.

[0049] The output unit 356 outputs information corresponding to the results identified by the identification unit 354, and information indicating the evaluation results by the evaluation unit 355. For example, the output unit 356 displays the above information on the screen display unit 320 or transmits it to an external device via the communication I / F unit 330.

[0050] The above is an example of the configuration of the risk assessment device 300. Next, an example of the operation of the risk assessment device 300 will be described with reference to Figures 7 and 8.

[0051] First, an example of the overall operation of the risk assessment device 300 will be described with reference to Figure 7. Referring to Figure 7, the candidate data creation unit 351 creates candidate data based on the prior information 341 (step S101). For example, the candidate data creation unit 351 creates candidate data corresponding to the number of candidates indicated by the unknown attribute information.

[0052] The candidate data transmission unit 352 transmits each candidate data created by the candidate data creation unit 351 to the model storage device 200 (step S102).

[0053] The inference result acquisition unit 353 acquires a confidence score for each candidate data as a result of inference based on the candidate data from the model storage device 200 (step S103).

[0054] The identification unit 354 identifies the range of possible unknown attributes based on the confidence score (step S104). For example, the identification unit 354 identifies the range of possible unknown attributes by excluding candidate values ​​for unknown attribute x1 that are judged to have little or no possibility of actually constituting the training data based on the confidence score.

[0055] The evaluation unit 355 evaluates the risks, such as privacy breaches, based on the results identified by the identification unit 354 (step S105). For example, the evaluation unit 355 evaluates the risk according to the number of candidate data excluded by the identification unit 354. As an example, the evaluation unit 355 can evaluate that the greater the number of candidate data excluded by the identification unit 354, the higher the risk of privacy breaches.

[0056] The above is an example of the overall operation of the risk assessment device 300. Next, the process of step S104 will be explained in more detail with reference to Figure 8. Referring to Figure 8, the identification unit 354 checks the value of the confidence score that corresponds to the label of the candidate data (step S201).

[0057] If the value of the confidence score corresponding to the label of the candidate data falls below a threshold (Step S201, Yes), the identification unit 354 excludes the candidate value of the unknown attribute included in the candidate data from the candidates (Step S202). On the other hand, if the value of the confidence score corresponding to the label of the candidate data exceeds a threshold (Step S201, No), the identification unit 354 does not exclude the candidate data.

[0058] If not all candidate data has been reviewed (step S203, No), the identification unit 354 checks the confidence score of the unreviewed candidate data (step S201). On the other hand, if all candidate data has been reviewed (step S203, Yes), the identification unit 354 terminates the process in step S104.

[0059] The above is a more detailed explanation of step S104.

[0060] Thus, the risk assessment device 300 includes an inference result acquisition unit 353 and a specification unit 354. With this configuration, the specification unit 354 can identify the range of possible unknown attributes by excluding candidate values ​​that satisfy the conditions based on the confidence score acquired by the inference result acquisition unit 353. As a result, it becomes possible to make a risk judgment according to the specified result. In other words, with the above configuration, even in situations where the value of the unknown attribute can be identified, an appropriate risk assessment can be performed.

[0061] In this embodiment, the case where there are 11 unknown attributes is illustrated as an example. However, this disclosure can be applied without any problems even when there are multiple unknown attributes.

[0062] For example, Figure 9 shows that the unknown attribute is x1 to x n This shows an example of prior information 341 in the case where there are multiple such items. For example, in Figure 9, attribute (x n+1 , ..., x d ) and label y are known, and the attributes (x1, ..., x n This example illustrates the case where the value of the attribute is unknown. In this case, the unknown attribute information will show information about the value of each unknown attribute.

[0063] As shown in Figure 9, if there are multiple unknown attributes, the candidate data creation unit 351 can create a number of candidate data corresponding to the combination of unknown attribute candidates, assuming that each unknown attribute can take one of the candidates. The candidate data transmission unit 352 and subsequent units can process the data in the same way as when there is only one unknown attribute. For example, as described above, even when there are multiple unknown attributes, the processing can be the same as when there is only one unknown attribute, except that the number of candidate data created by the candidate data creation unit 351 increases.

[0064] Furthermore, the model storage device 200 and the risk assessment device 300 may have configurations other than those illustrated in this embodiment. For example, Figure 10 shows another configuration example of the risk assessment device 300. Referring to Figure 10, the arithmetic processing unit 350 of the risk assessment device 300 can have an instruction unit 357 in addition to the configuration illustrated in Figure 4 by reading and executing the program 343.

[0065] The instruction unit 357 issues predetermined instructions to the model storage device 200 based on the results identified by the identification unit 354 or the results of the evaluation by the evaluation unit 355. For example, if the results identified by the identification unit 354 or the results of the evaluation by the evaluation unit 355 satisfy predetermined conditions, the instruction unit 357 can issue instructions on how to output the confidence score, which is score information.

[0066] For example, the instruction unit 357 can instruct the model storage device 200 to output only the value of the label with the maximum value as the confidence score if the results identified by the identification unit 354 meet predetermined conditions. As an example, suppose there is a confidence score of [0, 32, 0]. In this case, the instruction unit 357 can instruct the model storage device 200 to output information indicating only that the value of label 2 was 32 as the confidence score. The instruction unit 357 may also instruct the model storage device 200 to change any value that is 0 or less than a predetermined threshold to 0 or greater than the threshold and output it if such a value exists within the confidence score. As an example, suppose there is a confidence score of [0, 32, 0]. In this case, the instruction unit 357 can instruct the model storage device 200 to output the confidence score with a value such as [3, 32, 4]. The changed value may be determined by any means by the model storage device 200 or the risk assessment device 300. For example, as described above, the instruction unit 357 can instruct the model storage device 200 to output a confidence score in such a way that it reduces the possibility of identifying the range of possible unknown attributes from the confidence score.

[0067] The conditions under which the instruction unit 357 instructs the model storage device 200 can be set arbitrarily. For example, the instruction unit 357 can issue the above instruction when the number or percentage excluded by the identification unit 354 exceeds a predetermined threshold, or when the evaluation unit 355 evaluates the risk to be high. The instruction unit 357 may also issue instructions under conditions other than those exemplified above.

[0068] Furthermore, in this embodiment, an example was given in which the risk assessment system 100 has a model storage device 200 and a risk assessment device 300. However, the risk assessment system 100 may be composed of, for example, the model storage device 200 described in this embodiment and a single information processing device having the functions of a risk assessment device 300. The risk assessment system 100 may also employ other known modifications.

[0069] [Second Embodiment] Next, a second embodiment of the present disclosure will be described with reference to Figures 11 to 15. Figure 11 is a diagram showing an example configuration of the risk assessment system 400. Figure 12 is a block diagram showing an example configuration of the model storage device 500. Figure 13 is a block diagram showing an example configuration of the risk assessment device 600. Figure 14 is a diagram illustrating an example of processing by the specific unit 652. Figure 15 is a flowchart showing an example of operation of the risk assessment device 600.

[0070] In a second embodiment of this disclosure, a risk assessment system 400 is described that can perform risk assessment by identifying the range of possible values ​​for an unknown attribute when the decision tree 511, which is a learning model, is configured as a white box. For example, a model generated by machine learning may be configured as a black box, where only the output for a given input is exposed to the user, as exemplified in the first embodiment, or as a white box, where structural data of the model, such as the model's structure and branching conditions, is also exposed. As will be described later, the risk assessment system 400 in this embodiment acquires the structural data of the decision tree 511, which is information exposed by the white box configuration, and identifies the range of possible values ​​for the unknown attribute based on the acquired structural data. In other words, the risk assessment system 400 described in this embodiment acquires the structural data of the decision tree 511 as score information instead of a confidence score. The risk assessment system 400 then identifies the range of possible values ​​for the unknown attribute based on the acquired structural data.

[0071] Figure 11 shows an example configuration of the risk assessment system 400 in this embodiment. Referring to Figure 11, the risk assessment system 400 includes, for example, a risk assessment device 600 and a model storage device 500. As shown in Figure 11, the risk assessment device 600 and the model storage device 500 are connected to each other so that they can communicate with each other, for example, via a network.

[0072] The model storage device 500 is an information processing device that stores a decision tree 511, which is a learning model trained using training data. Figure 12 shows an example configuration of the model storage device 500. For example, referring to Figure 12, the model storage device 500 has a storage unit 510 in which the decision tree 511 is stored, as well as a structure information transmission unit 520. For example, the model storage device 500 has an arithmetic unit such as a CPU and a memory device, and the above processing unit can be realized by the arithmetic unit executing a program stored in the memory device. Note that the model storage device 500 may have a GPU or the like instead of the CPU mentioned above.

[0073] The memory unit 510 stores a decision tree 511, which is a pre-trained learning model. As described above, in this embodiment, a white-box configuration is adopted so that the structure and branching conditions of the decision tree 511 can be transmitted to an external device. The white-box configuration is used, for example, when conducting federative learning, in which models are trained while exchanging information between clients.

[0074] The structural information transmission unit 520 transmits information about the decision tree 511, which is a learning model, to the risk assessment device 600 in response to instructions from the risk assessment device 600. For example, the structural information transmission unit 520 transmits structural data such as the model structure of the decision tree 511, branching conditions indicating whether attribute values ​​are greater than or less than a threshold, and score values ​​indicating the number of training data for each label assigned to each node, to the risk assessment device 600 as information indicating the structure of the decision tree 511. The structural information transmission unit 520 may also transmit information about the decision tree 511 other than those exemplified above to the risk assessment device 600.

[0075] The configuration of the model storage device 500 is not limited to that illustrated in Figure 12. For example, the model storage device 500 may have the configuration of the model storage device 200 described with reference to Figure 2 in the first embodiment, in addition to the configuration illustrated in Figure 12. The model storage device 500 may also have configurations other than those illustrated above.

[0076] The risk assessment device 600 is an information processing device that uses structural data acquired from the model storage device 500 as score information to identify the range of possible unknown attributes. Furthermore, the risk assessment device 600 can perform risk assessments, such as privacy leakage, based on the identified results.

[0077] Figure 13 shows an example configuration of the risk assessment device 600. Referring to Figure 13, the risk assessment device 600 has, for example, an operation input unit 610, a screen display unit 620, a communication I / F unit 630, a storage unit 640, and an arithmetic processing unit 650 as its main components.

[0078] The operation input unit 610, screen display unit 620, and communication I / F unit 630 may have the same configuration as the operation input unit 310, screen display unit 320, and communication I / F unit 330 described in the first embodiment. Therefore, their description is omitted.

[0079] The storage unit 640 is a storage device such as a hard disk or memory. The storage unit 640 stores processing information and programs 643 necessary for various processes in the arithmetic processing unit 650. The programs 643 are read into the arithmetic processing unit 350 and executed to realize various processing functions. The programs 643 are pre-read from external devices or recording media via data input / output functions such as the communication I / F unit 330 and stored in the storage unit 340. The main information stored in the storage unit 340 includes, for example, pre-information 641 and structural information 642.

[0080] The prior information 641, like the prior information 341 described in the first embodiment, includes information already known about the training data used when training the decision tree 511 stored in the model storage device 500. For example, the prior information 641 may include information relating partial training data information and unknown attribute information. For example, the prior information 641 is acquired in advance by methods such as acquiring it from an external device via the communication I / F unit 630 or inputting it using the operation input unit 610, and is stored in the storage unit 640.

[0081] The structural information 642 includes information indicating the structural data of the decision tree 511 acquired by the structural information receiving unit 651 from the model storage device 500. For example, the structural information 642 is generated and updated in response to the structural information receiving unit 651, described later, acquiring structural data from the model storage device 500.

[0082] The arithmetic processing unit 650 includes an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 650 reads and executes the program 643 from the storage unit 640, thereby realizing various processing functions by having the hardware and the program 643 cooperate. Examples of the main processing functions realized by the arithmetic processing unit 650 include the structural information receiving unit 651, the identification unit 652, the evaluation unit 653, and the output unit 654. As mentioned above, the arithmetic processing unit 650 may have a GPU or the like instead of a CPU.

[0083] The structural information receiving unit 651 acquires structural data such as the structure and branching conditions of the decision tree 511 from the model storage device 500. The structural information receiving unit 651 may also send an instruction to the model storage device 500 to request the transmission of structural data at any time, and acquire the structural data transmitted from the model storage device 500 in response to that instruction. The structural information receiving unit 651 also stores the acquired structural data as structural information 642 in the storage unit 640.

[0084] The identification unit 652 identifies the range of possible unknown attributes based on the structural data, which is score information. For example, the identification unit 652 identifies candidate unknown attributes x1 (v 11 ...v 1k By excluding candidate values ​​from the structured data that are unlikely to have actually constituted the training data, or are considered unlikely to have done so, the range of possible values ​​for the unknown attribute is identified.

[0085] For example, the identification unit 652 refers to the structure data and identifies leaf nodes corresponding to score values ​​that are below a predetermined threshold. The identification unit 652 also checks the branching conditions of each node on the path between the identified leaf node and the root node in the decision tree 511. For example, the identification unit 652 checks whether there are any nodes on the path between the identified leaf node and the root node that perform branching due to an unknown attribute. If there are nodes that perform branching due to an unknown attribute, the identification unit 652 excludes candidate values ​​from the list of attributes that satisfy the branching conditions of each node on that path, including the value of the unknown attribute.

[0086] Specifically, for example, referring to Figure 14, the score value of a certain leaf node is [0, aa, bb], which includes a value that is less than or equal to a predetermined threshold of 0. Therefore, the identification unit 652 checks the branching conditions of each node that exists on the path between the leaf node and the root node, as shown in Figure 14. In the example shown in Figure 14, the checked path includes a node that branches due to an unknown attribute. Therefore, the identification unit 652 excludes candidate values ​​from the candidates that are combinations of attribute values ​​that include the value of the unknown attribute and satisfy the branching conditions of each node that exists on the path.

[0087] For example, as described above, the identification unit 652 checks whether there are any nodes that branch due to unknown attributes on the path between a leaf node and a root node that satisfy predetermined conditions, and excludes each node based on its branching conditions according to the result of the check. By making the above determination for each leaf node that satisfies the conditions, the identification unit 652 can identify the range of unknown attributes that can take place based on the structural data. Note that the thresholds mentioned above may be set arbitrarily.

[0088] Furthermore, the identification unit 652 may perform the same processing as the identification unit 354 described in the first embodiment based on the structural data to exclude candidate values ​​for unknown attributes from the candidates.

[0089] The evaluation unit 653 can evaluate risks such as privacy leaks based on the results identified by the identification unit 652. For example, the evaluation unit 653 may evaluate risks in accordance with the number of candidate data excluded by the identification unit 652, similar to the evaluation unit 355 described in the first embodiment.

[0090] The output unit 654 outputs information corresponding to the results identified by the identification unit 652, and information indicating the evaluation results by the evaluation unit 653. For example, the output unit 654 displays the above information on the screen display unit 620 or transmits it to an external device via the communication I / F unit 630.

[0091] The above describes an example configuration of the risk assessment device 600. The risk assessment device 600 may have a configuration similar to that of the risk assessment device 300 described in the first embodiment, in addition to the above configuration, or it may adopt various modifications of the risk assessment device 300 described in the first embodiment. For example, the risk assessment device 600 may have functions similar to those of the instruction unit 357 described in the first embodiment. Next, an example of the operation of the risk assessment device 600 will be described with reference to Figure 15.

[0092] Referring to Figure 15, the structural information receiving unit 651 acquires structural data such as the structure of the decision tree 511, branching conditions, and score values ​​from the model storage device 500 (step S301).

[0093] The identification unit 652 identifies the range of possible unknown attributes based on the structural data, which is score information (step S302). For example, the identification unit 652 identifies candidate unknown attributes x1 (v 11 ...v 1k The range of possible unknown attributes is identified by excluding candidate values ​​from the structural data that are deemed unlikely to have actually constituted the training data, or are considered unlikely to have done so. For example, the identification unit 652 checks whether there are any nodes that branch due to unknown attributes on the path between a leaf node and a root node that satisfy predetermined conditions, and then excludes nodes based on their branching conditions according to the results of the check, thereby identifying the range of possible unknown attributes.

[0094] The evaluation unit 653 evaluates the risks, such as privacy leaks, based on the results identified by the identification unit 652 (step S303). For example, the evaluation unit 653 may evaluate the risks in accordance with the number of candidate data excluded by the identification unit 652.

[0095] The above is an example of the operation of the risk assessment device 600.

[0096] Thus, the risk assessment device 600 has a structural information receiving unit 651 and a identification unit 652. With this configuration, the identification unit 652 can identify the range of possible unknown attributes by excluding candidate values ​​that satisfy the conditions based on the structural data acquired by the structural information receiving unit 651. As a result, it becomes possible to make a risk judgment according to the identified result. In other words, with the above configuration, even in situations where the value of the unknown attribute can be identified, an appropriate risk assessment can be performed.

[0097] [Third Embodiment] A third embodiment of this disclosure describes an example configuration of a processing unit 700, which is an information processing device capable of performing an evaluation-based identification process to identify the range of possible values ​​for an unknown attribute based on information about known attributes. Figure 16 shows an example of the hardware configuration of the processing unit 700. Referring to Figure 16, the processing unit 700 has, as an example, the following hardware configuration. ·CPU (Central Processing Unit) 701 (computing unit) ROM (Read Only Memory) 702 (Storage Device) • RAM (Random Access Memory) 703 (storage device) • Program group 704 loaded into RAM703 • Storage device 705 for storing the program group 704 • Drive device 706 for reading and writing to recording medium 710 outside the information processing device. • Communication interface 707 that connects to the communication network 711 outside the information processing device. • Input / output interface 708 for data input / output. • Bus 709 connecting each component

[0098] Furthermore, the processing unit 700 can realize the functions of the acquisition unit 721 and the identification unit 722 shown in Figure 17 by having the CPU 701 acquire the program group 704 and execute it. The program group 704 is, for example, stored in advance in the storage device 705 or ROM 702, and the CPU 701 loads it into RAM 703 or the like and executes it as needed. Alternatively, the program group 704 may be supplied to the CPU 701 via the communication network 711, or it may be stored in advance in the recording medium 710, and the drive device 706 may read the program and supply it to the CPU 701.

[0099] Figure 16 shows an example of the hardware configuration of the processing unit 700. The hardware configuration of the processing unit 700 is not limited to the case described above. For example, the processing unit 700 may consist of only a part of the configuration described above, such as not having a drive device 706.

[0100] The acquisition unit 721 acquires score information from a pre-trained model, which is a decision tree consisting of multiple nodes. This score information represents a value corresponding to the number of data points that fell into a node from the multiple training data used during the training of the decision tree.

[0101] The identification unit 722 identifies the range of possible values ​​for the unknown attribute, which is part of a set of multiple attributes included in the training data, based on the score information acquired by the acquisition unit 721.

[0102] Thus, the processing unit 700 has an acquisition unit 721 and a identification unit 722. With this configuration, the identification unit 722 can identify the range of possible values ​​for an unknown attribute, which is part of a set of multiple attributes included in the training data, based on the score information acquired by the acquisition unit 721. As a result, it becomes possible to make a risk judgment according to the identified result. In other words, with the above configuration, even in situations where the value of the unknown attribute can be identified, an appropriate risk assessment can be performed.

[0103] The above-mentioned processing unit 700 can be realized by incorporating a predetermined program into the information processing device such as the processing unit 700. Specifically, another form of the present invention is a program that enables the information processing device such as the processing unit 700 to perform the following processing: obtaining score information from a decision tree, which is a trained model consisting of multiple nodes, that indicates a value corresponding to the number of data points that fell into the nodes from among the multiple training data used during the training of the decision tree; and identifying the range of possible values ​​for an unknown attribute, which is part of the multiple attributes included in the training data, based on the obtained score information.

[0104] Furthermore, the processing method performed by the information processing device such as the processing device 700 described above involves the information processing device such as the processing device 700 obtaining score information from a decision tree, which is a trained model consisting of multiple nodes, that indicates a value corresponding to the number of data points that fell into the nodes from among the multiple training data used during the training of the decision tree, and then identifying the range of possible values ​​for the unknown attribute, which is a part of the multiple attributes included in the training data, based on the obtained score information.

[0105] Even if the invention is a program, or a recording medium readable by a computer on which the program is recorded, or a processing method having the above-described configuration, it can achieve the same functions and effects as the processing device 700 described above, and thus the objectives of the present invention described above can be achieved.

[0106] <Note> Some or all of the above embodiments may also be described as follows. The general outline of the processing apparatus and other components in the present invention will be described below. However, the present invention is not limited to the following configuration.

[0107] (Note 1) An acquisition unit that acquires score information from a pre-trained model, which is a decision tree consisting of multiple nodes, and which represents a value corresponding to the number of data points that fell into the nodes from among the multiple training data used during the training of the decision tree. Based on the score information acquired by the acquisition unit, the identification unit identifies the range of possible values ​​for an unknown attribute, which is part of a plurality of attributes included in the training data. has Processing device. (Note 2) The processing apparatus described in Appendix 1, It has a creation unit that creates multiple candidate data based on information indicating the values ​​of known attributes that it has in advance and information indicating candidate values ​​for unknown attributes. The acquisition unit acquires the score information by acquiring a plurality of inference results that are inferred as a result of inputting each of the plurality of candidate data created by the creation unit to the decision tree. Processing device. (Note 3) The processing apparatus described in Appendix 2, The aforementioned training data includes values ​​for multiple attributes and labels. The inference result indicates a value corresponding to the proportion of data points in the training data that correspond to each label, in the leaf nodes of the decision tree to which the candidate data belongs. The identification unit identifies the range of possible values ​​for the unknown attribute by excluding candidate values ​​from the inference results based on values ​​corresponding to labels that correspond to the candidate data. Processing device. (Note 4) The processing apparatus described in Appendix 3, The identification unit identifies the range of possible values ​​for the unknown attribute by excluding candidate values ​​from the inference results whose values ​​corresponding to the labels of the candidate data are below a predetermined threshold. Processing device. (Note 5) The processing apparatus described in Appendix 1, The acquisition unit acquires the score information by acquiring the structural information of the decision tree corresponding to each node included in the decision tree. The score information indicates a value corresponding to the proportion of data points corresponding to each label in the training data at the node. The identification unit identifies a leaf node corresponding to the score information containing a value less than or equal to a predetermined threshold, and, based on the score information corresponding to the node located on the path between the identified leaf node and the root node which is the first branch in the decision tree, identifies the range of possible values ​​for the unknown attribute. Processing device. (Note 6) The processing apparatus described in Appendix 5, The identifying unit determines the range of possible values ​​for the unknown attribute by checking whether there are any nodes on the path between the leaf node and the root node that perform branching based on the unknown attribute. Processing device. (Note 7) The processing apparatus described in Appendix 1, The system includes an instruction unit that, based on the results specified by the specified unit, instructs how the decision tree should output the score information. Processing device. (Note 8) The processing apparatus described in Appendix 1, The system includes an evaluation unit that evaluates the risk of the decision tree based on the results of the identification unit. Processing device. (Note 9) Information processing device, From a pre-trained model, which is a decision tree consisting of multiple nodes, score information is obtained that represents a value corresponding to the number of data points that fell into the nodes from the multiple training data used during the training of the decision tree. Based on the acquired score information, the range of possible values ​​for the unknown attribute, which is part of the multiple attributes included in the training data, is identified. Processing method. (Note 10) In an information processing device, From a pre-trained model, which is a decision tree consisting of multiple nodes, score information is obtained that represents a value corresponding to the number of data points that fell into the nodes from the multiple training data used during the training of the decision tree. Based on the acquired score information, the range of possible values ​​for the unknown attribute, which is part of the multiple attributes included in the training data, is identified. A program to perform the processing.

[0108] Although the present invention has been described above with reference to the embodiments described above, the present invention is not limited to the embodiments described above. Various modifications to the structure and details of the present invention can be made within the scope of the present invention as can be understood by those skilled in the art. [Explanation of Symbols]

[0109] 100 Risk Assessment Systems 200 Model Storage Unit 210 Receiver 220 Reasoning Department 230 Output section 240 Storage section 241 Decision Trees 300 Risk Assessment Device 310 Operation Input Section 320 Screen display section 330 Communication I / F Section 340 Storage section 341 Preliminary Information 342 Inference result information 343 Programs 350 Arithmetic Processing Unit 351 Candidate Data Creation Department 352 Candidate Data Transmission Unit 353 Inference result acquisition part 354 Specific part 355 Evaluation Department 356 Output section 357 Instruction section 400 Risk Assessment Systems 500 Model Storage Unit 510 Storage section 511 decision trees 520 Structural Information Transmission Unit 600 Risk Assessment Device 610 Operation Input Section 620 Screen display section 630 Communication I / F Section 640 Storage section 641 Preliminary Information 642 Structure information 643 Programs 650 Arithmetic Processing Unit 651 Structural Information Receiving Unit 652 Specific part 653 Evaluation Department 654 Output section 700 Processing Unit 701 CPU 702 ROM 703 RAM 704 Program Groups 705 Storage device 706 Drive unit 707 Communication Interface 708 Input / Output Interface 709 Bus 710 Recording media 711 Communications Network 721 Acquisition Department 722 Specific part

Claims

1. An acquisition unit that acquires score information from a pre-trained model, which is a decision tree consisting of multiple nodes, and which represents a value corresponding to the number of data points that fell into the nodes from among the multiple training data used during the training of the decision tree. Based on the score information acquired by the acquisition unit, the identification unit identifies the range of possible values ​​for an unknown attribute, which is part of a plurality of attributes included in the training data. has Processing device.

2. The apparatus according to claim 1, It has a creation unit that creates multiple candidate data based on information indicating the values ​​of known attributes that it has in advance and information indicating candidate values ​​for unknown attributes. The acquisition unit acquires the score information by acquiring a plurality of inference results that are inferred as a result of inputting each of the plurality of candidate data created by the creation unit to the decision tree. Processing device.

3. The apparatus according to claim 2, The aforementioned training data includes values ​​for multiple attributes and labels. The inference result indicates a value corresponding to the proportion of data points in the training data that correspond to each label, in the leaf nodes of the decision tree to which the candidate data belongs. The identification unit identifies the range of possible values ​​for the unknown attribute by excluding candidate values ​​from the inference results based on values ​​corresponding to labels that correspond to the candidate data. Processing device.

4. The apparatus according to claim 3, The identification unit identifies the range of possible values ​​for the unknown attribute by excluding candidate values ​​from the inference results whose values ​​corresponding to the labels of the candidate data are below a predetermined threshold. Processing device.

5. The apparatus according to claim 1, The acquisition unit acquires the score information by acquiring the structural information of the decision tree corresponding to each node included in the decision tree. The score information indicates a value corresponding to the proportion of data points corresponding to each label in the training data at the node. The identification unit identifies a leaf node corresponding to the score information containing a value less than or equal to a predetermined threshold, and, based on the score information corresponding to the node located on the path between the identified leaf node and the root node which is the first branch in the decision tree, identifies the range of possible values ​​for the unknown attribute. Processing device.

6. The processing apparatus according to claim 5, The identifying unit determines the range of possible values ​​for the unknown attribute by checking whether there are any nodes on the path between the leaf node and the root node that perform branching based on the unknown attribute. Processing device.

7. The apparatus according to claim 1, The system includes an instruction unit that, based on the results specified by the specified unit, instructs how the decision tree should output the score information. Processing device.

8. The apparatus according to claim 1, The system includes an evaluation unit that evaluates the risk of the decision tree based on the results of the identification unit. Processing device.

9. Information processing device, From a pre-trained model, which is a decision tree consisting of multiple nodes, score information is obtained that represents a value corresponding to the number of data points that fell into the nodes from the multiple training data used during the training of the decision tree. Based on the acquired score information, the range of possible values ​​for the unknown attribute, which is part of the multiple attributes included in the training data, is identified. Processing method.

10. In an information processing device, From a pre-trained model, which is a decision tree consisting of multiple nodes, score information is obtained that represents a value corresponding to the number of data points that fell into the nodes from the multiple training data used during the training of the decision tree. Based on the acquired score information, the range of possible values ​​for the unknown attribute, which is part of the multiple attributes included in the training data, is identified. A program to perform the processing.

Images