Model learning device, model learning method, and program

By generating synthetic input and evaluation results for supervised learning, the technique addresses the challenge of data leaks and noise in differential privacy, ensuring accurate and private label data for model training, resulting in improved model performance and privacy protection.

WO2024228228A1PCT designated stage expired Publication Date: 2024-11-07NIPPON TELEGRAPH & TELEPHONE CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/017014
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Existing supervised learning techniques face challenges in preventing data leaks and achieving balanced model performance when using sensitive data, as the noise introduced during synthetic data generation for differential privacy affects label accuracy, making it difficult to update training data effectively.

Method used

The approach involves creating a secondary training data set using synthetic input data and raw evaluation results, which are then used to generate tertiary input and label data, ensuring that labels are not synthetic and thus free from noise, allowing for differential privacy while maintaining model performance.

Benefits of technology

This method effectively prevents data leakage and ensures that labels guarantee differential privacy, enabling high-performance model learning by separating raw data from synthetic noise, thus tightening privacy budgets and improving model accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023017014_07112024_PF_FP_ABST
    Figure JP2023017014_07112024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention provides a supervised learning technology using training data generated as data that does not include influence of noise occurring when synthetic data is generated such that a label guarantees differential privacy. The present invention includes a secondary training data set generation unit that generates, through a method comprising generating synthetic data to serve as input to a model from primary input data constituting primary training data, each piece of which is an element of a primary training data set, through a predetermined synthetic data generation method, generating raw data indicating an evaluation result of output of the model for said synthetic data, and setting a pair of said synthetic data and said raw data as a pair of secondary input data and secondary label, a secondary training data set including, as each of elements, secondary training data formed of the pair of the secondary input data and the secondary label.
Need to check novelty before this filing date? Find Prior Art

Description

Model learning device, model learning method, and program

[0001] The present invention relates to a supervised learning technique that can prevent the leakage of training data.

[0002] Currently, machine learning, known as supervised learning, is used in a variety of application fields. As the range of application fields expands, there are also increasing opportunities to handle more sensitive data as the data used for learning (hereinafter referred to as training data). It is also becoming common for training data to be continuously updated along with the model. Here, training data in supervised learning refers to a set of input data that serves as input for the model and data (hereinafter referred to as labels) that indicate the evaluation results of the model's output for that input data. Furthermore, data that indicates the evaluation results of the model's output refers to, for example, data that indicates whether the output is correct or incorrect.

[0003] As this application field expands, there is a demand for techniques to conceal training data to prevent data leakage. For example, a method for concealing training data is described in Non-Patent Document 1. In the method described in Non-Patent Document 1, a privacy budget is specified in advance, and training data is generated as concealed data (hereinafter referred to as synthetic data) taking differential privacy into consideration.

[0004] Takayuki Miura, Masanori Kii, Atsutake Ichikawa, Koji Senda, and Eiyoshi Kimura, "A Study on Applying Synthetic Data Generation Technology to Medical Data," Research Report Computer Security (CSEC), vol. 6, pp. 1-8, 2022.

[0005] However, in the method described in Non-Patent Document 1, since the input data and labels that make up the training data are concealed at the same time, it is not possible to modify the labels when updating the training data. As a result, the labels remain affected by noise that occurs when generating synthetic data in a way that guarantees differential privacy, making it difficult to achieve training that achieves both a strict privacy budget and good model performance.

[0006] Therefore, the present invention aims to provide a supervised learning technique that uses training data generated as data that is free from the influence of noise that occurs when generating synthetic data in a manner that ensures differential privacy in labels.

[0007] One aspect of the present invention is a method of generating synthetic data to be input to the model from primary input data constituting the primary training data, which are elements of the primary training data set, by a predetermined synthetic data generation method, generating raw data indicating the evaluation results of the model's output for the synthetic data, and using the set of the synthetic data and the raw data as a set of secondary input data and secondary labels; or a method of generating synthetic data to be input to the model from primary input data constituting the primary training data, which are elements of the primary training data set, by a predetermined synthetic data generation method, generating raw data indicating the evaluation results of the model's output for the synthetic data, and using the set of the synthetic data and the raw data as a set of secondary input data and secondary labels; the synthetic data generation method of generating a set of synthetic data (hereinafter referred to as first synthetic data) that will be input to a model and synthetic data (hereinafter referred to as second synthetic data) that indicates an evaluation result of the model's output for the first synthetic data, generating raw data that indicates an evaluation result of the model's output for the first synthetic data using the second synthetic data, and treating the set of the first synthetic data and the raw data as a set of secondary input data and secondary label, the method comprising: a secondary training data set generation unit that generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) that is made up of sets of secondary input data and secondary labels; and a model learning unit that learns a model using the secondary training data set.

[0008] In one aspect of the present invention, a primary training data set is a set whose elements are training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data to be input to a model, and a primary label, which is raw data indicating an evaluation result of the model's output for the primary input data; a secondary training data set generation unit generates a pair of synthetic data (hereinafter referred to as first synthetic data), which is to be input to the model, and synthetic data (hereinafter referred to as second synthetic data), indicating an evaluation result of the model's output for the first synthetic data, from the primary training data, which is an element of the primary training data set, by a predetermined synthetic data generation method; and generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting of a pair of secondary input data and a secondary label by using the pair of the first synthetic data and the second synthetic data as a pair of secondary input data and a secondary label; The present invention includes a tertiary training data set generation unit that generates a tertiary training data set whose elements are training data consisting of pairs of tertiary input data and tertiary labels (hereinafter referred to as tertiary training data) by using a method in which secondary training data that is expected to be highly useful for improving model performance is extracted using a score calculated based on the distribution of secondary input data that constitutes elementary secondary training data, or a score calculated based on the output for secondary input data that constitutes secondary training data that is an element of the secondary training data set obtained using the model under training, generates raw data that indicates the evaluation results of the model's output for the secondary input data that constitutes the secondary training data using the secondary labels that constitute the secondary training data, and treats the pair of the secondary input data and the raw data as a pair of tertiary input data and tertiary labels; and a model learning unit that learns the model using the tertiary training data set.

[0009] In one aspect of the present invention, a primary training data set is a set whose elements are training data (hereinafter referred to as primary training data) consisting of a set of primary input data, which is raw data to be input to a model, and primary labels, which are raw data indicating the evaluation results of the model's output for the primary input data; a secondary training data set generation unit that generates synthetic data to be input to the model from primary input data that constitutes the primary training data, which is an element of the primary training data set, by a predetermined synthetic data generation method, and generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting only of secondary input data by using the synthetic data as secondary input data; and a tertiary training data set generation unit that generates a tertiary training data set whose elements are training data consisting of pairs of tertiary input data and tertiary labels (hereinafter referred to as tertiary training data) by using a score calculated based on a distribution of σ or a score calculated based on the output for secondary input data that is an element of the secondary training data set obtained using the model under training, generates raw data that indicates the evaluation results of the model's output for the secondary input data, and treats the pair of the secondary input data and the raw data as a pair of tertiary input data and tertiary labels; and a model learning unit that learns the model using the tertiary training data set.

[0010] According to the present invention, since the labels are generated as data that is not synthetic data, it is possible to make the labels free from the influence of noise that occurs when synthetic data is generated in a manner that guarantees differential privacy.

[0011] Fig. 1 is a block diagram showing the configuration of a model learning device 100 / 200. Fig. 2 is a flowchart showing the operation of the model learning device 100 / 200. Fig. 3 is a block diagram showing the configuration of a model learning device 300 / 400. Fig. 4 is a flowchart showing the operation of the model learning device 300 / 400. Fig. 5 is a diagram showing an example of the functional configuration of a computer that realizes each device in an embodiment of the present invention.

[0012] Hereinafter, embodiments of the present invention will be described in detail. Components having the same functions are given the same numbers, and duplicated explanations will be omitted.

[0013] Before describing each embodiment, the notation used in this specification will be explained.

[0014] ^ (caret) represents a superscript, e.g., x y^z Yes z is a superscript to x, and x y^z Yes z is a subscript to x. Also, _ (underscore) represents a subscript. For example, x y_z Yes z is a superscript to x, and x y_z Yes z is a subscript to x.

[0015] The superscripts "^" and "~" such as ^x and ~x for a certain letter x should be written directly above the "x", but due to restrictions on the notation in the specification, they are written as ^x and ~x.

[0016] First Embodiment A model learning device 100 learns a model through supervised learning. Here, the model is a function, and is configured as, for example, a neural network.

[0017] The model learning device 100 will be described below with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing the configuration of the model learning device 100. FIG. 2 is a flowchart showing the operation of the model learning device 100. As shown in FIG. 1, the model learning device 100 includes a secondary training data set generation unit 110, a model learning unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the model learning device 100. The recording unit 190 pre-records the model to be used at the start of learning. If the model is configured as a neural network, parameters such as the weights and biases of the neural network are recorded in the recording unit 190. The recording unit 190 may also pre-record a primary training data set instead of using it as input to the model learning device 100. Here, the primary training data set refers to a set of training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data that serves as input to the model, and primary labels, which are raw data that indicate the evaluation results of the model's output for the primary input data.

[0018] The operation of the model learning device 100 will be described with reference to FIG.

[0019] In S110, the secondary training data set generation unit 110 receives the primary training data set as input, and generates a set of synthetic data (hereinafter referred to as first synthetic data) that serves as input to the model from the primary training data, which are elements of the primary training data set, using a predetermined synthetic data generation method, and synthetic data (hereinafter referred to as second synthetic data) that indicates the evaluation results of the model's output for the first synthetic data. The unit then generates and outputs a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting of pairs of secondary input data and secondary labels, using a method in which the pairs of the first synthetic data and the second synthetic data are treated as pairs of secondary input data and secondary labels. Any method can be used as the synthetic data generation method, including, for example, a method based on statistics described in Reference Non-Patent Document 1, a method based on Bayesian statistics described in Reference Non-Patent Document 2, and a method based on deep learning described in Reference Non-Patent Document 3. Note that differential privacy is guaranteed when generating the synthetic data (see Non-Patent Document 1).

[0020] (Reference Non-Patent Document 1: Rina Okada, Shogo Masaki, Satoshi Hasegawa, and Satoshi Tanaka, "Privacy-Preserving Pseudo-Data Generation Method Using Statistical Values," Proceedings of the Computer Security Symposium 2017, vol. 2, 2017.) (Reference Non-Patent Document 2: Judea Pearl, "Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference," Morgan Kaufmann, 1988.) (Reference Non-Patent Document 3: Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni, "Modeling Tabular Data using Conditional GAN," NIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, No. 659, pp. 7335-7345, 2019.) Furthermore, the number of elements contained in the generated secondary training data set may be any number, and may be the same as the number of elements contained in the primary training data set, for example.

[0021] In S120, the model learning unit 120 receives the secondary training data set generated in S110 as input, learns a model using the secondary training data set, and outputs the model. Here, any supervised learning method can be used as the learning method.

[0022] According to an embodiment of the present invention, even if an attack is made to extract data used to train a trained model when estimating an output for an input using the model, it is possible to prevent leakage of raw data, which is the data from which the data used to train the model is generated.

[0023] Second Embodiment The model learning device 100 generates input data and labels that constitute training data as synthetic data, so the labels may be affected by noise that occurs when generating synthetic data in a manner that ensures differential privacy, which may result in a decrease in the performance of the model obtained by learning. Therefore, the model learning device 200 generates labels as data that is not synthetic data.

[0024] The model learning device 200 will be described below with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing the configuration of the model learning device 200. FIG. 2 is a flowchart showing the operation of the model learning device 200. As shown in FIG. 2, the model learning device 200 includes a secondary training data set generation unit 210, a model learning unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the model learning device 200. The recording unit 190 records in advance the model to be used at the start of learning. Alternatively, the recording unit 190 may record in advance the primary training data set instead of using it as input to the model learning device 200.

[0025] The operation of the model learning device 200 will be described with reference to FIG.

[0026] In S210, the secondary training data set generation unit 210 receives the primary training data set as input, generates pairs of secondary input data and secondary labels using a predetermined method using the primary training data set, and generates and outputs a secondary training data set whose elements are training data consisting of pairs of secondary input data and secondary labels (hereinafter referred to as secondary training data). Here, the predetermined method may be, for example, a method of generating synthetic data that serves as input to the model using a predetermined synthetic data generation method from primary input data that constitutes the primary training data, which are elements of the primary training data set, generating raw data that indicates the evaluation results of the model output for the synthetic data, and treating the pairs of the synthetic data and the raw data as pairs of secondary input data and secondary labels. Here, the raw data may be generated manually, for example. Alternatively, the predetermined method may involve generating a set of synthetic data (hereinafter referred to as first synthetic data) that will be input to a model from primary training data that are elements of a primary training data set using a predetermined synthetic data generation method, and synthetic data (hereinafter referred to as second synthetic data) that indicates an evaluation result of the model's output for the first synthetic data, using the second synthetic data to generate raw data that indicates an evaluation result of the model's output for the first synthetic data, and treating the set of the first synthetic data and the raw data as a set of secondary input data and secondary labels. Here, the raw data may be generated manually, for example, by referring to the second synthetic data.

[0027] As with the model learning device 100, the synthetic data generation method and the number of elements included in the generated secondary training data set may be arbitrary.

[0028] In S120, the model learning unit 120 receives the secondary training data set generated in S210 as input, learns a model using the secondary training data set, and outputs the model.

[0029] <<Modifications>> In the above explanation, the primary training data set is a set whose elements are training data consisting of pairs of primary input data and primary labels, but the primary training data set may be a set whose elements are training data consisting only of primary input data. In this case, only the operation of the secondary training data set generation unit 210 is different, as explained below.

[0030] In S210, the secondary training data set generation unit 210 receives the primary training data set as input, generates synthetic data to be input to the model from primary input data that are elements of the primary training data set using a predetermined synthetic data generation method, generates raw data that indicates the evaluation results of the model output for the synthetic data, and generates and outputs a secondary training data set whose elements are training data consisting of pairs of secondary input data and secondary labels (hereinafter referred to as secondary training data) using a method in which the pairs of the synthetic data and the raw data are treated as pairs of secondary input data and secondary labels. Here, the raw data may be generated manually, for example.

[0031] According to an embodiment of the present invention, even if an attacker attempts to extract data used to train a trained model when estimating an output for an input using the trained model, it is possible to prevent leakage of raw data, which is the data from which the data used to train the model is generated. Furthermore, by generating secondary labels as raw data, it is possible to prevent the influence of noise that occurs when generating synthetic data in a manner that ensures differential privacy, thereby enabling training that achieves both a strict privacy budget and high model performance.

[0032] Third Embodiment Similar to the model learning device 200, the model learning device 300 generates labels as data that is not synthetic data.

[0033] The model learning device 300 will be described below with reference to FIGS. 3 and 4. FIG. 3 is a block diagram showing the configuration of the model learning device 300. FIG. 4 is a flowchart showing the operation of the model learning device 300. As shown in FIG. 3, the model learning device 300 includes a secondary training data set generation unit 110, a tertiary training data set generation unit 320, a model learning unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the model learning device 300. The recording unit 190 pre-records the model to be used at the start of learning. Alternatively, the recording unit 190 may pre-record the primary training data set instead of using it as input to the model learning device 300.

[0034] The operation of the model learning device 300 will be described with reference to FIG.

[0035] In S110, the secondary training data set generation unit 110 receives the primary training data set as input, and generates a set of synthetic data (hereinafter referred to as first synthetic data) that will be input to the model from the primary training data that is an element of the primary training data set using a predetermined synthetic data generation method, and synthetic data (hereinafter referred to as second synthetic data) that indicates the evaluation results of the model's output for the first synthetic data, and generates and outputs a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting of pairs of secondary input data and secondary labels using a method in which the pair of the first synthetic data and the second synthetic data is treated as a pair of secondary input data and secondary labels.

[0036] In S320, the tertiary training data set generation unit 320 receives the secondary training data set generated in S110 as input, extracts secondary training data expected to be highly useful for improving the performance of the model using scores calculated based on the distribution of secondary input data constituting the secondary training data, which are elements of the secondary training data set, or scores calculated based on the output for secondary input data constituting the secondary training data, which are elements of the secondary training data set, obtained using the model being trained, generates raw data indicating the evaluation results of the model's output for the secondary input data constituting the secondary training data, and generates and outputs a tertiary training data set whose elements are training data consisting of pairs of tertiary input data and tertiary labels (hereinafter referred to as tertiary training data) by using the secondary labels constituting the secondary training data. Here, the raw data may be generated manually, for example, by referring to the secondary labels.

[0037] The score can be calculated based on the uncertainty of the model output or the entropy of the model output. Furthermore, if noise is large when generating synthetic data in a manner that ensures differential privacy, the distribution of the synthetic data may be biased. Therefore, the distribution of the primary input data may be used in addition to the distribution of the secondary input data when calculating the score.

[0038] The extracted secondary training data may include not only secondary training data extracted based on the score, but also secondary training data extracted randomly. Note that the number of extracted secondary training data may be one or more.

[0039] In S120, the model learning unit 120 receives the tertiary training data set generated in S320 as input, learns a model using the tertiary training data set, and outputs the model.

[0040] The model learning device 300 may learn a model by repeating the process of S320 and the process of S120. The repeated process may be performed until a predetermined condition is met (for example, until a predetermined number of times is reached or until the model output reaches a predetermined accuracy). Furthermore, the tertiary training data set generated by the second or subsequent process of S320 may be the union of the tertiary training data sets generated by the previous processes of S320.

[0041] According to an embodiment of the present invention, even if an attack to extract data used to train a trained model occurs when estimating an output for an input using the trained model, it is possible to prevent leakage of raw data, which is the data from which the data used to train the model is generated. Furthermore, by generating tertiary labels as raw data, it is possible to eliminate the influence of noise that occurs when generating synthetic data in a manner that ensures differential privacy, thereby enabling training that achieves both a stricter privacy budget and high model performance. Furthermore, by limiting the number of tertiary labels generated as raw data, it is possible to further tighten the privacy budget and reduce the amount of work required when manually generating tertiary labels.

[0042] Fourth Embodiment Although the model learning device 300 generates the secondary training data as a set of secondary input data and secondary input data, the secondary training data may be generated as secondary input data only. Therefore, the model learning device 400 generates the secondary training data as secondary input data only.

[0043] The model learning device 400 will be described below with reference to FIGS. 3 and 4. FIG. 3 is a block diagram showing the configuration of the model learning device 400. FIG. 4 is a flowchart showing the operation of the model learning device 400. As shown in FIG. 3, the model learning device 400 includes a secondary training data set generation unit 410, a tertiary training data set generation unit 320, a model learning unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the model learning device 400. The recording unit 190 pre-records the model to be used at the start of learning. Alternatively, the recording unit 190 may pre-record the primary training data set instead of using it as input to the model learning device 400.

[0044] The operation of the model learning device 400 will be described with reference to FIG.

[0045] In S410, the secondary training data set generation unit 410 receives the primary training data set as input, generates synthetic data to be used as input to the model from the primary input data that constitutes the primary training data, which is an element of the primary training data set, using a predetermined synthetic data generation method, and generates and outputs a secondary training data set whose elements are training data consisting only of the secondary input data (hereinafter referred to as secondary training data) using the synthetic data as secondary input data.

[0046] In S320, the tertiary training data set generation unit 320 receives as input the secondary training data set generated in S410, and extracts secondary input data that is expected to be highly useful for improving the performance of the model using a score calculated based on the distribution of secondary input data that are elements of the secondary training data set, or a score calculated based on the output for secondary input data that are elements of the secondary training data set obtained using the model under training, generates raw data that indicates the evaluation results of the model's output for the secondary input data, and generates and outputs a tertiary training data set whose elements are training data consisting of pairs of tertiary input data and tertiary labels (hereinafter referred to as tertiary training data) by using a method in which the pair of the secondary input data and the raw data is treated as a pair of tertiary input data and tertiary labels.

[0047] In S120, the model learning unit 120 receives the tertiary training data set generated in S320 as input, learns a model using the tertiary training data set, and outputs the model.

[0048] <<Modifications>> In the above explanation, the primary training data set is a set whose elements are training data consisting of pairs of primary input data and primary labels, but the primary training data set may be a set whose elements are training data consisting only of primary input data. In this case, only the operation of the secondary training data set generation unit 410 is different, as explained below.

[0049] In S410, the secondary training data set generation unit 410 receives the primary training data set as input, generates synthetic data to be input to the model from primary input data, which are elements of the primary training data set, using a predetermined synthetic data generation method, and generates and outputs a secondary training data set whose elements are training data consisting only of secondary input data (hereinafter referred to as secondary training data) using the synthetic data as secondary input data.

[0050] According to an embodiment of the present invention, even if an attack to extract data used to train a trained model occurs when estimating an output for an input using the trained model, it is possible to prevent leakage of raw data, which is the data from which the data used to train the model is generated. Furthermore, by generating tertiary labels as raw data, it is possible to eliminate the influence of noise that occurs when generating synthetic data in a manner that ensures differential privacy, thereby enabling training that achieves both a stricter privacy budget and high model performance. Furthermore, by limiting the number of tertiary labels generated as raw data, it is possible to further tighten the privacy budget and reduce the amount of work required when manually generating tertiary labels.

[0051] <Additional Note> The processing of each unit of each of the above-described devices may be realized by a computer, in which case the processing content of the functions to be possessed by each device is described by a program. Then, by loading this program into the recording unit 2020 of the computer 2000 shown in Fig. 5 and operating the arithmetic processing unit 2010, the input unit 2030, the output unit 2040, the auxiliary recording unit 2025, etc., the processing functions of each of the above-described devices are realized on the computer.

[0052] The device of the present invention may, for example, be a single hardware entity that includes an input unit capable of inputting signals from outside the hardware entity, an output unit capable of outputting signals to outside the hardware entity, a communication unit to which a communication device (e.g., a communication cable) can be connected for communication with outside the hardware entity, a CPU (Central Processing Unit) that serves as an arithmetic processing unit, which may also include cache memory and registers, RAM and ROM that serve as memory, an external storage device such as a hard disk, and buses that connect the input unit, output unit, communication unit, CPU, RAM, ROM, and external storage device so that data can be exchanged between them. If necessary, the hardware entity may also be provided with a device (drive) that can read and write recording media such as a CD-ROM. An example of a physical entity equipped with such hardware resources is a general-purpose computer.

[0053] The external storage device of the hardware entity stores the programs required to realize the above-mentioned functions and the data required for processing these programs (the programs may be stored in a ROM, which is a read-only storage device, for example, instead of an external storage device). Data obtained by processing these programs is stored in RAM, the external storage device, etc. as appropriate.

[0054] In the hardware entity, each program stored in an external storage device (or ROM, etc.) and the data required to process each program are loaded into memory as needed, and interpreted, executed, and processed by the CPU as appropriate. As a result, the CPU realizes predetermined functions (each component represented as a "... unit," "... means," etc.). In other words, each component of the embodiments of the present invention may be configured by a processing circuitry.

[0055] As described above, when the processing functions of the hardware entities (apparatuses of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entities should have are described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

[0056] The program describing the processing contents can be recorded on a computer-readable recording medium, such as a non-transitory recording medium, specifically a magnetic recording device, an optical disk, or the like.

[0057] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0058] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or transferred from a server computer in its own non-transitory storage device, the auxiliary storage unit 2025. Then, when executing a process, the computer loads the program stored in its own non-transitory storage device, the auxiliary storage unit 2025, into the storage unit 2020 and executes processing in accordance with the loaded program. Alternatively, as another execution mode of this program, the computer may load the program directly from a portable recording medium into the storage unit 2020 and execute processing in accordance with the program. Furthermore, the computer may execute processing in accordance with the received program each time a program is transferred from the server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the processing function by issuing an execution instruction and obtaining the results, thereby executing the above-described processing through a so-called ASP (Application Service Provider) type service. Note that the program in this embodiment includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0059] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0060] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

Claims

1. The primary training data set is a set of training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data that serves as the input to the model, and primary labels, which are raw data that indicate the evaluation results of the model's output for the primary input data. a secondary training data set generation unit that generates a secondary training data set whose elements are training data consisting of pairs of secondary input data and secondary labels (hereinafter referred to as secondary training data) by a method of generating synthetic data to be an input of a model from primary input data constituting primary training data that is an element of the primary training data set, by a predetermined synthetic data generation method, generating raw data indicative of an evaluation result of an output of the model for the synthetic data, and setting a pair of the synthetic data and the raw data as a pair of secondary input data and a secondary label, or by a method of generating a pair of synthetic data (hereinafter referred to as first synthetic data) to be an input of a model and synthetic data (hereinafter referred to as second synthetic data) indicative of an evaluation result of an output of the model for the first synthetic data, by a predetermined synthetic data generation method from primary training data that is an element of the primary training data set, generating raw data indicative of an evaluation result of an output of the model for the first synthetic data using the second synthetic data, and setting a pair of the first synthetic data and the raw data as a pair of secondary input data and a secondary label; and a model learning unit that learns a model using the secondary training data set.

2. A secondary training data set generation unit that defines a primary training data set as a set whose elements are training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data that serves as an input to a model, and primary labels, which are raw data that indicate the evaluation results of the model's output for the primary input data, by a predetermined synthetic data generation method from the primary training data that is an element of the primary training data set, generates a pair of synthetic data (hereinafter referred to as first synthetic data) that serves as an input to the model and synthetic data (hereinafter referred to as second synthetic data) that indicates the evaluation results of the model's output for the first synthetic data, and generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting of a pair of secondary input data and secondary labels by a method in which the pair of the first synthetic data and the second synthetic data is treated as a pair of secondary input data and secondary labels; a tertiary training data set generation unit that generates a tertiary training data set whose elements are training data consisting of a pair of tertiary input data and a tertiary label (hereinafter referred to as tertiary training data) by a method of extracting secondary training data that is expected to be of great utility to improving model performance using a score calculated based on a distribution of secondary input data that constitutes secondary training data that is an element of the secondary training data set, or a score calculated based on an output for secondary input data that constitutes the secondary training data that is an element of the secondary training data set obtained using a model under training, generating raw data that indicates an evaluation result of the model's output for the secondary input data that constitutes the secondary training data using secondary labels that constitute the secondary training data, and treating the pair of the secondary input data and the raw data as a pair of tertiary input data and a tertiary label; and a model learning unit that learns a model using the tertiary training data set.

3. A primary training data set is a set whose elements are training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data that serves as an input to a model, and primary labels, which are raw data that indicate the evaluation results of the model's output for the primary input data; a secondary training data set generation unit that generates synthetic data that serves as an input to the model from primary input data that constitutes the primary training data, which is an element of the primary training data set, by a predetermined synthetic data generation method, and generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting only of secondary input data by using the synthetic data as secondary input data; a tertiary training data set generation unit that generates a tertiary training data set whose elements are training data consisting of pairs of tertiary input data and tertiary labels (hereinafter referred to as tertiary training data) by a method of extracting secondary input data that is expected to be of great utility for improving model performance using a score calculated based on a distribution of secondary input data that is an element of the secondary training data set, or a score calculated based on an output for secondary input data that is an element of the secondary training data set obtained using a model under training, generating raw data that indicates an evaluation result of the model's output for the secondary input data, and treating the pair of the secondary input data and the raw data as a pair of tertiary input data and tertiary labels; and a model learning unit that learns a model using the tertiary training data set.

4. The primary training data set is a set of training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data that serves as the input to the model, and primary labels, which are raw data that indicate the evaluation results of the model's output for the primary input data. a secondary training data set generation step in which the model learning device generates synthetic data to be an input of a model from primary input data constituting primary training data that is an element of the primary training data set, by a predetermined synthetic data generation method, generates raw data indicative of an evaluation result of an output of the model for the synthetic data, and sets the pair of the synthetic data and the raw data as a pair of secondary input data and a secondary label, or generates a pair of synthetic data (hereinafter referred to as first synthetic data) to be an input of a model from primary training data that is an element of the primary training data set, by a predetermined synthetic data generation method, and synthetic data (hereinafter referred to as second synthetic data) indicative of an evaluation result of an output of the model for the first synthetic data, uses the second synthetic data to generate raw data indicative of an evaluation result of an output of the model for the first synthetic data, and sets the pair of the first synthetic data and the raw data as a pair of secondary input data and a secondary label; a model learning step in which the model learning device learns a model using the secondary training data set.

5. A primary training data set is defined as a set whose elements are training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data to be input to a model, and primary labels, which are raw data indicating the evaluation results of the model's output for the primary input data, and a secondary training data set generation step in which the model learning device generates a pair of synthetic data (hereinafter referred to as first synthetic data) to be input to the model and synthetic data (hereinafter referred to as second synthetic data) indicating the evaluation results of the model's output for the first synthetic data from the primary training data, which is an element of the primary training data set, by a predetermined synthetic data generation method, and generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting of a pair of secondary input data and secondary labels by a method in which the pair of the first synthetic data and the second synthetic data is treated as a pair of secondary input data and secondary labels; a tertiary training data set generation step in which the model learning device extracts secondary training data expected to be of great utility for improving model performance using a score calculated based on a distribution of secondary input data constituting secondary training data that is an element of the secondary training data set, or a score calculated based on an output for secondary input data constituting secondary training data that is an element of the secondary training data set obtained using a model under learning, generates raw data indicating an evaluation result of the model's output for the secondary input data constituting the secondary training data using secondary labels constituting the secondary training data, and generates a tertiary training data set whose elements are training data consisting of a pair of tertiary input data and a tertiary label (hereinafter referred to as tertiary training data) by using a method in which the model learning device treats the tertiary training data set as a pair of tertiary input data and a tertiary label; and a model learning step in which the model learning device learns a model using the tertiary training data set.

6. A primary training data set is defined as a set whose elements are training data (hereinafter referred to as primary training data) consisting of a pair of primary input data, which is raw data that serves as an input to a model, and primary labels, which are raw data that indicate the evaluation results of the model's output for the primary input data; a secondary training data set generation step in which the model learning device generates synthetic data, which serves as an input to the model, from primary input data that constitutes the primary training data, which is an element of the primary training data set, by a predetermined synthetic data generation method, and generates a secondary training data set whose elements are training data (hereinafter referred to as secondary training data) consisting only of secondary input data by using the synthetic data as secondary input data; a tertiary training data set generation step in which the model learning device extracts secondary input data that is expected to be of great utility for improving model performance using a score calculated based on a distribution of secondary input data that is an element of the secondary training data set, or a score calculated based on the output for secondary input data that is an element of the secondary training data set obtained using a model under learning, generates raw data that indicates an evaluation result of the model's output for the secondary input data, and generates a tertiary training data set whose elements are training data consisting of a pair of tertiary input data and a tertiary label (hereinafter referred to as tertiary training data) by using a method in which the model learning device treats the pair of the secondary input data and the raw data as a pair of tertiary input data and a tertiary label; and a model learning step in which the model learning device learns a model using the tertiary training data set.

7. A program for causing a computer to function as the model learning device according to any one of claims 1 to 3.

Citation Information

Patent Citations

  • Privacy protection data providing system and privacy protection data providing method

    JP2018097467A

  • On-device activity recognition

    JP2022120775A

  • Model generation system, secure computation device, analysis device, model generation method, and program

    WO2022264372A1