Machine learning model providing system, machine learning model providing method

Abstract

A machine learning model providing system and a machine learning model providing method are proposed, which can provide a machine learning model that uses personal information data without disclosing the personal information data to the user, and on the other hand, curb the provision of machine learning models that do not adequately protect personal information. The machine learning model providing system involved in the present disclosure includes a storage device that stores a collection of personal information data and a data directory, and an information processing device. The information processing device performs: obtaining the designation of metadata used in the generation of the machine learning model, the designation of the first data range, and the processing of the machine learning logic; the processing of generating the machine learning model according to the logic based on the personal information data corresponding to the designated metadata and the first data range; the processing of calculating and determining risks with respect to the generated machine learning model; and the processing of stopping the provision of the generated machine learning model when the risk is determined to be above a predetermined threshold.

Description

Technical Field

[0001] The present disclosure relates to a machine learning model providing system and a machine learning model providing method for generating a machine learning model and providing the generated machine learning model. Background Art

[0002] Patent Document 1 discloses a privacy-preserving data provision system that anonymizes a deep learning model obtained by applying a deep learning algorithm. This privacy-preserving data provision system applies a Laplace-distributed error to each parameter value of weight parameters and bias parameters included in the deep learning model. If the error for each parameter provided by the Laplace distribution exceeds a threshold range represented by a maximum and minimum value, the system limits the value to the threshold range.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-97467 Summary of the Invention

[0006] As machine learning advances and the effective use of various data grows, the usefulness of utilizing personal information data is gaining attention. Furthermore, as the roles of data collectors and users become increasingly separate, as is the case with smart cities, the demand for users to utilize the personal information collected by data collectors is expected to increase.

[0007] However, from the perspective of personal information protection, the use of personal information data in machine learning has not been developed due to the risk of leakage and the difficulty of providing it to third parties.

[0008] The privacy-protected data provision system disclosed in Patent Document 1 generates an anonymous model (machine learning model) that has undergone a predetermined anonymization process. However, the generated machine learning model is not evaluated in conjunction with personal information protection, leading to concerns that a machine learning model may be provided that does not adequately protect personal information.

[0009] The present disclosure is completed in view of the above-mentioned problems, and its purpose is to propose a machine learning model providing system and a machine learning model providing method, which can provide a machine learning model that utilizes personal information data without disclosing the personal information data to the user, and on the other hand curb the provision of machine learning models that do not adequately protect personal information.

[0010] The first disclosure relates to a machine learning model providing system that generates a machine learning model and provides the generated machine learning model.

[0011] The machine learning model providing system includes: a storage device that stores a collection of personal information data and a data directory of metadata related to the personal information data; and an information processing device.

[0012] The information processing device performs: acquisition processing, obtaining the designation of the metadata used in the generation of the machine learning model, the designation of the first data range that provides the data range related to the personal information data, and the logic of machine learning; model generation processing, generating the machine learning model in accordance with the logic based on the personal information data corresponding to the designated metadata and the first data range; determination risk calculation processing, calculating, with respect to the generated machine learning model, a determination risk representing the risk of determining an individual based on the output; and provision management processing, stopping the provision of the generated machine learning model when the determination risk becomes above a predetermined threshold.

[0013] The second disclosure relates to a machine learning model providing system that is different from the machine learning model providing system of the first disclosure and further includes the following features.

[0014] Each of the personal information data is associated with one or more pieces of ID information indicating a specific individual.

[0015] In the determined risk calculation process, the information processing device selects the personal information data as input data to the generated machine learning model, obtains output data as the output of the machine learning model for the input data, generates corresponding associated output information representing the ID information and a set of output data related to the input data associated with the ID information, and calculates the determined risk based on the associated output information.

[0016] The third disclosure relates to a machine learning model providing system that is different from the machine learning model providing system of the second disclosure and further includes the following features.

[0017] The output data is divided into categories.

[0018] In the definite risk calculation process, the information processing device calculates the number of elements u(i, j) of the set of output data that is included in the category and corresponds to the ID information in the associated output information for each combination (i, j) of the ID information and the category, and calculates the definite risk IR using the following formulas (1) and (2):

[0019]

Mathematical formula 1

[0020]

[0021]

Mathematical formula 2

[0022]

[0023] The fourth disclosure relates to a machine learning model providing system according to the second or third disclosure, further including the following features.

[0024] The information processing device generates a second data range that provides the data range in the definite risk calculation process, and selects the personal information data corresponding to the second data range as the input data.

[0025] The fifth disclosure relates to the machine learning model providing system according to the fourth disclosure, and further includes the following features.

[0026] The second data range provides a plurality of data ranges according to a plurality of partitions of different scales.

[0027] Disclosed in paragraph 6 is a method for providing a machine learning model that generates a machine learning model and provides the generated machine learning model.

[0028] In this machine learning model providing method, a collection of personal information data and a data directory of metadata related to the personal information data are collected and stored, the designation of the metadata used in the generation of the machine learning model, the designation of a first data range providing a data range related to the personal information data, and the logic of machine learning are obtained, the machine learning model is generated according to the logic based on the personal information data corresponding to the designated metadata and the first data range, and with respect to the generated machine learning model, a determined risk representing the risk of an individual determined based on the output is calculated, and when the determined risk is less than a predetermined threshold, the generated machine learning model is provided, and when the determined risk becomes above the threshold, the provision of the generated machine learning model is stopped.

[0029] The seventh disclosure relates to a method for providing a machine learning model according to the sixth disclosure, and further includes the following features.

[0030] Regarding each of the personal information data, it is associated with one or more ID information representing a specific individual, the personal information data is selected as input data to the generated machine learning model, output data as the output of the machine learning model for the input data is obtained, corresponding associated output information representing the ID information and the set of output data related to the input data associated with the ID information is generated, and the determined risk is calculated based on the associated output information.

[0031] The eighth disclosure relates to the machine learning model providing method according to the seventh disclosure, and further includes the following features.

[0032] The output data is divided into categories. For each combination (i, j) of the ID information and the category, the number of elements u(i, j) of the set of output data that is included in the category and corresponds to the ID information in the associated output information is calculated. The determined risk IR is calculated using the following formulas (1) and (2).

[0033]

Mathematical formula 3

[0034]

[0035]

Mathematical formula 4

[0036]

[0037] The machine learning model providing system and method disclosed herein enable the provision of machine learning models that utilize personal information data. Furthermore, if the risk is determined to be above a predetermined threshold, provision of the generated machine learning model is stopped. This prevents the provision of machine learning models whose outputs are likely to identify individuals and whose personal information protection is insufficient. Furthermore, the risk of personal information data leakage can be reduced.

[0038] Furthermore, the personal information data corresponding to the metadata and the first data range is used to generate the machine learning model. This eliminates the need to disclose personal information data to users. Furthermore, it is possible to provide machine learning models that are generated directly using personal information data, rather than statistically generated information. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a conceptual diagram used to illustrate the outline of the platform for building a machine learning model provision system.

[0040] Figure 2 This is a block diagram showing the structure of the machine learning model providing system involved in this embodiment.

[0041] Figure 3 This is a diagram showing an example of personal information data stored in a storage device.

[0042] Figure 4 This is a block diagram showing a configuration related to processing executed when the information processing apparatus acquires request information regarding provision of a data directory.

[0043] Figure 5 This is a flowchart showing a data catalog providing method implemented by the machine learning model providing system involved in this embodiment.

[0044] Figure 6 This is a block diagram showing a structure related to processing performed when an information processing device obtains request information regarding provision of a machine learning model.

[0045] Figure 7 This is a flowchart showing a machine learning model providing method implemented by the machine learning model providing system involved in this embodiment.

[0046] Figure 8 This is a diagram showing an example of input data, output data corresponding to the input data, and generated related output information.

[0047] Figure 9 It shows that the generated Figure 8 The diagram shows u(i, j) for each combination (i, j) of ID information and category in the case of associated output information.

[0048] Figure 10 1 is a diagram showing an example of u(i, j) and r(i, j) calculated for the u(i, j).

[0049] Figure 11 This is a flowchart showing the processing executed by the definite risk calculation processing unit.

[0050] (Explanation of Symbols)

[0051] 1: Platform; 2: Data provider; 3: User; 10: Machine learning model providing system; 100: Information processing device; 110: Catalog confirmation processing unit; 120: Model generation processing unit; 130: Risk determination calculation processing unit; 140: Provision management processing unit; 200: Storage device; 210: Personal information data; 220: Data catalog; IR: Risk determination. DETAILED DESCRIPTION

[0052] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, when the number, quantity, amount, range, etc. of each element is mentioned in the embodiments shown below, the ideas involved in the present disclosure are not limited to the mentioned quantities, except for the cases where it is specifically stated or the cases where the quantity is clearly determined in principle. In addition, the structures and the like described in the embodiments shown below are not necessarily required in the ideas involved in the present disclosure, except for the cases where it is specifically stated or the cases where it is clearly determined in principle. In addition, in each figure, the same symbol is added to the same or equivalent part, and its repeated description is appropriately simplified or omitted.

[0053] 1. Summary

[0054] The machine learning model providing system involved in this embodiment builds a platform for providing machine learning models in response to user requests. Figure 1 This is a conceptual diagram for explaining an overview of the platform 1 constructed by the machine learning model providing system 10.

[0055] Platform 1 stores data including personal information data provided by data provider 2 (hereinafter referred to as "personal information data"). User 3 receives a machine learning model by specifying the data to be used and the machine learning logic. The machine learning model received is generated based on the specified data and the specified logic.

[0056] Data Provider 2 represents the entity that collects and provides data, and is not limited to humans. Examples of Data Provider 2 include individuals, businesses, retail stores, surveillance cameras, user terminals, and automobiles. Examples of personal information data collected and provided by these Data Providers 2 include personal attributes (age, height, weight, etc.), customer data, purchase information (purchased items, purchase date and time, etc.), surveillance video data, terminal usage information, and mobile data. Examples of personal information data include location information, behavioral information, related media information, and physical information.

[0057] Furthermore, data may be provided actively or automatically. For example, it may be provided actively by operation of a predetermined device, or it may be provided continuously or automatically at predetermined intervals through communication via the Internet or a cable.

[0058] The platform 1 is constructed by a machine learning model providing system 10. The machine learning model providing system 10 includes an information processing device 100 and a storage device 200.

[0059] The information processing device 100 is connected to the storage device 200 and performs machine learning using the data stored in the storage device 200. The machine learning model generated by the machine learning is then output. The data used and the machine learning logic are specified by the user 3.

[0060] Information processing device 100 is, for example, a server configured on the Internet. In this case, user 3 specifies the data and logic to be used via communication on the Internet and receives the provision of a machine learning model. Typically, user 3 specifies the data and logic to be used and receives the provision of a machine learning model via an application or a browser.

[0061] As another example, the information processing device 100 may be a dedicated terminal or a general-purpose computer. In this case, the user 3 typically operates an HMI device (touch panel, keyboard, display, etc.) connected to the dedicated terminal or general-purpose computer to specify the data and logic to be used and receive the machine learning model.

[0062] Furthermore, the information processing device 100 may be a system composed of a plurality of computers. For example, a plurality of servers may cooperate to execute a specific process.

[0063] The storage device 200 stores personal information data provided by the data provider 2. The storage device 200 is, for example, a data server configured on the Internet. In this case, the storage device 200 acquires and stores personal information data from the data provider 2 via communication on the Internet.

[0064] The information processing device 100 and the storage device 200 are connected in a manner that allows information to be transmitted between them. For example, the information processing device 100 and the storage device 200 are connected via the Internet. Alternatively, the information processing device 100 and the storage device 200 are electrically or optically connected using a cable. Alternatively, the information processing device 100 and the storage device 200 include wireless devices and are connected to each other via wireless communication.

[0065] The information processing device 100 and the storage device 200 may be integrally configured. For example, the storage device 200 may be an auxiliary storage device of the information processing device 100 .

[0066] In platform 1 constructed by such a machine learning model providing system 10, personal information data must not be disclosed to third parties from the perspective of personal information protection. Furthermore, if the output of a generated machine learning model is highly likely to identify an individual, the machine learning model cannot be said to provide adequate personal information protection. For example, if the determined output of a machine learning model tends to output personal information data related to a specific individual as input, there is a concern that the determined output may have a meaning equivalent to that specific individual.

[0067] Therefore, the machine learning model providing system 10 involved in this embodiment generates a machine learning model that utilizes personal information data without disclosing the personal information data to user 3. Furthermore, the generated machine learning model calculates the risk of identifying an individual based on the output (hereinafter also referred to as "identified risk"), and if the identified risk exceeds a predetermined threshold, the provision of the generated machine learning model is stopped. The following describes an embodiment of the machine learning model providing system 10 and a method for providing a machine learning model.

[0068] 2. Machine Learning Model Provisioning System

[0069] 2-1. Structure

[0070] Figure 2 1 is a block diagram showing the structure of a machine learning model providing system 10 according to this embodiment. The machine learning model providing system 10 includes an information processing device 100 and a storage device 200 .

[0071] The information processing device 100 obtains information related to the request (request information) and performs processing corresponding to the request information. The processing result is then output. In the machine learning model providing system 10 according to this embodiment, the request to the information processing device 100 includes providing the data catalog 220 and providing the machine learning model.

[0072] The data directory 220 is a data directory containing metadata related to the personal information data 210, and is stored in the storage device 200. Metadata includes, for example, table names (such as "Personal Attributes," "Purchase Information," "Location Information," "Behavior Information," "Related Media," and "Physical Information"), data storage locations (such as URLs), data items (such as "Gender," "Age," "Address," "Product Name," "Price," and "Location"), data item formats (such as character strings, numerical values, and time), and data value ranges (such as minimum and maximum values).

[0073] The data directory 220 is updated by the information processing device 100. The information processing device 100 obtains the personal information data 210 from the storage device 200 and extracts metadata related to the personal information data 210. The information processing device 100 then updates the data directory 220 based on the extracted metadata.

[0074] For example, the data directory 220 is updated at every predetermined period. In this case, the information processing device 100 obtains information about the time when the data directory 220 was updated (hereinafter also referred to as the "update time") from the storage device 200, and determines whether the data directory 220 needs to be updated based on the update information.

[0075] The information processing device 100 includes a catalog confirmation processing unit 110, a model generation processing unit 120, a determined risk calculation processing unit 130, and a provision management processing unit 140. These units may be implemented as part of a program that describes the processes executed by the information processing device 100, or may be implemented by multiple computers that constitute the information processing device 100.

[0076] The catalog confirmation processing unit 110 performs processing related to updating and providing the data catalog 220. The model generation processing unit 120, the determined risk calculation processing unit 130, and the provision management processing unit 140 perform processing related to generating and providing machine learning models.

[0077] When information processing device 100 receives request information regarding provision of data catalog 220, it outputs data catalog 220 as an execution result. Furthermore, when information processing device 100 receives request information regarding provision of a machine learning model, it outputs the machine learning model as an execution result. Details of the processing performed by information processing device 100 upon receipt of each request information will be described later.

[0078] The personal information data 210 stored in the storage device 200 is associated with one or more pieces of ID information indicating a specific individual. Figure 3 2 is a diagram showing an example of personal information data 210. Figure 3 In (A), (B), and (C), personal information data 210 related to personal attributes, purchase information, and surveillance images are shown in tables. The table shows items in the first row, and each of the following rows shows one piece of personal information data 210.

[0079] like Figure 3 As shown, each personal information data 210 is associated with one or more ID information. Figure 3 In the example shown, the ID information is a character string corresponding to a specific individual. Figure 3 The ID information in the personal attributes shown in (A) indicates, for example, a person having each personal attribute. Figure 3 The ID information in the purchase information shown in (B) indicates, for example, the purchaser. Figure 3 The ID information in the surveillance video shown in (C) indicates, for example, the person shown in the video file.

[0080] Furthermore, the data of each item may be numerically represented. For example, the data of the item "gender" may be numerically represented by setting "male" to 0 and "female" to 1.

[0081] 2-2. Provision of data catalog

[0082] Hereinafter, a process (hereinafter also referred to as “data directory providing process”) executed when the information processing device 100 acquires request information regarding provision of the data directory 220 will be described. Figure 4 It is a block diagram showing a structure related to data directory providing processing.

[0083] After receiving the request information, the directory verification processing unit 110 obtains the update time of the data directory 220 and checks whether a certain period of time has passed since the update time. The certain period of time can be appropriately determined based on the environment in which the machine learning model providing system 10 is used, the personal information data 210 being processed, and other factors.

[0084] Then, when a certain period of time has passed since the update time, the data directory 220 is updated. At this time, the directory confirmation processing unit 110 obtains the personal information data 210 and extracts metadata from the personal information data 210. The directory confirmation processing unit 110 updates the data directory 220 based on the extracted metadata. The directory confirmation processing unit 110 then outputs the updated data directory 220.

[0085] If the predetermined period has not elapsed from the update time to the current time, the directory confirmation processing unit 110 acquires and outputs the data directory 220 without updating the data directory 220 .

[0086] Figure 5 This is a flowchart showing a data catalog providing method implemented by the machine learning model providing system 10 involved in this embodiment. Figure 5 The flowchart shown starts when the information processing device 100 obtains request information regarding provision of the data directory 220 , and the process is executed in each predetermined cycle.

[0087] In step S100, the directory confirmation processing unit 110 obtains the update time of the data directory 220. After step S100, the process proceeds to step S110.

[0088] In step S110, the directory confirmation processing unit 110 determines whether a certain period of time has elapsed from the update time obtained in step S100 to the current time. If the certain period of time has elapsed from the update time to the current time (step S110: "Yes"), the process proceeds to step S120. If the certain period of time has not elapsed from the update time to the current time (step S110: "No"), the process proceeds to step S140.

[0089] In step S120, the directory confirmation processing unit 110 obtains the personal information data 210 and extracts metadata of the personal information data 210. After step S120, the process proceeds to step S130.

[0090] In step S130, the directory confirmation processing unit 110 updates the data directory 220 based on the metadata extracted in step S120. After step S130, the process proceeds to step S140.

[0091] In step S140, the directory confirmation processing unit 110 obtains and outputs the data directory 220. After step S140, the process ends.

[0092] As described above, during the data directory provision process, the data directory 220 is updated when a certain time has passed since the data directory 220 was last updated. This allows the information in the data directory 220 to be provided to be kept up to date while minimizing the load on the data directory 220.

[0093] 2-3. Provision of Machine Learning Models

[0094] The following describes a process (hereinafter also referred to as “machine learning model provision process”) executed when the information processing device 100 obtains request information regarding provision of a machine learning model. Figure 6 is a block diagram showing the structure associated with machine learning model provision processing.

[0095] By requesting information, the data range (hereinafter also referred to as the "first data range") related to the metadata and personal information data 210 used in generating the machine learning model and the logic of the machine learning are specified.

[0096] Here, the personal information data 210 corresponding to the specified metadata and first data range is used to generate the machine learning model. In other words, the request information does not directly specify the data used to generate the machine learning model. Furthermore, the machine learning model providing system 10 does not need to disclose the personal information data 210 to user 3 in order to facilitate user 3's request.

[0097] On the other hand, the user 3 can designate metadata and the first data range from the data catalog 220 by receiving the provision of the data catalog 220 .

[0098] The specification of metadata typically specifies the data item. The specification of the first data range typically specifies the period related to the time point of data acquisition. For example, as metadata, the personal attributes "gender" and "age" are specified, and as the first data range, the period of 2020 / 10 / 01-2020 / 11 / 31 is specified with respect to the acquisition time point. In this case, the data used to generate the machine learning model becomes data related to "gender" and "age" within the range of the acquisition time point of 2020 / 10 / 01-2020 / 11 / 31.

[0099] As other examples, a case where the metadata is designated as a table name and a case where the first data range is designated as a range of locations to acquire data are described.

[0100] Machine learning logic typically includes learning logic and evaluation logic. Furthermore, it may also include other machine learning logic, such as transformation logic for using data synthesis parameters as input for learning data. Machine learning logic can be specified by directly inputting source code or by selecting from a library provided by the machine learning model providing system 10.

[0101] Here, typically, the content of the output of the machine learning model (such as a label, etc.) is determined by learning logic.

[0102] Furthermore, a plurality of designated first data ranges may be provided according to the logic of machine learning. For example, a data range related to data used for learning and a data range related to data used for verification may be provided independently.

[0103] The model generation processing unit 120 obtains the personal information data 210 corresponding to the metadata and the first data range specified in the request information. The model generation processing unit 120 then generates a machine learning model based on the machine learning logic specified in the request information. The generated machine learning model is then passed to the specific risk calculation processing unit 130 and the provision management processing unit 140.

[0104] The model generation processing unit 120 typically performs processing related to source code interpretation, learning, verification, and evaluation. However, when machine learning logic is selected from a library, processing related to source code interpretation may not be performed.

[0105] The deterministic risk calculation unit 130 calculates the deterministic risk (deterministic risk calculation process) based on the machine learning model generated by the model generation unit 120. This deterministic risk calculation process uses personal information data 210 as input and verifies the output of the generated machine learning model. Details of the deterministic risk calculation process will be described later. The calculated deterministic risk is then transmitted to the provision management unit 140.

[0106] The provision management processing unit 140 determines whether to provide the machine learning model generated by the model generation processing unit 120 based on the determined risk calculated by the determined risk calculation processing unit 130. If the determined risk is less than a predetermined threshold, the provision management processing unit 140 provides the machine learning model. If the determined risk exceeds the predetermined threshold, the provision management processing unit 140 stops providing the machine learning model. Furthermore, when the provision of the machine learning model is stopped, a warning regarding the stopped provision of the machine learning model is issued.

[0107] In addition, it can also be configured to delete the generated machine learning model when the provision of the machine learning model is stopped.

[0108] Figure 7 1 is a flowchart illustrating a machine learning model providing method implemented by the machine learning model providing system 10 according to this embodiment. Figure 7 The flowchart shown starts when the information processing device 100 obtains request information regarding provision of a machine learning model, and the process is executed for each predetermined period.

[0109] In step S200, the model generation processing unit 120 obtains the personal information data 210 corresponding to the metadata and the first data range specified by the request information. After step S200, the process proceeds to step S210.

[0110] In step S210, the model generation processing unit 120 generates a machine learning model based on the personal information data 210 acquired in step S200 and in accordance with the machine learning logic specified by the request information. After step S210, the process proceeds to step S220.

[0111] In step S220, the deterministic risk calculation unit 130 performs deterministic risk calculation processing on the machine learning model generated in step S210 to calculate the deterministic risk. The details of the deterministic risk calculation processing will be described later. After step S220, the process proceeds to step S230.

[0112] In step S230, the provision management processing unit 140 determines whether the determined risk calculated in step S220 is greater than a predetermined threshold. If the determined risk is greater than the predetermined threshold (step S230: "Yes"), the process proceeds to step S240. If the determined risk is less than the predetermined threshold (step S230: "No"), the process proceeds to step S250.

[0113] In step S240, the provision management processing unit 140 stops providing the machine learning model generated in step S210. Furthermore, a warning regarding the stopped provision of the machine learning model is issued. After step S240, the process ends.

[0114] In step S250, the provision management processing unit 140 provides the machine learning model generated in step S210. After step S250, the process ends.

[0115] As described above, during the machine learning model provision process, if the risk is determined to be above a predetermined threshold, provision of the generated machine learning model is stopped. This prevents the provision of machine learning models whose outputs are likely to identify individuals and whose personal information protection is insufficient.

[0116] 3. Determine the risk calculation process

[0117] The following describes the finalized risk calculation process performed by the finalized risk calculation processing unit 130 .

[0118] First, the risk calculation processing unit 130 determines that the personal information data 210 is selected as input data to the machine learning model generated by the model generation processing unit 120 .

[0119] Here, the risk calculation processing unit 130 determines that it generates a second data range that provides a data range related to the personal information data 210, and selects the personal information data 210 corresponding to the second data range as input data. Furthermore, since the input data for the generated machine learning model is selected, the second data range provides a data range with the same content as the first data range. For example, if the first data range covers a period related to the acquisition time, the second data range also covers a period related to the acquisition time. Furthermore, the selected personal information data 210 is data related to metadata related to the requested information.

[0120] The second data range is typically generated so as not to overlap with the first data range, in order to avoid duplication with data used in learning, verification, and evaluation.

[0121] Furthermore, the second data range may provide multiple data ranges based on multiple partitions of different scales. For example, if the second data range is a data range for a period related to the acquisition time point, the second data range may provide data ranges for minutes, hours, days, and months.

[0122] By providing multiple data ranges for the second data range in this manner, based on multiple partitions at different scales, it is possible to obtain outputs from machine learning models generated for input data at different scales. This can reduce the risk of underestimating the risk due to differences in scale.

[0123] Next, the risk calculation processing unit 130 determines the output data of the generated machine learning model for the input data. In addition, when input conversion logic is provided as request information, it can be appropriately applied to the input data.

[0124] Next, the specific risk calculation processing unit 130 generates associated output information indicating correspondence between the ID information and a set of output data related to the input data associated with the ID information. Figure 8 This is a conceptual diagram for explaining related output information. Figure 8 Examples of input data, output data corresponding to the input data, and generated related output information are shown. In addition, numbers are examples for indicating the correspondence between input data and output data.

[0125] exist Figure 8 In the example, the associated output information shows the correspondence between each of the three IDs (jfda093jt, gq409jf32, and 65k262f) and the set of output data associated with each ID. Here, the associated output information shows that output 4 is particularly biased toward cases where the ID is jfda093jt. In other words, there is a concern that output 4 is synonymous with a specific individual with the ID jfda093jt. Furthermore, there is a high probability that the output identifies the individual. Thus, the associated output information provides information regarding identified risks.

[0126] Therefore, the definite risk calculation processing unit 130 calculates the definite risk based on the associated output information. Here, the calculation of the definite risk is performed, for example, as follows.

[0127] First, the output data is divided into categories. The method of providing categories can also be appropriately performed based on the output of the machine learning model.

[0128] For example, if the output is a one-dimensional continuous value, the number of categories can be determined using the Sturges formula, and the level width can be determined based on the number of categories and the maximum and minimum output values ​​to provide a histogram. Furthermore, each level can also be used as a category.

[0129] Alternatively, when the output is an image, the distance between two images is predefined, and agglomerative hierarchical clustering is performed using the distance to classify each output into a certain number of groups. Furthermore, each group can also be used as a class.

[0130] Alternatively, when the output is provided by discrete values ​​or labels, each value or label can also be considered as a category.

[0131] Next, for each combination (i, j) of ID information and category, the number of elements u(i, j) in the set of output data corresponding to the ID information in the associated output information and included in the category is calculated. Here, i represents each ID information and j represents each category.

[0132] Figure 9 Shows the generation Figure 8 In the case of the associated output information shown in FIG, u(i, j) is obtained for each combination (i, j) of ID information and category. Figure 9 , the values ​​of each output are considered categories.

[0133] Then, the risk IR is calculated using the following equations (1) and (2).

[0134]

Mathematical formula 5

[0135]

[0136]

Mathematical formula 6

[0137]

[0138] exist Figure 10 , an example of u(i, j) and r(i, j) calculated for this u(i, j) are shown. In this case, the determined risk IR becomes 4 / 5 of the maximum value of r(i, j).

[0139] Furthermore, when multiple data ranges are provided for the second data range (for example, when multiple data ranges are provided based on multiple partitions of different scales), the definite risk calculation processing unit 130 may calculate the definite risk IR for each of the data ranges. Furthermore, for each definite risk IR, a weighted average value calculated using the number of ID information items associated with each data range may be used to calculate the final definite risk.

[0140] Figure 11The process ( Figure 7 Flowchart of step S220 in FIG.

[0141] In step S221, the final risk calculation processing unit 130 generates the second data range. After step S221, the process proceeds to step S222.

[0142] In step S222, the definite risk calculation processing unit 130 selects the personal information data 210 corresponding to the second data range generated in step S221 as input data. After step S222, the process proceeds to step S223.

[0143] In step S223, the specific risk calculation processing unit 130 obtains output data for the generated machine learning model with respect to the input data selected in step S222. After step S223, the process proceeds to step S224.

[0144] In step S224, the specific risk calculation processing unit 130 generates relevant output information based on the output data acquired in step S223. After step S224, the process proceeds to step S225.

[0145] In step S225, the definite risk calculation processing unit 130 calculates the definite risk IR based on the correlation output information generated in step S224. After step S225, the process ends.

[0146] 4. Feedback to users

[0147] The provision management processing unit 140 may also be configured to return the following feedback to the user 3 together with the issuance of a warning when it is determined that the risk IR has become greater than a predetermined threshold and the provision of the machine learning model is stopped.

[0148] One feedback is the category where the feedback r(i, j) is above the threshold. Figure 10 For example, when the threshold is set to 0.5, r(i, j) associated with the combination of C and the third category exceeds the threshold. Therefore, the third category is fed back. This can be eliminated by changing the output label or category. Therefore, it is also recommended to further change the output label or category in the machine learning logic as feedback.

[0149] 1 Feedback is feedback that has been grouped into a single category of individuals (ID information). Figure 10 This corresponds to a combination of D and the second category. This can be expected to be eliminated by expanding the data range associated with the first data range. Therefore, further expanding the data range associated with the first data range can also be recommended as feedback.

[0150] 5. Effect

[0151] As described above, the machine learning model providing system 10 and the machine learning model providing method according to this embodiment can provide a machine learning model that utilizes personal information data 210. Furthermore, if the risk is determined to be above a predetermined threshold, provision of the generated machine learning model is stopped. This prevents the provision of machine learning models whose outputs are likely to identify individuals and whose personal information protection is insufficient. Furthermore, the risk of personal information data leakage can be reduced.

[0152] Furthermore, the personal information data 210 corresponding to the metadata and the first data range becomes data used in generating the machine learning model. This eliminates the need to disclose the personal information data 210 to the user 3. Furthermore, a machine learning model generated directly using the personal information data 210, rather than using statistical information, can be provided.

[0153] Furthermore, the second data range may also provide multiple data ranges according to multiple partitions of different scales. This can reduce the possibility that the risk is calculated to be too small due to differences in scale.

Claims

1. A machine learning model providing system, which generates a machine learning model and provides the generated machine learning model, characterized in that: include: A storage device storing a collection of personal information data and a data directory of metadata related to the personal information data; as well as information processing device, The information processing device performs: Acquisition processing for acquiring a designation of the metadata used in generating the machine learning model, a designation of a first data range providing a data range related to the personal information data, and machine learning logic; Model generation processing, generating the machine learning model according to the logic based on the personal information data corresponding to the specified metadata and the first data range; a determined risk calculation process of calculating, with respect to the generated machine learning model, a determined risk indicating a risk of determining an individual based on the output; as well as providing a management process for stopping provision of the generated machine learning model when the determined risk exceeds a predetermined threshold, wherein: Each of the personal information data is associated with one or more ID information indicating a specific individual. In the definite risk calculation process, the information processing device selecting the personal information data as input data to the generated machine learning model, obtaining output data that is the output of the machine learning model with respect to the input data, Divide the output data into categories, generating associated output information indicating correspondence between the ID information and a set of the output data associated with the input data and associated with the ID information; For each combination (i, j) of the ID information and the category, the number of elements u(i, j) of the set of output data that is included in the category and corresponds to the ID information in the associated output information is calculated. The determined risk IR is calculated based on the associated output information using the following formulas (1) and (2): 。 2. The machine learning model providing system according to claim 1, wherein: In the definite risk calculation process, the information processing device A second data range is generated to provide the data range, and the personal information data corresponding to the second data range is selected as the input data.

3. The machine learning model providing system according to claim 2, wherein: The second data range provides a plurality of data ranges according to a plurality of partitions of different sizes.

4. A method for providing a machine learning model, generating a machine learning model and providing the generated machine learning model, characterized in that: The machine learning model provides a method Collect and store a collection of personal information data and a data directory about metadata related to the personal information data, Obtaining the designation of the metadata used in generating the machine learning model, the designation of a first data range providing a data range related to the personal information data, and the logic of machine learning, generating the machine learning model according to the logic based on the personal information data corresponding to the specified metadata and the first data range, With respect to the generated machine learning model, calculating a determined risk representing a risk determined for an individual based on the output, When the determined risk is less than a predetermined threshold, the generated machine learning model is provided, and when the determined risk becomes greater than the threshold, the generated machine learning model is stopped from being provided, wherein, Each of the personal information data is associated with one or more ID information indicating a specific individual. selecting the personal information data as input data to the generated machine learning model, obtaining output data that is the output of the machine learning model with respect to the input data, Divide the output data into categories, generating associated output information indicating correspondence between the ID information and a set of the output data associated with the input data and associated with the ID information; For each combination (i, j) of the ID information and the category, the number of elements u(i, j) of the set of output data that is included in the category and corresponds to the ID information in the associated output information is calculated. The determined risk IR is calculated based on the associated output information using the following formulas (1) and (2): 。

Images