Machine learning system, server apparatus, client apparatus, machine learning method, and machine learning program

Abstract

The machine learning system (500) includes a client device (200) with inherent learning data and a server device (100) with supervised data. Both the client device (200) and the server device (100) maintain AI models. Furthermore, in the machine learning system (500), the learning data of the client device (200) and the supervised data of the server device (100) are kept separate to advance learning. The server device (100) includes an unauthorized client detection unit (2530) that identifies unauthorized client devices that are attempting to interfere with the learning of the AI ​​model in the server device (100).

Description

Technical Field

[0001] This disclosure relates to machine learning systems, server devices, client devices, machine learning methods, and machine learning programs. Background Technology

[0002] Federated learning is a technique that aggregates learned models from multiple clients to generate a large-scale AI model. Hereinafter, the learned model will be referred to as an "AI model." AI is an abbreviation for Artificial Intelligence. Federated learning is characterized by the following: each client possesses its own learning data, and this data is not publicly shared. Federated learning can be categorized into horizontal federated learning and vertical federated learning.

[0003] In federated learning, there is a threat of learning-impairing attacks against the server, which utilize illegitimate data that does not conform to the normal learning methods. Examples of server-side learning-impairing attacks include malicious, unauthorized clients. Particularly in large-scale horizontal federated learning, learning solely from biased data across clients can reduce the reliability of the AI ​​model generated on the server.

[0004] Patent document 1 discloses the following technique: In horizontal federated learning, the server averages the models uploaded from each client and excludes clients whose models deviate significantly from the average. This technique is disclosed to prevent a decrease in the reliability of AI models.

[0005] Non-patent document 1 discloses a defense method in longitudinal federated learning that uses encryption to defend against attacks that misidentify backdoor images containing specific patterns as other specific inference results.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent No. 6231944

[0009] Non-patent literature

[0010] Non-patent literature 1: Zou, Tianyuan, et al. "Defending batch-level labelinference and replacement attacks in vertical federated learning." IEEETransactions on Big Data, 2022. Summary of the Invention

[0011] The problem that the invention aims to solve

[0012] The defense technique against learning-impeding attacks against servers described in Patent Document 1 can only be applied to horizontal federated learning, which is similar to information sent from the client to the server. Therefore, there is a problem that it cannot be applied to vertical federated learning.

[0013] Furthermore, the defense method described in Non-Patent Document 1 is a defense method against attack methods that provide inputs that adversely affect the server in longitudinal federated learning. This defense method only makes the derivation of the supervisory data held by the server impossible through encryption. Therefore, there is a problem that it cannot prevent attacks on the input data.

[0014] The purpose of this disclosure is to prevent learning-impairing attacks on the server in machine learning systems performing federated learning. In particular, the purpose of this disclosure is to detect malicious and unauthorized clients even when the information sent from each client to the server is dissimilar, as in longitudinal federated learning.

[0015] Methods for solving problems

[0016] The machine learning system disclosed herein comprises a client device with inherent learning data and a server device with supervised data, wherein the client device and the server device respectively possess an AI (Artificial Intelligence) model. The machine learning system promotes learning by keeping the learning data of the client device and the supervised data of the server device from each other. The server device includes an illegal client detection unit that deduces illegal client devices that are carrying out attacks that hinder the learning of the AI ​​model in the server device.

[0017] Invention Effects

[0018] In the machine learning system disclosed herein, in a federated learning machine learning system, the unauthorized client detection unit of the server device deduces unauthorized client devices that are launching attacks that hinder the learning of the AI ​​model on the server device. Therefore, according to the machine learning system disclosed herein, the following effect is achieved: even when the information sent from various client devices to the server device is not similar, malicious unauthorized client devices can be detected. Attached Figure Description

[0019] Figure 1 This is a diagram illustrating an example of the overall structure of the machine learning system in Implementation Method 1.

[0020] Figure 2 This is a diagram illustrating a structural example of the server device according to Embodiment 1.

[0021] Figure 3 This is a diagram illustrating an example of the functional structure of the server device in Implementation Method 1.

[0022] Figure 4 This is a flowchart illustrating the operation of the server device in the machine learning system of Implementation Method 1.

[0023] Figure 5 This is a flowchart illustrating the operation of the server device in the machine learning system of Implementation Method 1.

[0024] Figure 6 This is a diagram illustrating a structural example of a server device according to a variation of Embodiment 1.

[0025] Figure 7 This is a flowchart illustrating the operation of the server device in the machine learning system of Embodiment 2.

[0026] Figure 8 This is a diagram illustrating an example of the functional structure of the client device in Implementation Method 3.

[0027] Figure 9 This is a flowchart illustrating the actions of the client device in the machine learning system of Implementation Method 3. Detailed Implementation

[0028] The embodiments will now be described using the accompanying drawings. In the drawings, identical or equivalent parts are labeled with the same reference numerals. In the description of the embodiments, identical or equivalent parts are appropriately omitted or simplified. Arrows in the drawings primarily indicate data flow or processing flow.

[0029] Implementation method 1.

[0030] This embodiment describes methods to prevent learning-impeding attacks against vertical federated learning. In horizontal federated learning, the server aggregates AI models learned by each client using data on different learning objects but with the same attributes. Specifically, in horizontal federated learning, all clients have AI models with identical structures. In many forms of horizontal federated learning, the generated AI models are ultimately distributed to all clients and the server.

[0031] On the other hand, in vertical federated learning, each client uses data on different attributes of the same object to learn the AI ​​model. To meet this requirement, in vertical federated learning, each client and server has its own inherent AI model, which advances learning in a way that combines all the AI ​​models into a single AI model. Vertical federated learning is characterized by the following: the AI ​​models possessed by all clients and the server are constructed differently; each client is unaware of the AI ​​models possessed by other clients; and the output is held only by the server.

[0032] Structural description

[0033] Figure 1 This is a diagram illustrating an example of the overall structure of the machine learning system 500 according to this embodiment.

[0034] The machine learning system 500 includes a server device 100 and client devices 200 that communicate with the server device 100. Multiple client devices 200 are available.

[0035] In the machine learning system 500 of this embodiment, the server device 100 and the client device 200 each possess their own inherent AI models. The server device 100 has supervised data. The client device 200 has its own learning data. Furthermore, in the machine learning system 500, the learning data of the client device 200 and the supervised data of the server device 100 are not disclosed to each other to advance learning.

[0036] In the following description, the server device will sometimes be referred to simply as the server. Additionally, the client device will sometimes be referred to simply as the client.

[0037] As a specific example, the machine learning system 500 of this embodiment can be applied to sensitive judgments such as health diagnosis. Figure 1 In this context, server device 100 is, for example, a computer located in a large hospital as a main computer. Client device 200 is, for example, a computer located in a client such as a health diagnostic facility or an organization or shop that processes health information and is the client for which the application is made.

[0038] Client devices 200, acting as clients of health diagnostic facilities or institutions / stores that process health information, collect individual health information. The client devices 200 then process the collected information using various AI models and send it to a server device 100 located in a large hospital. The server device 100 then performs a corresponding health diagnosis on the individual and determines any diseases.

[0039] For such health diagnostic systems, one could consider attacks where malicious clients manipulate the input data during learning to alter the AI ​​model that builds the system—a type of learning-impairing attack. There are instances where malicious clients aim to increase visitors to their health diagnostic facilities, institutions, or shops, thereby causing the entire system to crash. In this embodiment, it is possible to identify or deduce the malicious client sending the manipulated input data, protecting the AI ​​model on the server from learning-impairing attacks.

[0040] As another specific example, this embodiment can be applied to the determination of an individual's credit information for purposes such as financing. The server processing unit 2000, described later, is, for example, a computer installed in the main computer of a financial institution where an individual applies for a loan. The client device 200 is, for example, a computer installed in another financial institution or a store where an individual purchases goods. In this method, each financial institution or store collects an individual's financial information. Based on this, the information is processed using various AI models and then sent to the server processing unit 2000 installed in the financial institution where the individual applies for a loan. Based on this, the server processing unit 2000 installed in the financial institution where the individual applies for a loan determines the individual's financial situation and determines whether financing is available.

[0041] For such credit information assessment systems, one could consider attacks where malicious clients manipulate the input data during learning to alter the AI ​​model, thereby hindering the operations of financial institutions. The malicious client's aim is to illegally change the assessment results, causing financial institutions with host computers or server equipment to conduct unreasonable transactions or financing. In this embodiment, it is possible to identify or deduce the malicious client sending the manipulated input data, protecting the AI ​​model on the server from such attacks.

[0042] Furthermore, this embodiment can also be applied to machine learning systems used in FA (Factory Automation) equipment. FA is short for Factory Automation. The server processing unit 2000, described later, is, for example, a computer such as a main computer located in the parent factory. The client device is, for example, a computer located in a factory that is a manufacturing client. In this approach, when manufacturing a product, each factory, as a manufacturing client, is responsible for a part of its process. However, from the viewpoint of technology protection, factories that are manufacturing clients may hide their processes, the equipment used in the processes, and the equipment data in the processes from the parent factory or other factories that are manufacturing clients. Here, in order to automate and refine the inspection, the integration of measurement information from the FA equipment in each factory can be considered. In this approach, each factory that is a manufacturing client processes the data obtained from each device through an AI model and sends it to a main computer or server device located in the parent factory. Based on this, the main computer or server device located in the parent factory performs the inspection. For this inspection system, for example, an attack method could be considered where a malicious client manipulates the input data during learning to change the AI ​​model. The following objectives can be considered: to conceal the illegality of components even if they do not meet specifications in a malicious client. In this embodiment, it is possible to identify or deduce the malicious client that sent the manipulated input data, thus protecting the AI ​​model on the server from such attacks.

[0043] Figure 2 This is a diagram illustrating a structural example of the server device 100 according to this embodiment.

[0044] Hereinafter, the devices of server device 100 and client device 200 will sometimes be referred to as devices of machine learning system 500.

[0045] Furthermore, the hardware description below will use server device 100 as an example. The same description can be applied to the hardware of client device 200 as to server device 100.

[0046] Server device 100 is a computer. Server device 100 includes a processor 910, and other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls the other hardware.

[0047] As functional elements, the server device 100 includes a server input receiving unit 1000, a server processing unit 2000, an inference result acquisition unit 3000, a monitoring data input unit 4000, a calculation progress output unit 5000, a server output unit 6000, and a storage unit 150.

[0048] The functions of the server input receiving unit 1000, server processing unit 2000, inference result acquisition unit 3000, monitoring data input unit 4000, calculation progress output unit 5000, and server output unit 6000 are implemented by software. The storage unit 150 is located in the memory 921. Alternatively, the storage unit 150 may be located in the auxiliary storage device 922, or it may be distributed between the memory 921 and the auxiliary storage device 922.

[0049] Processor 910 is a device for executing machine learning programs. Machine learning programs are programs that implement the functions of the various devices in machine learning system 500.

[0050] The processor 910 is an IC that performs computational processing. Specific examples of processor 910 include CPU, DSP, and GPU. IC is short for Integrated Circuit. CPU is short for Central Processing Unit. DSP is short for Digital Signal Processor. GPU is short for Graphics Processing Unit.

[0051] Memory 921 is a storage device for temporarily storing data. Specific examples of memory 921 are SRAM or DRAM. SRAM is short for Static Random Access Memory. DRAM is short for Dynamic Random Access Memory.

[0052] Auxiliary storage device 922 is a storage device for storing data. A specific example of auxiliary storage device 922 is an HDD. Furthermore, auxiliary storage device 922 can also be a removable storage medium such as an SD (registered trademark) memory card, CF card, NAND flash memory, floppy disk, optical disk, compact disk, Blu-ray (registered trademark) disc, or DVD. Additionally, HDD is short for Hard Disk Drive; SD (registered trademark) is short for Secure Digital; CF is short for CompactFlash; and DVD is short for Digital Versatile Disk.

[0053] Input interface 930 is a port for connecting input devices such as a mouse, keyboard, or touch panel. Specifically, input interface 930 is a USB connector. Additionally, input interface 930 can also be a port for connecting to a LAN. USB stands for Universal Serial Bus. LAN stands for Local Area Network. Figure 2 The document describes one input interface 930, but multiple input interfaces 930 may exist.

[0054] Output interface 940 is a port for cable connection to output devices such as monitors. Specifically, output interface 940 is either a USB terminal or an HDMI (trademarked) terminal. Specifically, the monitor is an LCD. Output interface 940 is also called a monitor interface. HDMI (trademarked) is short for High Definition Multimedia Interface. LCD is short for Liquid Crystal Display. Figure 2 The document describes one output interface 940, but multiple output interfaces 940 may exist.

[0055] The communication device 950 has a receiver and a transmitter. The communication device 950 connects to communication networks such as LAN, the Internet, telephone lines, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is short for Network Interface Card.

[0056] The machine learning program is executed in various devices of the machine learning system 500. The machine learning program is read into processor 910 and executed by processor 910. Memory 921 stores not only the machine learning program but also the operating system (OS). Processor 910 executes the machine learning program while executing the OS. The machine learning program and OS can also be stored in auxiliary storage device 922. The machine learning program and OS stored in auxiliary storage device 922 are loaded into memory 921 and executed by processor 910. Furthermore, part or all of the machine learning program can be incorporated into the OS.

[0057] The various devices in the machine learning system 500 can also have multiple processors instead of processor 910. These multiple processors share the execution of the machine learning program. Each processor is a device that executes the machine learning program in the same way as processor 910.

[0058] Data, information, signal values, and variable values ​​used, processed, or output by the machine learning program are stored in registers or caches within memory 921, auxiliary storage device 922, or processor 910.

[0059] Alternatively, the word "part" in each component of the machine learning system 500 can be replaced with "circuit," "process," "step," "processing," or "line." The machine learning program causes the computer to execute a process, replacing "part" with "processing" in each component of the machine learning system 500. Alternatively, the word "processing" in "processing" can be replaced with "program," "program product," "computer-readable storage medium storing the program," or "computer-readable recording medium recording the program." Furthermore, the machine learning method is a method performed by executing a machine learning program through each component of the machine learning system 500.

[0060] Machine learning programs can also be provided by storing them on computer-readable recording media. Alternatively, machine learning programs can also be provided as a program product.

[0061] Function description

[0062] Figure 3 This is a diagram illustrating an example of the functional structure of the server device 100 according to this embodiment.

[0063] use Figure 3 The functional structure of the server device 100 in this embodiment will be described.

[0064] The server input receiving unit 1000 receives input from each client device 200 and outputs the input directly to the server input storage unit 2100.

[0065] The server processing unit 2000 receives input from each client device as client input from the server input receiving unit 1000. Additionally, the server processing unit 2000 receives monitoring data as input from the monitoring data input unit 4000. Then, the server processing unit 2000 outputs the inference result for each client input to the inference result acquisition unit 3000. Furthermore, the server processing unit 2000 outputs the calculation progress status to the calculation progress output unit 5000. Finally, the server processing unit 2000 outputs the transmission value to each client device to the server output unit 6000.

[0066] The following is a detailed description of the functions within the Server Processing Unit 2000.

[0067] The server processing unit 2000 includes a server input storage unit 2100, an AI model weight storage unit 2200, an AI model actual calculation unit 2300, an inference result output unit 2400, and a learning device unit 2500.

[0068] The server input storage unit 2100 receives and stores the client inputs output from the server input receiving unit 1000. The stored client inputs are then output to the AI ​​model actual calculation unit 2300. At this time, there may also be cases where the output to the AI ​​model actual calculation unit 2300 is processed using input from the illegal client calculation unit 2532.

[0069] The AI ​​model weight storage unit 2200 receives input from the loss function calculation unit 2510, updates the weights of the AI ​​model, and saves the weights. The saved AI model weights are then output to the AI ​​model actual calculation unit 2300.

[0070] The AI ​​model actual calculation unit 2300 performs inference using the client inputs from the server input storage unit 2100 and the weights of the AI ​​model input from the AI ​​model weight storage unit 2200. The inference results are output to the inference result output unit 2400.

[0071] The inference result output unit 2400 takes the inference result calculated by the AI ​​model actual calculation unit 2300 as input, and directly outputs the obtained inference result to the loss function calculation unit 2510, the contribution calculation unit 2531 of each input, and the inference result acquisition unit 3000.

[0072] The learning device unit 2500 receives the inference results received from the inference result output unit 2400 and the supervision data received from the supervision data input unit 4000 as inputs, calculates the updated values ​​of the AI ​​model weights, and outputs them to the AI ​​model weight storage unit 2200.

[0073] In addition, in the learning device unit 2500, the progress status of the calculation is output to the calculation progress output unit 5000, and the value to be sent to each client is output to the server output unit 6000.

[0074] The learning device unit 2500 includes a loss function calculation unit 2510, a monitoring data storage unit 2520, an illegal client detection unit 2530, and a server output storage unit 2540.

[0075] The loss function calculation unit 2510 takes as input the inference result received from the inference result output unit 2400 and the supervision data stored in the supervision data storage unit 2520, and calculates its loss function. The loss function calculation unit 2510 uses this loss function to calculate the updated values ​​of the AI ​​model weights and outputs them to the AI ​​model weight storage unit 2200. Furthermore, the loss function calculation unit 2510 outputs the values ​​used for AI model updates in each client device to the server output storage unit 2540.

[0076] The monitoring data storage unit 2520 stores the monitoring data received from the monitoring data input unit 4000 and outputs it to the loss function calculation unit 2510. Additionally, the calculation progress information corresponding to this output is output to the calculation progress output unit 5000.

[0077] The unauthorized client detection unit 2530 deduces the identity of unauthorized client devices that are performing attacks that interfere with the learning of the AI ​​model in the server device 100.

[0078] The illegal client detection unit 2530 receives the inference results from the inference result output unit 2400, identifies or deduces the illegal client device, and then outputs it to the server input storage unit 2100.

[0079] The unauthorized client detection unit 2530 compares the inference results obtained from the input from the client device 200 with the monitoring data, thereby deriving the unauthorized client device. When comparing the inference results and the monitoring data, the unauthorized client detection unit 2530 calculates the contribution of the client device 200 to the inference results and uses this contribution to deduce the unauthorized client device.

[0080] Specifically, the illegal client detection unit 2530 calculates the similarity between the supervision data and the inference result, and determines whether the supervision data and the inference result contradict each other based on the similarity. If the supervision data and the inference result contradict each other, the illegal client detection unit 2530 treats the inference result as an incorrect inference and deduces the client device that contributes significantly to the inference result as an illegal client device. Sometimes, incorrect inference results are referred to as false labels.

[0081] The following is a more detailed explanation.

[0082] The illegal client detection unit 2530 includes a contribution calculation unit 2531 for each input, an illegal client calculation unit 2532, and an illegal determination unit 2533.

[0083] The contribution calculation unit 2531 of each input receives the inference result of the inference result output unit 2400 as input, deduces the extent to which each client device affects the inference result, and outputs its calculation result to the illegal client calculation unit 2532.

[0084] The illegal client calculation unit 2532 receives input from the contribution calculation unit 2531 from each input, identifies or deduces the client device suspected of being illegal, and outputs the deduction result to the server input storage unit 2100.

[0085] The illegality determination unit 2533 calculates the similarity between the monitoring data and the inference results, and determines whether the monitoring data and the inference results contradict each other based on the similarity.

[0086] The server output storage unit 2540 stores the output values ​​to the client device input from the loss function calculation unit 2510 and outputs them to the server output unit 6000. The output values ​​to the client device are the values ​​used for AI model updates in the client device.

[0087] The inference result acquisition unit 3000 inputs the inference result from the inference result output unit 2400 and outputs the inference result to the personnel or external device of the management server device 100.

[0088] The monitoring data input unit 4000 inputs monitoring data for learning from personnel or external devices of the management server device 100, and outputs the monitoring data to the server processing unit 2000.

[0089] The calculation progress output unit 5000 inputs the output information of the supervision data storage unit 2520 and outputs the inference result to the personnel or external device of the management server device 100.

[0090] The server output unit 6000 receives the server output value from the server output storage unit 2540, which is the output value to each client device, and outputs the value to each client device during learning.

[0091] Description of the action

[0092] Next, the operation of the machine learning system 500 in this embodiment will be described. The operation steps of the machine learning system 500 are equivalent to a machine learning method. Furthermore, the program that implements the operation of the machine learning system 500 is equivalent to a machine learning program.

[0093] Figure 4 and Figure 5 This is a flowchart illustrating the operation of the server device 100 in the machine learning system 500 of this embodiment.

[0094] Machine learning involving federated learning consists of two phases: a learning phase that updates the AI ​​model and an evaluation phase that utilizes the learned AI model. This implementation mainly focuses on the learning phase, and the calculation of the updated values ​​of the AI ​​model is performed twice, in steps S220 and S420.

[0095] In this embodiment, each update is referred to as the first round and the second round. In addition, the first round consists of step S200, step S210, and step S220. Additionally, the second round consists of step S400, step S410, and step S420.

[0096] Hereinafter, Figure 4 and Figure 5 the flowchart of

[0097] In Figure 4 , the processing from step S100 to step S400 is described. In Figure 5 , the processing from step S410 to step S500 is described.

[0098] Figure 4 and Figure 5 The flowchart during learning shown in

[0099] represents one iteration of vertical federated learning. The input parameter me is required. In addition, the value of me is a parameter that can be set for each iteration according to the progress of learning or the presence of an attacker. Figure 4 and Figure 5 The flowchart shown in

[0100] represents one iteration of vertical federated learning, but batch processing that aggregates multiple iterations can also be performed.

[0101] <Server Input Acceptance Processing: Step S100>

[0102] First, in step S100, the server input acceptance unit 1000 accepts inputs from all clients. The inputs zi (i = 1, 2,..., k) from the clients accepted by the server input acceptance unit 1000 are input to the server processing unit 2000. Here, k is the number of clients participating in vertical federated learning. The input from the clients to the server processing unit 2000 is saved in the server input storage unit 2100.

[0103] <Calculation of Update Value of AI Model (First Round): Steps S200 to S220>

[0104] <<Actual Calculation Processing of AI Model: Step S200>>

[0105] Next, the updated values ​​of the AI ​​model are calculated (round 1).

[0106] First, in step S200, the AI ​​model actual calculation unit 2300 uses the AI ​​model on the server to infer the inference result based on the integrated input from the client. For the input Z = (z1, z2, ..., zk) stored in the server input storage unit 2100, the AI ​​model actual calculation unit 2300 retrieves the weights w of the AI ​​model from the AI ​​model weight storage unit 2200 for calculation, and outputs the inference result p = (p1, p2, ..., pN) to the inference result output unit 2400. Here, N is the number of classification destinations in the AI ​​model, and each pi represents the probability that the output result is of class i.

[0107] <<Loss Function Calculation and Processing: Steps S210 and S220>>

[0108] Next, in step S210, the loss function calculation unit 2510 calculates the loss function L used for AI model updating. This calculation utilizes inputs from the inference result output unit 2400 and the supervision data storage unit 2520. Specifically, the inference result output unit 2400 outputs the probability distribution p = (p1, p2, ..., pN) of N classes. Furthermore, the input from the supervision data storage unit 2520 uses an N-dimensional vector t = (t1, t2, ..., tN) where the positive solution label ti is 1 and all others are 0. At this time, the loss function is the sum of squares error Σ(ti-pi)^2 or the cross-entropy error -Σtilog(pi). Additionally, the values ​​input through the supervision data input unit 4000 are stored in the supervision data storage unit 2520.

[0109] Furthermore, in step S220, the loss function calculation unit 2510 performs backpropagation on the calculated loss function L to calculate the output value of the updated AI model for the client side. L / zi (i=1,2,…,k) and store it in the server output storage unit 2540.

[0110] <Processing for determining consistency of inference results: Step S300>

[0111] Furthermore, in step S300, the illegality determination unit 2533 of the illegal client detection unit 2530 confirms whether the inference result stored in the inference result output unit 2400 is consistent with the inference result stored in the supervision data storage unit 2520. For example, the illegality determination unit 2533 calculates the similarity between the supervision data and the inference result, and determines whether the supervision data and the inference result contradict each other based on the similarity.

[0112] If the inference results are inconsistent, i.e., the monitoring data contradicts the inference results, the existence of an illegal client is suspected, and the process proceeds to step S310.

[0113] If the inference results are consistent, meaning the monitoring data does not contradict the inference results, it is determined that there is no illegal client, and the process proceeds to step S400.

[0114] Furthermore, the term "consistency" as used here can be defined in various ways depending on the characteristics of the data to be used.

[0115] As an example of the definition of "consistency," consistency can be defined by whether the maximum likelihood inference result of the inference result output unit 2400 is consistent with the maximum likelihood inference result of the corresponding inference result in the supervision data storage unit 2520. Specifically, the output of a function that produces different outputs when the maximum likelihood inference results are consistent and inconsistent is defined as "similarity." For example, the output of a function that outputs 0 when consistent and 1 when inconsistent is defined as "similarity." In this case, the illegality determination unit 2533 determines that the supervision data and the inference result are not contradictory when the "similarity" is 0, and that the supervision data and the inference result are contradictory when the "similarity" is 1.

[0116] Furthermore, beyond the maximum likelihood inference result, "consistency" can also be defined by incorporating a distance space into the supervision data and the inference result, and determining a threshold for that distance. Specifically, the distance calculated in the distance space is called "similarity." In this case, if the "similarity" is less than a certain threshold, the illegality determination unit 2533 determines that the supervision data and the inference result are not contradictory. Conversely, if the "similarity" is greater than a certain threshold, the illegality determination unit 2533 determines that the supervision data and the inference result are contradictory.

[0117] <Contribution Calculation Processing (Mislabeling): Step S310>

[0118] In step S310, the contribution calculation unit 2531 for each input calculates the contribution of each client based on the maximum likelihood inference result of the inference result output unit 2400. Here, the maximum likelihood inference result of the inference result output unit 2400 is... Figure 4 In the flowchart, these are represented as mislabels. In step S310, all client inputs are treated as inputs to the model on the server, and contributions are calculated using Grad-Cam. Furthermore, this contribution calculation is not limited to a single mislabel; contributions corresponding to multiple mislabels can be superimposed. For example, there is a method where contributions are calculated separately for multiple mislabels with high likelihood, and a weighted average is performed for each contribution, using likelihood as the weight. In addition to this weighted average, various methods that incorporate likelihood as the weight can also be applied.

[0119] <Illegal client computing process (mislabeling): Step S311>

[0120] In step S311, the illegal client computing section 2532 detects the top me clients that contribute highly to the mislabeling as illegal clients. Then, the illegal client computing section 2532 excludes the detected illegal clients from the candidates for recalculating the updated value of the AI model in the second round described later. Then, it transfers to step S400.

[0121] <Calculation of the updated value of the AI model (second round): Steps S400 to S420>

[0122] In step S400, the loss function calculation section 2510 recalculates the loss function L' based on the input values of the clients not excluded in step S311. In this calculation, for the inputs saved in the server input saving section 2100, only the inputs not excluded in step S311 are retained, and the calculation is performed in the same manner as in steps S200 and S210. Specifically, for the corresponding inputs, the weights w of the AI model are called from the AI model weight saving section 2200 for calculation, and the inference is performed by the AI model actual calculation section 2300 using the AI model on the server, and the inference result q = (q1, q2,..., qN) is output to the inference result output section 2400. On this basis, the loss function calculation section 2510 uses the input q = (q1, q2,..., qN) from the inference result output section 2400 and the input t = (t1, t2,..., tN) from the supervised data saving section 2520 to calculate the loss function L' for updating the AI model. In addition, at the end of this processing, the utilization completion of the supervised data is passed from the supervised data saving section 2520 to the calculation progress output section 5000, thereby outputting the calculation progress.

[0123] Next, in step S410, the loss function calculation section 2510 performs the error backpropagation method on the calculated loss function to update the weights of the AI model on the server saved in the AI model weight saving section 2200.

[0124] Furthermore, in step S420, the loss function calculation section 2510 performs the error backpropagation method on the calculated loss function to calculate the output value of the updated value for the AI model only for the clients not excluded in step S311 or step S321. L’ / zi, and stores it in the server output saving section 2540.

[0125] <Server output processing: Step S500>

[0126] Finally, in step S500, the server output unit 6000 sends the value stored in the server output storage unit 2540 to the client.

[0127] The above describes the actions related to the learning phase. In the evaluation phase, only steps S100 and S200 are performed. The actions in the evaluation phase will be explained below.

[0128] First, in step S100, input from all clients is accepted. Input zi (i=1, 2, ..., k) from the clients is accepted by the server input acceptance unit 1000 and input to the server processing unit 2000. Input from the clients to the server processing unit 2000 is saved to the server input storage unit 2100.

[0129] Next, in step S200, the AI ​​model actual calculation unit 2300 performs inference using the AI ​​model on the server. The AI ​​model actual calculation unit 2300 calculates the weights w of the AI ​​model from the AI ​​model weight storage unit 2200 based on the input stored in the server input storage unit 2100, and outputs the inference result p = (p1, p2, ..., pN) to the inference result output unit 2400. By sending this inference result to the inference result acquisition unit 3000, the evaluation phase processing is completed.

[0130] Other structures

[0131] In this embodiment, the functions of each device in the machine learning system 500 are implemented through software. As a variation, the functions of each device in the machine learning system 500 can also be implemented through hardware.

[0132] Specifically, each device in the machine learning system 500 has electronic circuitry 909 instead of processor 910.

[0133] Figure 6 This is a diagram illustrating a structural example of a server device 100, a modified example of this embodiment.

[0134] Electronic circuit 909 is a dedicated electronic circuit that implements the functions of server input receiving unit 1000, server processing unit 2000, inference result acquisition unit 3000, supervisory data input unit 4000, calculation progress output unit 5000, and server output unit 6000. Specifically, electronic circuit 909 can be a single circuit, a composite circuit, a programmable processor, a parallel programmable processor, a logic IC, a GA, an ASIC, or an FPGA. GA is short for Gate Array. ASIC is short for Application Specific Integrated Circuit. FPGA is short for Field-Programmable Gate Array.

[0135] The functions of the server input receiving unit 1000, server processing unit 2000, inference result acquisition unit 3000, monitoring data input unit 4000, calculation progress output unit 5000, and server output unit 6000 can be implemented by a single electronic circuit or distributed among multiple electronic circuits.

[0136] As another variation, some functions of the server input receiving unit 1000, server processing unit 2000, inference result acquisition unit 3000, monitoring data input unit 4000, calculation progress output unit 5000, and server output unit 6000 may be implemented by electronic circuits, while the remaining functions may be implemented by software. Alternatively, some or all of the functions of the server input receiving unit 1000, server processing unit 2000, inference result acquisition unit 3000, monitoring data input unit 4000, calculation progress output unit 5000, and server output unit 6000 may also be implemented by firmware.

[0137] The processor and electronic circuits are each referred to as processing lines. That is, the functions of the server input receiving unit 1000, server processing unit 2000, inference result acquisition unit 3000, supervisory data input unit 4000, calculation progress output unit 5000, and server output unit 6000 are implemented by the processing lines.

[0138] Explanation of the effects of this implementation method

[0139] In this embodiment, a machine learning system as described below is presented. Alternatively, a machine learning device can be used instead of a machine learning system.

[0140] In a machine learning device, clients with built-in learning data and servers with supervised data maintain their respective AI models, ensuring that the data is not shared between them to advance learning. The machine learning device also includes an unauthorized client detection unit that identifies or determines unauthorized clients that are attempting attacks that hinder the learning of the AI ​​models.

[0141] In addition, the illegal client detection unit is installed on the server. It compares the inference results obtained from the input from the client with the supervision data. Based on the deduction or identification of the illegal client, the AI ​​model learns as a whole.

[0142] In addition, in the comparison between the inference results and the monitoring data in the illegal client detection department, the contribution of the inference results is used to infer or determine the illegal client.

[0143] Furthermore, contributions are calculated based on the similarity between the supervised data and the inference results, as well as the contributions among clients that participated in the same server input. In cases where the supervised data and inference results contradict each other, clients with high contributions to incorrect inferences are deduced or identified as illegitimate clients, improving the learning efficiency of the AI ​​model. Conversely, in cases where the supervised data and inference results are consistent, clients with low contributions to correct inferences are deduced or identified as illegitimate clients, further improving the learning efficiency of the AI ​​model.

[0144] In addition, machine learning devices use information derived or identified from illegitimate clients to update the server's AI model.

[0145] In addition, the machine learning device also sends information about the illegal clients it has deduced or identified to the clients, which then use this information to integrate models among themselves, thereby promoting more efficient federated learning.

[0146] Furthermore, in this embodiment, in vertical federated learning, attacks that maliciously interfere with the AI ​​model's learning are prevented by deduceing or identifying the illegitimate client conducting the attack. Vertical federated learning is a type of federated learning where input neurons to the server are not shared among clients.

[0147] As described above, in the machine learning system of this embodiment, by calculating the contributions of all clients to the input of the supervised data and comparing them horizontally, clients that cause adverse effects in vertical federated learning can be identified. In particular, in defense techniques against horizontal federated learning, which is frequently used for federated learning, the models uploaded from each client are averaged, and clients that deviate significantly from the average model are excluded, thereby preventing a decrease in the reliability of the AI ​​model. However, in vertical federated learning, since each client processes different attributes, it cannot be said that the information sent from each client to the server is similar. Therefore, defense techniques against learning-impairing attacks against horizontal federated learning cannot be applied to vertical federated learning. In the machine learning system of this embodiment, even when the information sent from each client to the server is not similar, malicious and unauthorized clients can be deduced or identified, thus preventing learning-impairing attacks on the server.

[0148] Implementation method 2.

[0149] In this embodiment, the differences from Embodiment 1 and the points added to Embodiment 1 will be explained.

[0150] In this embodiment, structures having the same functions as in Embodiment 1 are labeled with the same reference numerals, and their descriptions are omitted.

[0151] In this embodiment, methods to further improve the learning efficiency of AI models are described.

[0152] Structural description

[0153] The structure of the machine learning system 500 in this embodiment is the same as that described in Embodiment 1.

[0154] Function description

[0155] In Implementation 1, when there is a contradiction between the monitoring data and the inference results, the illegal client detection unit 2530 deduces the client device that contributes highly to the false tag as an illegal client device.

[0156] In this embodiment, the illegal client detection unit 2530, when the monitoring data and the inference result do not contradict each other, treats the inference result as the correct inference result and deduces client devices that contribute little to the inference result as illegal client devices. Furthermore, the correct inference result is sometimes referred to as a positive label or a true label.

[0157] Description of the action

[0158] Figure 7 This is a flowchart illustrating the operation of the machine learning system 500 in this embodiment.

[0159] exist Figure 7 The document describes the processes from steps S100 to S400. The flowchart from step S410 onwards is consistent with... Figure 5 same.

[0160] exist Figure 7 In the above, the following describes the implementation method 1. Figure 4 The flowchart is supplemented with steps S320 and S321. These processes can exclude clients that send illegal data with minimal impact, as well as clients that perform legitimate learning but reduce learning efficiency. This improves the learning efficiency of the AI ​​model. Furthermore, the cases that can be handled in this embodiment are the same as in Embodiment 1. However, compared to Embodiment 1, it excludes clients that send illegal data with minimal impact, as well as clients that perform legitimate learning but reduce learning efficiency, thus improving the learning efficiency of the AI ​​model.

[0161] The following is a description of the actions.

[0162] Similar to Implementation Method 1, during the learning phase, the calculation of the updated value of the AI ​​model is performed twice, in steps S220 and S420. The first round consists of steps S200, S210, and S220. The second round consists of steps S400, S410, and S420.

[0163] Furthermore, in this embodiment, the evaluation phase is the same as in embodiment 1; therefore, only the learning phase will be described.

[0164] Figure 7 and Figure 5 The flowchart shown during the learning process is similar to that of Implementation 1, illustrating one iteration of longitudinal learning. In this implementation, parameters me and mc need to be input. Furthermore, the values ​​of me and mc are parameters that can be set for each iteration based on the progress of learning or the presence of an attacker. Parameter me can be input at any time before step S311, which utilizes parameter me within an iteration. In particular, since this parameter me is determined after calculating the contribution to the false labels in step S310, it can be set as a threshold for the contribution. Similarly, parameter mc can also be input at any time before step S321, which utilizes parameter mc within an iteration. In particular, since this parameter mc is determined after calculating the contribution to the positive labels in step S320, it can be set as a threshold for the contribution. Figure 7 and Figure 5The flowchart shown represents one iteration of vertical federated learning, but it is also possible to perform batch processing that aggregates multiple iterations.

[0165] <Server input acceptance processing: Step S100>

[0166] The processing of Step S100 is the same as that in Embodiment 1.

[0167] <Calculation of updated value of AI model (first round): Steps S200 to S220>

[0168] The processing of Steps S200 to S220 is the same as that in Embodiment 1.

[0169] <Inference result consistency determination processing: Step S300>

[0170] In Step S300, the illegal determination unit 2533 confirms whether the inference result stored in the inference result output unit 2400 is consistent with the inference result stored in the supervised data storage unit 2520.

[0171] If the inference results are inconsistent, that is, the supervised data and the inference results are contradictory, it is suspected that there is an illegal client, and the process transfers to Step S310.

[0172] If the inference results are consistent, that is, the supervised data and the inference results are not contradictory, in order to improve the learning efficiency, the process transfers to Step S320.

[0173] In addition, the process of determining whether the supervised data and the inference results are consistent or contradictory in Step S300 is the same as that in Embodiment 1. Also, the definition of "consistent" here is the same as that in Embodiment 1.

[0174] <Contribution calculation processing (mislabeling): Step S310>

[0175] <Illegal client calculation processing (mislabeling): Step S311>

[0176] The processing of Steps S310 to S311 is the same as that in Embodiment 1.

[0177] <Contribution calculation processing (correct labeling): Step S320>

[0178] In Step S320, the contribution calculation unit 2531 for each input calculates the contribution of each client to the maximum likelihood inference result of the inference result output unit 2400. Here, the maximum likelihood inference result of the inference result output unit 2400 is Figure 7 represented as the correct label in the

[0179] <Illegal client calculation processing (correct labeling): Step S321>

[0180] Furthermore, from the perspective of learning efficiency, the illegal client calculation unit 2532 excludes the lower mc clients that contribute little to the positive label from the candidates for recalculating the updated value of the AI model in the second round described below. Then, it proceeds to step S400.

[0181] <Calculation of the updated value of the AI model (second round): Step S400 to Step S420>

[0182] In step S400, the loss function calculation unit 2510 recalculates the loss function L' based on the input values of the remaining clients that were not excluded in steps S311 and S321. In this calculation, for the inputs saved in the server input save unit 2100, only the inputs that were not excluded in steps S311 and S321 are retained, and the calculation is performed in the same manner as in steps S200 and S210.

[0183] The difference in the processing of step S400 from Embodiment 1 is that only the inputs that were not excluded in steps S311 and S321 are retained. Other processing is the same as in Embodiment 1.

[0184] In addition, the processing of steps S410 to S420 is also the same as in Embodiment 1.

[0185] <Server output processing: Step S500>

[0186] The processing of step S500 is the same as in Embodiment 1.

[0187] Explanation of the effects of this embodiment

[0188] As described above, in the machine learning system of this embodiment, in steps S310 and S311, the contributions of all clients to the input of the supervised data are calculated, and the clients are compared horizontally. Thus, it is possible to identify the clients that have an adverse impact in vertical federated learning. Furthermore, in steps S320 and S321, it is possible to exclude the clients that have sent illegal data but have little impact, as well as the clients that have reduced the learning efficiency despite having performed legitimate learning. As a result, the learning efficiency of the AI model can be further improved.

[0189] Embodiment 3.

[0190] In this embodiment, mainly the points that are different from Embodiments 1 and 2 and the points added to Embodiments 1 and 2 are described.

[0191] In this embodiment, the structures having the same functions as those in Embodiments 1 and 2 are labeled with the same reference numerals, and their descriptions are omitted.

[0192] In this embodiment, the machine learning system 500 performs segmented federated learning without interaction between multiple client devices 200.

[0193] In this embodiment, it is explained that the machine learning method described in Embodiments 1 and 2 is applied not only to vertical federated learning but also to segmented federated learning as a broader method of federated learning that includes vertical federated learning.

[0194] Segmented federated learning is a type of federated learning characterized by each client and server maintaining its own AI model, with the output inference results held solely by the server. Furthermore, segmented federated learning can be applied not only to the vertical federated learning described in embodiments 1 and 2, but also to horizontal federated learning.

[0195] Define the structure of a general partitioned federated learning approach.

[0196] First, the server and each client have their own AI model, and the server outputs inference results from its AI model. These inference results are held solely by the server. Each client inputs data to the server during inference, utilizing a portion of the neurons in the server's input unit, predetermined in advance. Furthermore, the neurons used by each client may overlap; if the neurons used by each client are not overlapping, this corresponds to the vertical federated learning in embodiments 1 and 2. In addition to communication between the server and clients, AI models can be shared among clients. However, sharing AI models among clients is not limited to the entire AI model but also includes sharing portions of the AI ​​model. Furthermore, in vertical federated learning, the client's AI model is completely segmented; therefore, computations related to this sharing are not considered. In this embodiment, a technique to prevent learning-impeding attacks against the above-described segmented federated learning is proposed.

[0197] In this embodiment, the same specific example as in Embodiment 1 can be processed. Furthermore, more generally, federated learning can be applied more flexibly when the scope of data collected by participating institutions is uncertain. For example, it can also be applied to machine learning systems used in FA devices.

[0198] The server processing unit 2000 is, for example, a main computer located at the parent factory. The client is, for example, a computer located at a factory that is a manufacturing client. In this embodiment, when manufacturing a product, each factory, as a manufacturing client, is responsible for a part of the process. The factories for each process are not in a one-to-one correspondence; multiple factories are utilized for optimal production planning. However, from a technical protection point of view, the factories that are manufacturing clients hide their processes, the equipment used in the processes, and the equipment data from the parent factory or other factories that are manufacturing clients.

[0199] Here, to automate and refine the inspection process, the integration of information from each factory is considered. In this approach, each factory, as the subject of manufacturing orders, processes the data obtained from its various devices using an AI model and sends it to a main computer located at the parent factory. The main computer at the parent factory then performs the inspection. Specifically, unlike the vertical federated learning of Implementation Method 1, this implementation allows for efficient and refined inspection even if the factories responsible for each process change. Furthermore, by integrating the AI ​​models among the factories responsible for each process, a more sophisticated AI model can be constructed. For this inspection system, for example, it is possible to consider an attack method where a malicious client manipulates the input data during learning to alter the AI ​​model. An example could be a malicious client attempting to conceal its illegality even if it does not produce a component that meets specifications. In this case, this implementation can identify or deduce the malicious client that sent the manipulated input data, protecting the AI ​​model on the server from such attacks.

[0200] This embodiment can also be extended in the same way to other specific examples listed in Embodiment 1, health diagnosis, and the determination of personal credit information. Firstly, in health diagnosis, even when the person undergoing the diagnosis regularly visits different hospitals, the diagnosis can still be performed using the same principle as described above. Furthermore, in the determination of personal credit information, there are cases where customers change the shops they use, and the data collection agency may also change. In such cases, credit information can still be determined. In this embodiment, even under such circumstances, learning-impairing attacks against the AI ​​model can be addressed.

[0201] As described above, in this embodiment, by appending illegal client information to the server's output value and the client's response based on the appended information, not only can the learning efficiency of the server's AI model be improved, but also the learning efficiency of the client's AI model can be improved.

[0202] Regarding the structure and operation of server device 100

[0203] The following is a detailed description of this embodiment. First, the server will be described.

[0204] In this embodiment, similar to embodiments 1 and 2, input values ​​from clients with the same inference results are collected, and their contributions are calculated. In this embodiment, the structure of the server device 100 is similar to... Figure 3 The functional structure diagram is the same. Furthermore, the operation of server device 100 is the same as... Figure 4 and Figure 5 or Figure 7The flowcharts are the same.

[0205] In this embodiment, the information sent from the server to the client and the actions performed on the client are determined according to the specific implementation between the server and the client. Therefore, in this embodiment, the machine learning methods described in embodiments 1 and 2 can be applied to segmented federated learning.

[0206] The following describes the operation of the server. In particular, the part that causes the difference from implementation methods 1 and 2 will be explained. Specifically, regarding... Figure 4 and Figure 5 or Figure 7 The steps S310, S320 and S500 in the flowchart will be explained.

[0207] In particular, in this embodiment, AI model learning is performed on the premise of multiple inputs to the same supervised data. Furthermore, this premise is a reasonable assumption in machine learning that processes large amounts of data for learning.

[0208] Therefore, in this embodiment, similar to embodiments 1 and 2, the input of parameters me and mc is required.

[0209] Furthermore, the settings for the values ​​of parameters me and mc, the input timing of parameters me and mc, and the setting of the threshold are the same as in implementation methods 1 and 2.

[0210] First, the differences between steps S310 and S320, namely the contribution calculation of the inference result, will be explained. In embodiments 1 and 2, the contribution of each client was evaluated horizontally for a single inference result. However, in this embodiment, there is also the possibility that the client provides the server with all the input values. In this case, it is impossible to correctly evaluate the horizontal contribution of each client. Therefore, in this embodiment, multiple identical inference results are collected, and the contribution of each client is calculated. Furthermore, an example of contribution calculation is shown below.

[0211] The following explains step S310. The purpose of step S310 is to determine the ranking of contributions to mislabeled data. Therefore, firstly, the client set is ordered based on the similarity between the supervision data and the inference result. Thus, it is determined that the more a client participates in server inputs whose inference results are similar to the supervision data, the greater its contribution. At this point, clients participating in the same server inputs are in the same order. Based on this, among the clients participating in the same server inputs, rankings are assigned according to their contributions to the supervision data. Furthermore, the ranking relationship with other server inputs is maintained at this time. Through the above, the ranking of contributions to mislabeled data can be determined. Similarly, in step S320, the ranking of contributions to positive labels can also be determined.

[0212] Next, step S500 will be explained, which involves publishing the differences in the gradients to each client. In this embodiment, the AI ​​model can be updated by sharing the AI ​​model between clients. However, merging the model of a client currently engaging in a learning-impairing attack into other clients would further exacerbate the attack. Therefore, in this embodiment, when publishing gradients to each client, the server sends the client's ID, which is determined to be a legitimate client, to the client to prevent learning-impairing attacks caused by model sharing between clients. Specifically, the server sends the client the value used to update the client's AI model. L / In addition to zi, a set B = {2, 5, 6, ..., N-1} of legitimate client IDs is also sent. Furthermore, the client itself can also send reliability information for each client to perform control in accordance with this requirement.

[0213] The above is an explanation related to the server.

[0214] Structure and operation of client device 200

[0215] Next, we will explain the client-side aspects. In this implementation, the client needs to perform actions based on the server's behavior.

[0216] Figure 8 This is a diagram illustrating an example of the functional structure of the client device 200 in this embodiment.

[0217] Furthermore, the client described in this embodiment is related to the client device 200 that performs normal processing. A client device performing a learning-impedance attack may not follow the following description in order to make the attack successful.

[0218] As functional elements, the client device 200 of this embodiment includes a server return value input receiving unit 10, a client processing unit 20, an AI model weight output unit 30, an AI model weight input receiving unit 40, a local data storage unit 50, and a server send value output unit 60. The AI ​​model weight output unit 30 is also referred to as an other client AI model weight output unit. The AI ​​model weight input receiving unit 40 is also referred to as an other client AI model weight input receiving unit.

[0219] The hardware structure of the client device 200 in this embodiment is the same as that of the server device 100, as described in Embodiment 1.

[0220] The server return value input receiving unit 10 receives a server return value containing AI model update information from the server. The server return value input receiving unit 10 outputs the server return value containing AI model update information from the server to the AI ​​model weight storage unit 21 and the AI ​​model weight calculation unit 23.

[0221] The client processing unit 20 receives AI model update information from the server return value input receiving unit 10 and updates the weights of the AI ​​model internally. Furthermore, with the aim of integrating with the AI ​​models of other clients, the client processing unit 20 outputs its own AI model weights to the other client AI model weight output unit 30. Additionally, the client processing unit 20 updates the AI ​​model by receiving the AI ​​weights of other clients from the other client AI model weight input receiving unit 40. Then, the client processing unit 20 performs inference based on the client's AI model using locally stored data input from the local data storage unit 50 and outputs the output value to the server to the server sending value output unit 60.

[0222] The following is a detailed description of the client processing unit 20.

[0223] The client processing unit 20 includes an AI model weight storage unit 21, an AI model actual calculation unit 22, and an AI model weight calculation unit 23.

[0224] The AI ​​model weight storage unit 21 outputs the weights of the AI ​​model at a certain moment to the AI ​​model weight calculation unit 23, and receives the updated weights of the AI ​​model from the AI ​​model weight calculation unit 23. Furthermore, based on the values ​​input from the server-returned input receiving unit 10, the AI ​​model storage unit 21 outputs the AI ​​model to the AI ​​model weight output unit 30 of other clients, thus sharing the weights of the AI ​​model with other clients. Moreover, the AI ​​model weight storage unit 21 outputs the weights of the AI ​​model to the AI ​​model actual calculation unit 22 in order to update the weights of the next AI model during the learning phase, or to perform inference during the evaluation phase.

[0225] The AI ​​model actual calculation unit 22 takes into input the weights of the AI ​​model from the AI ​​model weight storage unit 21 and the data from the local data storage unit 50, performs client-based AI model inference, and outputs the output value to the server to the server sending value output unit 60.

[0226] The AI ​​model weight calculation unit 23 calculates the updated weights of the AI ​​model based on the updated AI model values ​​input from the server return value input receiving unit 10 and the AI ​​model weights at a certain moment input from the AI ​​model weight storage unit 21. The AI ​​model weight calculation unit 23 then outputs the updated AI model weights to the AI ​​model weight storage unit 21.

[0227] Then, the AI ​​model weight calculation unit 23 calculates the updated weights of the AI ​​model and outputs the updated weights of the AI ​​model to the AI ​​model weight storage unit 21. At this time, the AI ​​model weight calculation unit 23 calculates the updated weights of the AI ​​model using the following information.

[0228] • The information from the legitimate client entered by the server's return value into the reception department (10).

[0229] • The weights of the AI ​​model at a certain moment, input from the AI ​​model weight storage unit 21.

[0230] • Input AI weights from other clients in the AI ​​model weight input reception unit 40

[0231] The AI ​​model weight output unit 30 of other clients will output the weights of the AI ​​model input from the AI ​​model weight storage unit 21 to other clients.

[0232] The AI ​​model weight input receiving unit 40 receives AI models from other clients and outputs them to the AI ​​model weight calculation unit 23.

[0233] The local data storage unit 50 inputs and saves the local data stored separately by each client, and outputs the local data to the AI ​​model actual calculation unit 22.

[0234] The server value output unit 60 outputs the values ​​sent to the server from the AI ​​model actual calculation unit 22 to the server.

[0235] Figure 9 This is a flowchart illustrating the operation of the client device 200 in the machine learning system 500 of this embodiment.

[0236] The following describes the client's actions. Machine learning involving federated learning consists of two phases: a learning phase that updates the AI ​​model and an evaluation phase that utilizes the learned AI model. This implementation primarily focuses on the learning phase.

[0237] First, the action flow of updating the AI ​​model during the learning phase is recorded.

[0238] First, in step S600, the server return value input receiving unit 10 confirms whether there is a return value from the server, i.e., a server return value. Specifically, it is assumed that no server return value exists at the start of learning, and in this case, the process proceeds to step S800. If a server return value exists, the process proceeds to step S610.

[0239] In step S610, the server return value input receiving unit 10 receives the server return value.

[0240] Next, in step S700, the AI ​​model weight calculation unit 23 updates the AI ​​model stored in the AI ​​model weight storage unit 21 using the server return value received by the server return value input receiving unit 10.

[0241] In step S710, the AI ​​model weight calculation unit 23 determines whether there is integration with AI models from other clients. If there is integration with AI models from other clients, the process proceeds to step S711. Otherwise, the process proceeds to step S800.

[0242] In step S711, the AI ​​model weight output unit 30 sends its own AI model to a designated other client. Specifically, the AI ​​model weight output unit 30 uses the information about a reliable client contained in the server's return value to send its own AI model to the designated other client. In step S711, a method of sending the AI ​​model to a number specified by the server can be considered, or a method of determining the destination of the AI ​​model on the client side based on the reliability received from the server can be considered.

[0243] In step S712, the AI ​​model weight calculation unit 23 updates the AI ​​model stored in the AI ​​model weight storage unit 21 based on the server's return value. Specifically, the AI ​​model weight calculation unit 23 uses the reliable client information contained in the server's return value to receive AI models from other clients received by the AI ​​model weight input receiving unit 40, and updates the AI ​​model. In step S712, a method of integrating AI models with numbers specified by the server, or a method of integrating AI models by independently weighting them on the client side based on the reliability received from the server, can be considered.

[0244] In step S800, in the AI ​​model actual calculation unit 22, the output is calculated using local data called from the local data storage unit 50 and the AI ​​model stored in the AI ​​model weight storage unit 21.

[0245] Finally, in step S900, the output calculated in the AI ​​model actual calculation unit 22 is sent to the server via the server sending value output unit 60.

[0246] The above describes the actions related to the learning phase. In the evaluation phase, only steps S800 and S900 are processed. The actions in the evaluation phase are described below.

[0247] First, in step S800, in the AI ​​model actual calculation unit 22, the output is calculated using local data called from the local data storage unit 50 and the AI ​​model stored in the AI ​​model weight storage unit 21.

[0248] Based on this, in step S900, the output calculated in the AI ​​model actual calculation unit 22 is sent to the server through the server sending value output unit 60.

[0249] Explanation of the effects of this implementation method

[0250] The machine learning system in this embodiment is applied to segmented federated learning where there is no interaction between clients. The unauthorized client detection unit can prevent learning-impeding attacks by deducing or identifying unauthorized clients that are conducting attacks that hinder the AI ​​model's learning.

[0251] As described above, segmented federated learning is generally defined in this embodiment. In this embodiment, by appending illegal client information to the server's output value and the client's response based on the appended information, not only can the learning efficiency of the server's AI model be improved, but the learning efficiency of the client's AI model can also be improved.

[0252] The machine learning systems described in embodiments 1-3 above can be applied, for example, to federated learning based on sensitive information, such as diagnosis in the medical field or credit information determination in the financial field. In particular, the machine learning systems described in embodiments 1-3 can limit the scope of disclosure of sensitive information while maintaining device performance; therefore, as a result, more sophisticated machine learning systems can be constructed.

[0253] Furthermore, the machine learning systems described in embodiments 1-3 can be applied to various systems such as computer-aided design (FA), safety management systems, and autonomous mobile robots. In particular, the machine learning systems disclosed herein are suitable for use in AI for industrial robots.

[0254] In the above embodiments 1 to 3, each part of the machine learning system's devices was described as an independent functional block. However, the structure of the machine learning system may not be as described in the above embodiments. The functional blocks of the machine learning system can have any structure as long as they can perform the functions described in the above embodiments. In addition, the machine learning system may not be a single device, but a system composed of multiple devices.

[0255] Alternatively, multiple parts of embodiments 1 to 3 can be combined. Or, only a portion of these embodiments can be implemented. Furthermore, these embodiments can be implemented in any combination, either as a whole or in part.

[0256] That is, in embodiments 1 to 3, each embodiment can be freely combined, or any constituent element of each embodiment can be modified, or any constituent element can be omitted in each embodiment.

[0257] Furthermore, the above-described embodiments are essentially preferred examples and are not intended to limit the scope of this disclosure, its applications, or its uses. The above-described embodiments can be modified as needed. For example, the steps described using flowcharts or sequence diagrams can be appropriately altered.

[0258] Label Explanation

[0259] 10: Server Return Value Input Receiving Department; 20: Client Processing Department; 30: AI Model Weight Output Department; 40: AI Model Weight Input Receiving Department; 50: Local Data Storage Department; 60: Server Send Value Output Department; 21: AI Model Weight Storage Department; 22: AI Model Actual Calculation Department; 23: AI Model Weight Calculation Department; 100: Server Device; 150: Storage Department; 200: Client Device; 500: Machine Learning System; 1000: Server Input Receiving Department; 2000: Server Processing Department; 3000: Inference Result Acquisition Department; 4000: Supervision Data Input Department; 5000: Calculation Progress Output Department; 6000: Service Server output unit; 2100: Server input storage unit; 2200: AI model weight storage unit; 2300: AI model actual calculation unit; 2400: Inference result output unit; 2500: Learning device unit; 2510: Loss function calculation unit; 2520: Supervision data storage unit; 2530: Illegal client detection unit; 2540: Server output storage unit; 2531: Contribution calculation unit; 2532: Illegal client calculation unit; 2533: Illegal determination unit; 909: Electronic circuit; 910: Processor; 921: Memory; 922: Auxiliary storage device; 930: Input interface; 940: Output interface; 950: Communication device.

Claims

1. A machine learning system comprising a client device with inherent learning data and a server device with supervised data, wherein the client device and the server device each possess an AI (Artificial Intelligence) model, and the machine learning system promotes learning by keeping the learning data of the client device and the supervised data of the server device mutually confidential, wherein... The server device includes an illegal client detection unit that identifies illegal client devices that are launching attacks that hinder the learning of the AI ​​model in the server device.

2. The machine learning system according to claim 1, wherein, The illegal client detection unit deduces the illegal client device by comparing the inference results obtained from the input from the client device with the supervision data.

3. The machine learning system according to claim 2, wherein, When comparing the inference result with the supervision data, the illegal client detection unit calculates the contribution of the client device to the inference result and uses the contribution to the inference result to deduce the illegal client device.

4. The machine learning system according to claim 3, wherein, The illegal client detection unit calculates the similarity between the supervision data and the inference result, and determines whether the supervision data and the inference result contradict each other based on the similarity. If the supervision data and the inference result contradict each other, the inference result is regarded as an incorrect inference result, and the client device that contributes more to the inference result is deduced as the illegal client device.

5. The machine learning system according to claim 4, wherein, If the monitoring data does not contradict the inference result, the illegal client detection unit will take the inference result as the correct inference result and deduce the client device that contributes little to the inference result as the illegal client device.

6. The machine learning system according to any one of claims 1 to 5, wherein, The server device includes a learning unit that uses information from the unauthorized client device to update the AI ​​model in the server device.

7. The machine learning system according to any one of claims 1 to 6, wherein, The server device has a server output unit that outputs information about the unauthorized client device to the client device. The client device integrates the AI ​​model with other client devices based on the information from the illegal client device.

8. The machine learning system according to any one of claims 1 to 6, wherein, The machine learning system comprises multiple client devices, each with its own learning data, which serve as client devices. The multiple client devices and the server device each maintain an AI model. The machine learning system performs segmented federated learning without interaction among the multiple client devices.

9. The machine learning system according to any one of claims 1 to 6, wherein, The machine learning system comprises multiple client devices, each with its own learning data, which serve as client devices. The multiple client devices and the server device each maintain an AI model. The machine learning system performs longitudinal federated learning among the multiple client devices without sharing the input neurons to the server device.

10. A server device comprising a machine learning system, the machine learning system having a client device having inherent learning data and a server device having supervised data, the client device and the server device each holding an AI (Artificial Intelligence) model, the machine learning system advancing learning by keeping the learning data of the client device and the supervised data of the server device mutually confidential, wherein, The server device includes an illegal client detection unit that identifies illegal client devices that are launching attacks that hinder the learning of the AI ​​model in the server device.

11. A client device included in a machine learning system, the machine learning system comprising a client device having inherent learning data and a server device having supervised data, the client device and the server device each holding an AI (Artificial Intelligence) model, the machine learning system advancing learning by keeping the learning data of the client device and the supervised data of the server device mutually confidential, wherein, The client device obtains information from the server device about an unauthorized client device that is launching an attack that is hindering the learning of the AI ​​model in the server device. Based on the information about the unauthorized client device, the client device integrates the AI ​​model with other client devices. The server device derives information about the unauthorized client device and outputs the information about the unauthorized client device obtained through the deriving.

12. A machine learning method for a machine learning system, the machine learning system comprising a client device having inherent learning data and a server device having supervised data, wherein the client device and the server device each hold an AI (Artificial Intelligence) model, and the machine learning system promotes learning by keeping the learning data of the client device and the supervised data of the server device mutually confidential, wherein... The server device deduces the identity of an unauthorized client device that is launching an attack that is hindering the learning of the AI ​​model within the server device.

13. A machine learning program for a machine learning system having a client device with inherent learning data and a server device with supervised data, the client device and the server device each holding an AI (Artificial Intelligence) model, the machine learning system advancing learning by keeping the learning data of the client device and the supervised data of the server device mutually confidential, wherein... The machine learning program enables the computer to perform unauthorized client detection processing, in which unauthorized client devices that are conducting attacks that hinder the learning of the AI ​​model in the server device are identified.

Images