Information processing system

Abstract

To provide an information processing system capable of securing confidentiality of an inference model while protecting privacy of medical data.SOLUTION: An information processing system 1 includes a first information processing apparatus and a second information processing apparatus configured to communicate over a network. The first information processing apparatus includes a first partial model configured to include a first input layer, a first intermediate layer which is at least one of intermediate layers in a group of intermediate layers, and a first output layer. The second information processing apparatus includes a second partial model configured to include a second intermediate layer including an intermediate layer different from the intermediate layer constituting the first intermediate layer in the group of intermediate layers, and a second output layer. The second information processing apparatus includes: an inference target data acquisition unit configured to acquire inference target data; and an inference unit configured to perform predetermined inference processing on the inference target data by using the first partial model and the second partial model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system that performs inference processing or learning processing using an inference model. [Background Art]

[0002] Systems are known that apply machine learning technology to medical data such as medical images acquired by medical imaging apparatuses (modalities) or medical information acquired from medical information systems, and perform inference on a predetermined disease (disease detection, benign / malignant differentiation, prognosis prediction, risk prediction, etc.).

[0003] Non-Patent Document 1 discloses a technology in which when an image is input, image compression is performed by a local first information processing apparatus, and the compressed result is transferred to a second information processing apparatus provided in a remote cloud service to perform disease segmentation.

[0004] Non-Patent Document 2 discloses a technology in which a model composed of an encoder and a decoder is learned using a medical image as an input, and feature extraction is performed using an inference model (encoder unit). [Prior Art Documents] [Non-Patent Documents]

[0005] [Non-Patent Document 1] Zihao Liu et al.,“Orchestrating Medical Image Compression and Remote Segmentation Networks”,International Conference on Medical Image Computing and Computer-Assisted Intervention-MICCAI2020. [Non-Patent Document 2] Behzad Bozorgtabar et al., “SALAD: Self-Supervised Aggregation Learning for Anomaly Detection on X-Rays”, International Conference on Medical Image Computing and Computer-Assisted Intervention-MICCAI2020. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] In the technology described in Non-Patent Document 1, when performing medical image segmentation, the medical images that are the data to be inferred are sent to a cloud service. In this case, since medical data such as medical images are sent from within the hospital to an external information processing device, it is difficult to protect the privacy of medical data.

[0007] On the other hand, as in the technology described in Non-Patent Document 2, the inference model may be built on an information processing device managed by the user performing the inference. In such cases, it is difficult to prevent the user from copying and distributing the inference model to third parties, or from misusing the model by modifying it, and therefore the confidentiality of the inference model cannot be guaranteed.

[0008] The present invention aims to provide an information processing system that can ensure the confidentiality of inference models while protecting the privacy of medical data. [Means for solving the problem]

[0009] To solve the above problems, an information processing system according to one aspect of the present invention comprises a first information processing device and a second information processing device that can communicate with the first information processing device via a network, and uses an inference model based on a neural network configured to include a first input layer, a group of intermediate layers, a first output layer and a second output layer, wherein the first output layer and the second output layer are provided on different information processing devices. I recommend An information processing system that performs logical processing, It has an acquisition unit that acquires data to be used for inference, The first information processing device comprises a first input layer, a first intermediate layer which is at least some of the intermediate layers in the intermediate layer group, and a first output layer, forming a first partial model. It has a first inference unit that performs a first inference process using, The second information processing device is a second partial model comprising a second intermediate layer which includes an intermediate layer different from the intermediate layer which constitutes the first intermediate layer among the intermediate layer group, and a second output layer. It has a second inference unit that performs a second inference process using , The first inference unit outputs the first output information of the inference performed on the data to be inferred using the first input layer and the first intermediate layer to the second information processing unit, and inputs the second output information output by the second intermediate layer of the second information processing unit to the first output layer, thereby outputting the result of the first inference process from the first output layer to the first information processing unit, and the second inference unit inputs the second output information obtained by inputting the first output information acquired from the first information processing unit to the second intermediate layer to the second output layer, thereby outputting the result of the second inference process from the second output layer to the second information processing unit. . [Effects of the Invention]

[0010] According to the present invention, it is possible to protect the privacy of medical data while ensuring the confidentiality of the inference model. [Brief explanation of the drawing]

[0011] [Figure 1] A diagram showing the configuration of the information processing system according to the first and third embodiments. [Figure 2] A diagram showing the hardware configuration of the first information processing device according to the first embodiment. [Figure 3] A schematic diagram of the inference model according to the first embodiment. [Figure 4] A flowchart illustrating the inference process of the information processing system according to the first embodiment. [Figure 5] A diagram showing the configuration of the information processing system according to the second and fourth embodiments. [Figure 6] A schematic diagram of the inference model according to the second embodiment. [Figure 7] A flowchart illustrating the learning process of the inference model according to the second embodiment. [Figure 8] Configuration diagrams of the inference models according to the third and fourth embodiments. [Figure 9] A flowchart illustrating the inference process of the information processing system according to the third embodiment. [Figure 10] A flowchart illustrating the learning process of the information processing system according to the fourth embodiment. [Modes for carrying out the invention]

[0012] The present invention is preferably applicable to medical data such as raw data (signal data) acquired by modalities and diagnostic data generated from raw data by image reconstruction. Modalities include, for example, X-ray CT scanners, MRI scanners, SPECT scanners, PET scanners, and electrocardiographs. The data to be inferred and the training data may include not only medical data, but also information related to patient privacy, such as age, gender, and disease information. Furthermore, the target data is not limited to medical data. For example, image data depicting people, text data based on document data, and audio data based on speech are all acceptable as long as they can be processed for inference using a neural network.

[0013] Hereinafter, the inference process using an inference model in the information processing system of the present invention will be described in the first embodiment and the third embodiment. The first embodiment and the third embodiment differ from each other in the inference model that performs inference and the flow of inference processing. The learning process of the inference model in the information processing system of the present invention will be described in the second embodiment and the fourth embodiment. As will be described later, the inference model used in the inference process is not limited to an inference model generated through the learning process of at least one of the second embodiment and the fourth embodiment of the present invention. The inference model used in the inference process is a trained inference model learned based on machine learning or deep learning by a known method or the learning process described in the present invention. Here, the trained inference model only needs to have been subjected to learning processing so as to satisfy predetermined conditions, and the trained inference model may be used as a target for additional learning, transfer learning, fine tuning, or the like. Therefore, as additional learning of a trained inference model learned by a known method, learning processing by the learning process described later may be performed, or learning processing may be performed in the reverse procedure.

[0014] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0015] [First Embodiment] Here, the information processing system 1 of the present invention will be described with reference to Fig. 1. The information processing system 1 according to the present invention includes a first information processing apparatus 2 and a second information processing apparatus 3 capable of communicating with the first information processing apparatus 2 via a network.

[0016] Here, Fig. 2 shows an example of a specific configuration of the first information processing apparatus 2. In this example, the first information processing apparatus 2 includes a CPU 200, a GPU 201, a RAM 202, a ROM 203, and a storage device 204, which are connected via a system bus 205. A display device 206 and an input device 207 such as a mouse and a keyboard are connected to the first information processing apparatus 2. The second information processing apparatus 3 may have the same configuration, or may be configured from a part of the configuration of the first information processing apparatus 2.

[0017] The functional configuration of the information processing system 1 and the inference model in the present invention will be described below with reference to Figure 3. The information processing system 1 comprises a first information processing device 2 and a second information processing device 3 that can communicate with the first information processing device 2 via a network. Here, for example, the first information processing device 2 is an information processing device managed by the user of the inference model, and the second information processing device 3 is an information processing device managed by the provider of the inference model. The first information processing device 2 and the second information processing device 3 each have a submodel, which is part of an inference model that performs inference processing on medical data to be inferred and outputs the execution result. The inference model is a neural network comprising a first input layer, a group of hidden layers, a first output layer, and a second output layer, with the first output layer provided as part of the submodel of the first information processing device 2. The second output layer is provided as part of the submodel of the second information processing device 3. In the inference processing step, the inference model is a trained inference model that has been trained through a predetermined training process. Through the training process, parameters for outputting inference results are determined, and a model consisting of these parameters paired with a network model is defined as the inference model.

[0018] The first information processing device 2 has a first partial model comprising a first input layer, a first intermediate layer which is at least a portion of the intermediate layers in the intermediate layer group, and a first output layer, as part of the inference model described above. For example, as shown in Figure 3, the first intermediate layer has N1 intermediate layers, from the first intermediate layer to the N1th intermediate layer.

[0019] The second information processing device 3 also has a second partial model comprising a second intermediate layer that includes intermediate layers different from those constituting the first intermediate layer, and a second output layer. For example, as shown in Figure 3, the second intermediate layer has N2 intermediate layers from the N1+1 intermediate layer to the N1+N2 intermediate layer. Here, the first information processing device 2 and the second information processing device 3 may each have intermediate layers with common parameters. For example, in addition to the above, the first and second intermediate layers may have N3 common intermediate layers, not shown in Figure 3, from the N1+N2+1 intermediate layer to the N1+N2+N3 intermediate layer. The configuration of each information processing device will be described below.

[0020] The first information processing device 2 is an information processing device that can be operated by a user of the inference model who has the authority to manage the medical data to be inferred, such as a medical professional. On the other hand, the second information processing device 3 is an information processing device owned by the model provider who has the authority to manage the inference model used for inference. The second information processing device 3 is located on a server outside the first information processing device 2 and is configured to communicate via network 4.

[0021] The first information processing device 2 has a storage unit 20 that stores model information of the first partial model, inference target data, etc. The storage unit 20 may be composed of an external device to the first information processing device 2. The first information processing device also includes an inference target data acquisition unit 21 that acquires medical data to be inferred, and an inference unit that performs inference processing on the medical data to be inferred, and a first inference unit 22 that performs the first inference processing using the first partial model.

[0022] The second information processing device 3 includes a second inference unit 31, which is located on a server outside the first information processing device 2, receives the results of the first inference processing from the first information processing device 2, and performs the second inference processing using the second partial model.

[0023] The information processing system 1 according to the present invention performs inference processing by dividing an inference model based on a neural network comprising a first input layer, a group of hidden layers, a first output layer, and a second output layer between a first partial model of the first information processing device 2 and a second partial model of the second information processing device 3, thereby ensuring the confidentiality of the inference model while protecting the privacy of medical data. Specifically, the user of the inference model, who is the administrator of the data to be inferred, does not need to transmit the medical data to be inferred to an external information processing device, and can obtain inference results while protecting the privacy of medical data.

[0024] Furthermore, the provider of the inference model can ensure the confidentiality of the inference model by loading only a portion of the inference model into the first information processing device 2.

[0025] Here, the inference process by the information processing system 1 in this embodiment will be explained using Figure 4.

[0026] In step S40, the data acquisition unit 121 of the first information processing device 2 acquires the medical data to be inferred, transmits the acquired medical data to the first inference unit 22 which constitutes the inference unit, and proceeds to the next step.

[0027] In step S41, the inference unit performs a predetermined inference process using the first and second subset models. Specifically, it inputs the medical data to be inferred into the first input layer constituting the first inference unit 22, performs inference processing in the first to N1 intermediate layers, and transmits intermediate information to the second inference unit 31. The second inference unit 31 receives the intermediate information transmitted from the first inference unit 22 and performs inference processing using the N1+1 to N1+N2 intermediate layers. Then, it transmits the inference result from the N1+N2 intermediate layer to the second output layer constituting the second subset model and the first output layer constituting the first subset model, and proceeds to the next step.

[0028] In this embodiment, the N1+N2 layer is the final layer constituting the group of intermediate layers, and this is an example of the final intermediate layer. However, the N1+N2 layer does not have to be the final intermediate layer; the N1+N2+1 layer to the final intermediate layer may be common intermediate layers in each submodel. In this case, the inference result from the N1+N2 intermediate layer is transmitted to the N1+N2+1 layer constituting each submodel. Then, processing from each of the N1+N2+1 layers to the final intermediate layer is executed, and the inference result from the final intermediate layer is transmitted to each output layer.

[0029] In step S42, the first output layer of the first submodel and the second output layer of the second submodel each take intermediate information from the N+2th intermediate layer as input and output the inference result for a predetermined inference process. The output of the inference result to the second information processing device 3, which is the distributor of the inference model, has the effect of preventing unintended inferences. Specifically, if an image unintended by the provider of the inference model is input by a user who is the recipient of the inference model, for example, if a small amount of noise information is added and multiple inferences are performed, the inference process can be detected from the likelihood distribution of the output by the second output layer and the difference in the results of multiple inference processes by the second output layer. When the second information processing device 3 detects an unintended inference process, it stops the use of the inference model. Specifically, it does not allow the input of information to the intermediate layer provided only in the second information processing device 3. This configuration ensures the confidentiality of the information in the intermediate layer constituting the second submodel of the second information processing device 3.

[0030] The information processing system 1 described in the present invention, by performing the above-described inference model and inference processing steps, can ensure the confidentiality of the inference model that performs inference on the medical data to be inferred, while protecting the privacy of the medical data to be inferred.

[0031] [Second Embodiment] In the first embodiment, an inference process using an inference model based on a neural network comprising a first input layer, a group of hidden layers, a first output layer, and a second output layer was described. In this embodiment, a learning process using the inference model will be described.

[0032] The functional configuration of the information processing system 1 in this embodiment will be explained using Figure 5. Note that the learning process may be performed before inference in the inference model in the first embodiment, or the inference model may undergo additional learning after the learning process has been performed.

[0033] In this embodiment, the information processing system 1 has a storage unit 40 that stores training data and information about an inference model. Furthermore, it includes a training data acquisition unit 41 that acquires training data from the storage unit 40, and a first learning unit 42 that learns a first partial model based on the acquired training data. Here, the learning unit in the information processing system 1 includes a first learning unit 42 that processes the first partial model and a second learning unit 51 that processes the second partial model.

[0034] The learning unit is a unit that inputs training data constituting the training data into the first input layer of the inference model and trains the hidden layer group and the first output layer. The learning unit trains the hidden layer group and the first output layer by backpropagation using loss information calculated using the correct answer data constituting the training data and the first output layer, and trains the inference model by using the parameters related to the first output layer obtained by this training as parameters related to the second output layer.

[0035] With this configuration, the information processing system 1 can perform the training process for an inference model while ensuring the confidentiality of medical data such as training data.

[0036] The learning process of the inference model in this embodiment will be explained below using Figures 6 and 7.

[0037] In step S70, the training data acquisition unit 41 acquires training data, which consists of pairs of training data and correct answer data, from the storage unit 40. The training data acquisition unit 41 transmits the information of the training data to the first learning unit 42 and proceeds to the next step.

[0038] In step S71, the first learning unit 42 inputs the learning data to the first input layer, propagates it forward through the hidden layer group, namely the first hidden layer, the N1th hidden layer, the N1+12th hidden layer, and the N1L-N2nd hidden layer, calculates loss information using the output obtained through the first output layer and the correct data, and proceeds to the next step.

[0039] In step S72, the first learning unit 42 learns the first output layer and the hidden layer group using the backpropagation method with the calculated loss information, then transmits the parameters related to the first output layer to the second learning unit 51 and proceeds to the next step. Alternatively, the learning process up to this step may be repeated for a predetermined number of learning data or epochs before proceeding to the next step, or the next step may be executed after each epoch or after each predetermined learning process.

[0040] In step S73, the second learning unit 52 determines the parameters for the second output layer by reusing and copying the parameters for the first output layer as the parameters for the second output layer, and then terminates the learning process. Note that the parameters for the second output layer may not be a copy of the parameters for the first output layer, but may be determined by the second information processing device 4 using loss information from the first learning unit 42.

[0041] This configuration eliminates the need to transmit both the correct answer data and the training data that constitute the training data from the user's device of the inference model to an external source, thus ensuring the confidentiality of medical data such as training data. Furthermore, since a portion of the inference model is stored confidentially in the second information processing device 3, which is the information processing device on the provider's side of the inference model, the confidentiality of the inference model can also be ensured.

[0042] [Third Embodiment] This embodiment describes inference processing using an inference model having a different network configuration from the first embodiment. Configurations similar to those in the first embodiment will be omitted from explanation as appropriate. Regarding the functional configuration, in addition to Figure 1, the second information processing device 3 may further include a unit for acquiring inference target data for verification.

[0043] Here, the inference model in this embodiment will be explained using Figure 8. The inference model in this embodiment is an inference model based on a neural network, comprising a first input layer, a second input layer, a group of intermediate layers including the first intermediate layer, the N1st intermediate layer, the N1+1st intermediate layer, the N1+N2st intermediate layer, the N1+N2+1st intermediate layer, the N1+N2+N3st intermediate layer, and further comprising a first output layer and a second output layer. The information processing system 1 places a first partial model having part of the configuration of the inference model in the first information processing device 2, and places a second partial model having part of the configuration of the inference model in the second information processing device. The first partial model comprises a first input layer, a second input layer, and at least some of the intermediate layers from the group of intermediate layers, namely the first intermediate layer and the first output layer. For example, as shown in Figure 8, the first intermediate layer has N1 intermediate layers from the first intermediate layer to the N1st intermediate layer, and N3 intermediate layers from the N1+N2+1st intermediate layer to the N1+N2+N3st intermediate layer. The second submodel consists of a second input layer, a second hidden layer which includes a group of hidden layers different from that of the first hidden layer, and a second output layer. For example, as shown in Figure 8, the second hidden layer has N1+N2 hidden layers, from the first hidden layer to the N1+N2 hidden layer. In this case, the group of hidden layers from the N1+1 hidden layer to the N1+N2 hidden layer is different from that of the first hidden layer. When a user performs inference processing using the inference model, they perform inference processing using the first input layer, the group of hidden layers, and the first output layer provided in the first information processing device 2. On the other hand, when the model provider performs inference processing using verification data for inference, they perform inference processing using the second input layer, the group of hidden layers, and the second output layer.

[0044] Note that the intermediate layers that make up the intermediate layer group are not limited to this pattern.

[0045] For example, the first submodel does not necessarily have to have N3 hidden layers from the N1+N2+1 to the N1+N2+N3 hidden layer. Alternatively, the second submodel in the server-side second information processing device 3 may have the group of hidden layers that constitute the neural network, namely the 1st, N1st, N1+1, N1+N2+1, and N1+N2+N3 hidden layers, and the user-side first submodel of the inference model may be a submodel composed of some of the hidden layers from this group. With this configuration, the verification of the server-side inference model can be performed on the server-side information processing device.

[0046] Furthermore, while the number of intermediate layers constituting the intermediate layer group and the configuration of each information processing device can be combined as appropriate, by providing the first to N1 intermediate layer groups in the first partial model that constitutes the first information processing device 2, which is the user-side information processing device of the inference model, inference and learning become possible without transmitting medical data to the information processing device on the provider side of the inference model.

[0047] Furthermore, the number of intermediate layers constituting the intermediate layer group may vary, and there may be multiple instances of data being exchanged between intermediate layers and information processing devices.

[0048] With this configuration, the information processing system 1 can not only ensure the confidentiality of the data to be inferred and the inference model, but also ensure the confidentiality of the data to be inferred for verification by the provider of the inference model.

[0049] The inference flow by the information processing system 1 in this embodiment will be explained using Figure 9 below.

[0050] In step S900, the data acquisition unit 21 acquires the medical data to be inferred from the storage unit 20 and proceeds to the next step.

[0051] In step S901, the inference unit determines the corresponding path depending on whether the acquisition or input of the medical data to be inferred is to the first input layer of the first submodel constituting the first information processing device 2, or to the second input layer of the second submodel constituting the second information processing device 3. Specifically, if the medical data to be inferred is input to the first input layer, the unit performs inference processing using the first path, which uses the intermediate layer group and the first output layer. In the example in Figure 8, the intermediate layer group constituting the first path consists of the first intermediate layer to the N1 intermediate layer of the first submodel, the N1+1 intermediate layer to the N1+N2 intermediate layer of the second submodel, and the N1+N2+1 intermediate layer to the N1+N2+N3 intermediate layer of the first submodel. Furthermore, if the medical data to be inferred for verification is input to the second input layer, the unit performs inference processing using the second path, which uses the intermediate layer group and the second output layer. In the example in Figure 8, the intermediate layer group constituting the second path consists of the first intermediate layer to the N1+N2 intermediate layer of the second partial model and the N1+N2+1 intermediate layer to the N1+N2+N3 intermediate layer of the first partial model. That is, if the medical data to be inferred is input to the first input layer, the inference unit proceeds to step S902, and if the medical data to be inferred is input to the second input layer, it proceeds to step S903.

[0052] In step S902, the inference unit performs inference processing using the first path, with the first and second intermediate layers, which constitute the inference model, and the first output layer. With this configuration, the user of the inference model does not need to transmit the medical data to be inferred and the inference results to the second information processing device 3, which is an information processing device on the provider side of the inference model, and the provider of the inference model can ensure the confidentiality of the inference model by keeping a part of the intermediate layer confidential. The information processing system 1 terminates this flow after performing the inference processing.

[0053] In step S903, the inference unit performs inference processing using the second path, with the first and second intermediate layers, which constitute the intermediate layer group of the inference model, and the second output layer. With this configuration, the provider of the inference model can ensure the confidentiality of the inference model by concealing a part of the intermediate layers of the inference model, and even when the inference model is updated by a learning process such as the second embodiment or the fourth embodiment described later, it can perform inference on its own verification data without transmitting it to the first information processing device 2, which is the user-side information processing device of the inference model. This configuration also has the effect of preventing an unintended decrease in the accuracy of the inference model.

[0054] (Variation 1) In the above-described embodiment, a configuration was described in which the inference unit outputs the inference result using either the first output layer or the second output layer.

[0055] In this modified example, a configuration is described in which the inference unit outputs the inference result using both the first output layer and the second output layer. The inference processing described in the third embodiment is performed on the verification data for inference. Furthermore, the path of the inference processing is not limited to this example.

[0056] Furthermore, in this modified example, the tasks of the first output layer and the second output layer may be different. Specifically, the first output layer constituting the first partial model in the first information processing device 2, which is the user's information processing device, performs segmentation processing on image data, while the second output layer constituting the second partial model in the second information processing device 3, which is the information processing device on the provider's side of the inference model, performs classification processing on image data. With this configuration, the provider of the inference model can detect unintended use of the inference model due to multiple inference processes, and the user of the inference model can obtain the inference results of highly confidential tasks such as segmentation using only the first information processing device 2, thereby ensuring the confidentiality of medical data. In such cases, a multi-task inference model such as the publicly known Mask R-CNN can be applied as the inference model used. By placing some highly confidential tasks of the multi-task inference model on the user's information processing device of the inference model and some tasks on the provider's information processing device of the inference model, it is possible to prevent unintended inference processing by the inference model, in addition to ensuring the confidentiality of the inference model and the data to be inferred.

[0057] [Fourth Embodiment] In this embodiment, a learning process using an inference model having a different network configuration from the second embodiment will be described. Configurations similar to those in the second embodiment will be omitted from the description as appropriate. Regarding the functional configuration, in addition to Figure 5, the second information processing device may include a training data acquisition unit and a verification inference target data acquisition unit.

[0058] Here, the inference model in this embodiment will be explained using Figure 8. The inference model in this embodiment is a neural network-based inference model that, like the third embodiment, comprises a first input layer, a second input layer, a group of hidden layers consisting of the Nth hidden layer, the N+1th hidden layer, the N+2nd hidden layer, the N+3rd hidden layer, and further includes a first output layer and a second output layer.

[0059] Information processing system 1 places a first partial model having part of the inference model's configuration in the first information processing device 2, and places a second partial model having part of the inference model's configuration in the second information processing device. The first partial model includes a first input layer, a second input layer, a first intermediate layer which is at least part of the intermediate layer group, and a first output layer. The second partial model includes a second input layer, a second intermediate layer which includes an intermediate layer group different from the first intermediate layer, and a second output layer.

[0060] When a user trains an inference model, the training process is performed using the first input layer, hidden layer group, and first output layer provided in the first information processing device 2, which is the user's information processing device. The second output layer and the first hidden layer in the second information processing device 3 complete the training process by reusing the parameters related to the first output layer and the parameters of the first hidden layer in the first information processing device 2, which were determined by the training process.

[0061] On the other hand, when the model provider is learning, the learning process is performed using the second input layer, hidden layer group, and second output layer of the inference model, which is the information processing device on the model provider's side, the second information processing device 3. The first output layer and the first hidden layer in the first information processing device 2 complete the learning process by reusing the parameters related to the second output layer and the parameters related to the first hidden layer in the second information processing device 3 that were determined by the learning process.

[0062] With this configuration, Information Processing System 1 not only ensures the confidentiality of the inference model, but also eliminates the need to transmit either the ground truth data or the training data that constitute the training data to an external device, thereby ensuring the confidentiality of medical data. Furthermore, by having the model provider perform inference on the inference target data for model verification using the second input layer, hidden layer group, and second output layer, it is possible to verify whether the learning process by the user of the inference model and the learning process by the provider of the inference model are being performed appropriately.

[0063] The learning flow by the information processing system 1 in this embodiment will be explained below using Figure 10.

[0064] In step S1000, the training data acquisition unit 41 acquires training data from the storage unit 40 and proceeds to the next step.

[0065] In step S1001, the learning unit determines which inference process to execute among the inference models, depending on whether the acquisition of training data or the input of training data constituting the training data is an input to the first input layer of the first submodel constituting the first information processing device 2, or an input to the second input layer of the second submodel constituting the second information processing device 3. Specifically, if training data is input to the first input layer, the learning unit performs a learning process using the first path, which performs inference processing using the hidden layer group and the first output layer. If training data is input to the second input layer, the learning unit performs a learning process using the second path, which performs inference processing using the hidden layer group and the second output layer. That is, if training data is input to the first input layer, the learning unit proceeds to step S1002, and if training data is input to the second input layer, it proceeds to step S1004.

[0066] In step S1002, the learning unit forward propagates the learning data to the first input layer, the first hidden layer, the second hidden layer, and the first output layer. It also calculates the loss information between the output from the first output layer and the correct data, and determines the parameters for each layer by backpropagation based on the loss information.

[0067] In step S1003, the parameters related to the first output layer are copied and reused for the parameters related to the second output layer, and then the process is terminated.

[0068] In step S1004, the learning unit forward propagates the learning data to the second input layer, the first hidden layer, the second hidden layer, and the second output layer. It also calculates the loss information between the output from the second output layer and the correct data, and determines the parameters for each layer by backpropagation based on the loss information.

[0069] Step S100 5 In this, the parameters relating to the second output layer are related to the parameters relating to the first output layer. When the parameters are copied and reused, the process is terminated. Note that the first output layer and the second output layer The parameters related to the power layer do not necessarily have to be copies of the parameters related to one of the output layers. The loss information calculated by the information processing unit is used to determine, through learning, the other information processing unit. That's fine.

[0070] With this configuration, Information Processing System 1 not only ensures the confidentiality of the inference model, but also eliminates the need to transmit either the ground truth data or the training data that constitute the training data to an external device, thereby ensuring the confidentiality of medical data. Furthermore, by having the model provider perform inference on the inference target data for model verification using the second input layer, hidden layer group, and second output layer, it is possible to verify whether the learning process by the user of the inference model and the learning process by the provider of the inference model are being performed appropriately.

[0071] (Modification 2) In the learning processes described in the second and fourth embodiments, the case in which the inference model is trained using the backpropagation method was explained.

[0072] This modified example describes a case where the learning process for an inference model is performed using a learning method other than backpropagation.

[0073] For example, you could use methods like Synthetic Gradient, which trains a model to estimate the gradient that will be obtained for each layer; Feedback Alignment, which uses a fixed random matrix when backpropagating errors; Target Prop, which propagates the target rather than the error; or any other method.

[0074] (Other examples) Furthermore, the present invention can also be realized by performing the following process: that is, supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or CPU or MPU, etc.) of that system or device read and execute the program.

Claims

1. The system comprises a first information processing device and a second information processing device that can communicate with the first information processing device via a network, An inference model based on a neural network comprising a first input layer, a group of hidden layers, a first output layer, and a second output layer, wherein the first output layer and the second output layer are provided on different information processing devices, and the inference is performed using this inference model, It has an acquisition unit that acquires data to be used for inference, The first information processing device is The inference unit has a first inference unit that performs a first inference process using a first partial model comprising the first input layer, a first intermediate layer which is at least some of the intermediate layers of the intermediate layer group, and the first output layer. The second information processing device is The second inference unit performs a second inference process using a second partial model that includes a second intermediate layer, which is different from the intermediate layer that constitutes the first intermediate layer among the group of intermediate layers, and the second output layer. The first inference unit outputs first output information of the inference performed on the data to be inferred using the first input layer and the first intermediate layer to the second information processing device, and inputs second output information output by the second intermediate layer of the second information processing device to the first output layer, thereby outputting the result of the first inference process from the first output layer to the first information processing device. The second inference unit inputs the second output information, obtained by inputting the first output information acquired from the first information processing device into the second intermediate layer, into the second output layer, thereby outputting the result of the second inference process from the second output layer to the second information processing device. An information processing system characterized by the following:

2. The neural network further has a second input layer, The information processing system according to claim 1, characterized in that the second partial model further includes the second input layer.

3. When the inference target data is input to the second input layer, The information processing system according to claim 2, characterized in that the second inference unit inputs the output information of the inference performed on the data to be inferred using the second input layer and at least the second intermediate layer to the second output layer, thereby outputting the result of the second inference process from the second output layer to the second information processing device.

4. A training data acquisition unit that acquires training data, The system further comprises a learning unit that learns the first submodel and the second submodel using the aforementioned training data, The aforementioned learning unit, A learning unit that inputs the training data constituting the training data into the first input layer of the inference model and trains the intermediate layer group and the first output layer, The information processing system according to claim 1, characterized in that the intermediate layer group and the first output layer are trained by backpropagation using loss information calculated using the correct answer data constituting the training data and the first output layer, and the inference model is trained by using the parameters of the first output layer obtained by the training as parameters of the second output layer.

5. The neural network further comprises a second input layer, The second partial model further includes the second input layer, When the learning unit receives the training data constituting the training data as input to the first input layer, The inference model is trained by using the loss information calculated using the correct data constituting the training data and the first output layer, the intermediate layer group and the first output layer are trained by backpropagation, and the parameters related to the first output layer obtained by this training are used as parameters related to the second output layer. When the training data constituting the training data is input to the second input layer, The information processing system according to claim 4, characterized in that the intermediate layer group and the second output layer are trained by backpropagation using loss information calculated using the correct answer data constituting the training data and the second output layer, and the inference model is trained by using the parameters of the second output layer obtained by the training as parameters of the first output layer.

Images