Information processing system, information processing device, information processing method, and program

Abstract

To perform learning of an inference model while securing privacy of medical data and confidentiality of the inference model.SOLUTION: An information processing system 800 includes a first information processing apparatus 900 to be operated by a person who provides training data, and a second information processing apparatus 1000 managed by a person who provides an inference model. The first information processing apparatus 900 includes: a training data acquisition unit 902 which acquires training data; and a first learning unit 903 which performs first learning processing by inputting the training data to a first partial model composed of an input layer and a partial intermediate layer of the inference model. The second information processing apparatus 1000 includes a second learning unit 1002 which performs second learning processing by inputting output of the first learning processing to a second partial model composed of layers of the inference model, which are different from those of the first partial model.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a program that perform learning processing on an inference model. [Background technology]

[0002] Systems are known that apply machine learning techniques to medical data, such as medical images acquired by medical imaging devices (modalities) and medical information acquired from medical information systems, to make inferences about specific diseases (disease detection, benign / malignant distinction, prognosis prediction, risk prediction, etc.).

[0003] Non-Patent Document 1 discloses a technology in which a model consisting of an encoder and decoder is trained using a medical image as input, and feature extraction is performed using an inference model (encoder section). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Behzad Bozorgtabar et al., “SALAD: Self-SuperviSed Aggregation Learning for Anomaly Detection on X-RayS”, International Conference on Medical Image Computing and Computer-ASSiSted Intervention-MICCAI2020. Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology described in Non-Patent Document 1, since the learning of the inference model is performed by an information processing device on the side of the provider of the inference model, it is necessary to transmit the teacher data managed by the user of the inference model to the information processing device on the side of the provider of the inference model, which makes it difficult to protect privacy. On the other hand, if the inference model is stored in an information processing device managed by the user of the inference model, it is difficult to prevent the user from copying the inference model and distributing it to a third party or from unauthorized use such as modifying the inference model, so the confidentiality of the inference model cannot be guaranteed.

[0006] The present invention aims to provide an information processing system that can learn an inference model while ensuring the privacy of training data and the confidentiality of the inference model. [Means for solving the problem]

[0007] In order to solve the above problems, an information processing system according to one aspect of the present invention includes a first information processing device operated by an administrator of teacher data, and a second information processing device capable of communicating with the first information processing device via a network, Multi-layer An information processing system that performs a learning process to learn an inference model based on a neural network, the inference model is constructed by combining any one of a plurality of first partial models with a second partial model common to the plurality of first partial models, the first partial model being constructed from an input layer and a part of an intermediate layer of the inference model, and the second partial model being constructed from a layer different from the first partial model of the inference model, The first information processing device includes a teacher data acquisition unit that acquires teacher data, and a teacher data acquisition unit that acquires the teacher data. Record number a first learning unit that performs a first learning process by inputting the first partial model into a second partial model, and the second information processing device Record number A second learning unit that performs a second learning process by inputting the second partial model into the second partial model. The first learning unit generates the plurality of first partial models. do. Effect of the Invention

[0008] According to the present invention, it is possible to train an inference model while ensuring the privacy of training data and the confidentiality of the inference model. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram showing the configuration of an information processing system according to a first embodiment. [Diagram 2] Schematic diagram of an inference model relating to the first embodiment. [Diagram 3] FIG. 2 is a diagram showing a hardware configuration of a first information processing apparatus according to the first embodiment. [Figure 4] FIG. 4 is a flowchart showing an inference process of the information processing system according to the first embodiment. [Diagram 5] FIG. 13 is a diagram showing the configuration of an information processing system according to a second embodiment. [Figure 6] Schematic diagram of an inference model relating to the second embodiment. [Figure 7] FIG. 11 is a flowchart showing an inference process of the information processing system according to the second embodiment. [Figure 8] FIG. 13 is a diagram showing the configuration of an information processing system according to a third embodiment. [Figure 9] Schematic diagram of an inference model relating to the third embodiment. [Figure 10] FIG. 11 is a flowchart showing a learning process of an information processing system according to a third embodiment. [Figure 11] FIG. 13 is a schematic diagram of an information processing system according to a fourth embodiment. [Figure 12] Schematic diagram of an inference model relating to the fourth embodiment. [Figure 13] FIG. 13 is a flowchart showing a learning process of an information processing system according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present invention can be preferably applied to medical data such as raw data (signal data) acquired by a modality and medical image data for diagnosis generated by image reconstruction from the raw data. The modality includes, for example, an X-ray CT device, an MRI device, a SPECT device, a PET device, and an electrocardiograph. Note that the data to be inferred and the teacher data may be not only medical data but also information related to the privacy of a patient, such as age, sex, and disease information.

[0011] Hereinafter, the inference process by the inference model in the information processing system of the present invention will be described in the first and second embodiments. In addition, in the third and fourth embodiments, the learning process of the inference model in the information processing system of the present invention will be described. Note that, as described later, the inference model used in the inference process is not limited to the inference model generated through the learning process of the third and fourth embodiments of the present invention. The inference model used in the inference process is a trained inference model trained based on machine learning or deep learning by a known method or a learning process described in the present invention. Here, the trained inference model may be a model in which a learning process has been performed so as to satisfy a predetermined condition, and the trained inference model may be used as a target for additional learning, transfer learning, fine tuning, etc. Therefore, a learning process by a learning process described later may be performed as additional learning of a trained inference model trained by a known method, or the learning process may be performed in the reverse order.

[0012] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0013] [First embodiment] Here, an information processing system 1 of the present invention will be described with reference to Fig. 1. The information processing system 1 of the present invention is composed of a first information processing device 2, a second information processing device 3, and a network 4 that communicatively connects both information processing devices.

[0014] Here, the configuration of the information processing system 1 in the present invention will be described with reference to FIG. 2. The information processing system 1 is composed of a first information processing device 2 and a second information processing device 3. The first information processing device 2 and the second information processing device 3 each have a partial model that is a part of an inference model that performs inference processing on medical data to be inferred and outputs the execution result. The inference model here is a trained model based on a neural network composed of an input layer, an intermediate layer, and an output layer. Parameters for outputting the inference result are determined by the learning processing, and a model that pairs the parameters with the network model is defined as the inference model.

[0015] The first information processing device 2 executes a first inference process using a first partial model consisting of an input layer and at least a part of the intermediate layers among the trained inference models described above, and the second information processing device 3 executes a second inference process using a second partial model consisting of a layer different from the first partial model among the trained inference models described above. The configuration of each information processing device will be described below.

[0016] Here, the first information processing device 2 is an information processing device that can be operated by a user of the inference model who has the authority to manage the medical data to be inferred, for example, a medical professional. On the other hand, the second information processing device 3 is an information processing device owned by a provider of the model who has the authority to manage the inference model used for inference. The second information processing device 3 exists in a server outside the first information processing device 2, and is configured to be able to communicate via a network 4.

[0017] The first information processing device 2 includes an acquisition unit 11 that acquires medical data to be inferred, a first inference unit 12 that performs a first inference process on the medical data to be inferred using a first partial model composed of an input layer and at least a part of the intermediate layers of a learned inference model based on a neural network composed of an input layer, an intermediate layer and an output layer, and an output unit 13 that outputs a first inference result by the first inference process to a second information processing device 3, which is another information processing device having a second partial model composed of a layer different from the layer constituting the first partial model of the inference process.

[0018] The first information processing device 2 also has a storage unit 10 that stores a first partial model, which is a part of the trained inference model, and medical data to be inferred. The first information processing device 2 also has an inference result acquisition unit 14 that acquires the result of the second inference by the second information processing device 3, which is another information processing device, and a display control unit 15 that displays the acquired inference result on a display device.

[0019] (Storage unit 10) The storage unit 10 stores a first partial model including an input layer of the trained inference model and medical data to be inferred. The storage unit 10 stores, as the first partial model, a network corresponding to the partial model and trained parameters corresponding to the network in association with each other. The medical data to be inferred may be medical data automatically transferred from a modality or an external image server. Note that a part of the trained inference model refers to a continuous part from one layer to another layer, but is not limited thereto, and may be a continuous part from one neuron to another neuron or an isolated neuron. The partial model may be a plurality of non-adjacent parts in the trained inference model.

[0020] (Acquisition part 11) The acquisition unit 11 acquires medical data of the inference target from the storage unit 10 and transmits the acquired medical data of the inference target to the first inference unit 12.

[0021] (First Inference Part 12) The first inference unit 12 acquires a first partial model from the storage unit 10, and performs a first inference using the first partial model on the medical data to be inferred. The first inference result using the first partial model is transmitted to the output unit 13. In this embodiment, the first partial model is a first partial model that is configured with an input layer and at least a part of the intermediate layers among the trained inference models, and transmits an output from the intermediate layer to the output unit 13. Here, the output of the intermediate layer is tensor information, and when the inference model is a model based on CNN, the output is a feature map.

[0022] (Output section 13) The output unit 13 transmits the first inference result to another information processing device, that is, the second information processing device 3. When there are multiple inference models, the output unit 13 outputs information on the inference model corresponding to the partial model used in the first inference to the second information processing device.

[0023] (Inference result acquisition unit 14) The inference result acquisition unit 14 acquires the inference result of the second inference process for the medical image data that is the inference target from the second information processing device 3. When the inference result acquisition unit 14 acquires the inference result, it transmits the inference result to the display control unit 15.

[0024] (Display control unit 15) The display control unit 15 controls displaying on a display device the inference results acquired by the inference result acquisition unit 14. The display device is a display attached to an information processing device, a mobile terminal of a hospital staff member via an external server, or the like.

[0025] Here, the first information processing device 2 may be configured by a computer including a processor, a memory, a storage, and the like. In this case, a program stored in the storage is loaded into the memory, and the processor executes the program, thereby realizing the functions and processing of the storage unit 10, the acquisition unit 11, the first inference unit 12, the output unit 13, the inference result acquisition unit 14, the display control unit 15, and the like. However, this configuration is not limited, and for example, all or part of the configuration of the first information processing device 2 may be realized by a dedicated processor (such as ASIC) or FPGA. Alternatively, part of the arithmetic processing may be executed by a processor such as a GPU or DSP. In addition, the first information processing device 2 may be configured by a single piece of hardware, or may be configured by multiple pieces of hardware. For example, the functions and processing of the first information processing device 2 may be realized by multiple computers working together using cloud computing or distributed computing.

[0026] 3 shows an example of a specific configuration of the first information processing device 2. In this example, the local information processing device 2 has a CPU 20, a GPU 21, a RAM 22, a ROM 23, and a storage device 24, which are connected via a system bus 25. In addition, the local information processing device 2 is connected to a display device 26 and an input device 27 such as a mouse and a keyboard.

[0027] By configuring the first information processing device 2 as described above, the user of the inference model, who is the administrator of the medical data, does not need to transmit the medical data to be inferred to an external information processing device, and the privacy of the medical data can be protected. In addition, the provider of the inference model can ensure the confidentiality of the inference model by installing only a part of the inference model in the first information processing device 2.

[0028] On the other hand, the second information processing device 3 is present in a server outside the first information processing device 2, and is configured with a second inference unit 71 that performs a second inference process using a second partial model composed of a layer different from the layer constituting the first partial model, among the trained inference models based on a neural network composed of an input layer, an intermediate layer, and an output layer, which performs an inference process on medical data, as an input, using a first partial model composed of an input layer and at least a part of the intermediate layers, of the trained inference models for the medical data to be inferred. The second information processing device 3 also has a storage unit 70 that stores the second partial model.

[0029] In this embodiment, the second partial model is composed of an intermediate layer different from the intermediate layer constituting the first inference model among the trained inference models, and an output layer. Note that the network configuration of the above-mentioned partial model is an example, and the number of partial models and the number of information processing devices can be changed as appropriate.

[0030] (Storage unit 70) The storage unit 70 stores a network corresponding to the second partial model and learned parameters corresponding to the network in association with each other. Note that the partial model refers to a continuous part from one layer to another layer, but is not limited thereto, and may be a continuous part from one neuron to another neuron or an isolated neuron. Also, the partial model may be a plurality of non-adjacent parts in the inference model.

[0031] (Second inference part 71) The second inference unit 12 acquires a second partial model from the storage unit 10, and performs a second inference using the second partial model on the medical data to be inferred. Then, it transmits the inference result by the second inference to the first information processing device 2. Here, when the selection information of the trained inference model is acquired from the first information processing device 2, it acquires a second partial model corresponding to the trained inference model, and performs the second inference. Here, in this embodiment, since the second partial model is a model having an output layer, the inference result is output in the second information processing device. On the other hand, the second partial model may be composed of only an intermediate layer, and the output by the intermediate layer may be transmitted to the first information processing device 2.

[0032] By configuring the second information processing device 3 as described above, the provider of the inference model can load only a portion of the trained inference model into the first information processing device 2 and keep that portion of the trained inference model within the information processing device 3 that the provider owns and manages, thereby ensuring the confidentiality of the inference model.

[0033] The information processing system consisting of the first information processing device 2 and the second information processing device 3 described above can protect the privacy of the medical data to be inferred while ensuring the confidentiality of the inference model that performs inference on the medical data to be inferred.

[0034] Here, the inference process by the information processing system 1 in this embodiment will be described with reference to FIG.

[0035] In step S40, the acquisition unit 11 in the first information processing device 2 acquires medical data of the inference target. When the acquisition unit 11 acquires the medical data of the inference target, the acquisition unit 11 transmits the acquired medical data of the inference target to the first inference unit 12, and proceeds to the next step.

[0036] In step S41, the first inference unit 12 in the first information processing device 2 executes a first inference process on the medical data to be inferred, using a first partial model composed of an input layer and at least a part of the intermediate layers. After executing the first inference process, the first inference unit 12 transmits an inference result of the first inference process to the output unit 13, and proceeds to the next step.

[0037] In step S42, the output unit 13 in the first information processing device 2 outputs the inference result of the first inference to the second information processing device 2. 3 and proceed to the next step.

[0038] In step S43, the second inference unit 71 in the second information processing device 3 performs the second inference process using a second partial model composed of layers different from the layers constituting the first partial model. In this embodiment, the second partial model is composed of an inference model including an intermediate layer different from the intermediate layer constituting the first partial model. The second partial model further has an output layer, and outputs an inference result for the medical data to be inferred. After transmitting the inference result for the medical data to be inferred to the first information processing device 2, the second inference unit 71 proceeds to the next step.

[0039] The second inference unit 71 may store the inference result by the second inference process in the storage unit 70 and present the inference result in response to an external access. The output destination is not limited to the first information processing device 2, and the result may be transmitted to a specified information terminal or a contact.

[0040] In step S44, the inference result acquisition unit 14 in the first information processing device 2 acquires the second inference result from the second inference unit 71 of the second information processing device 3. Upon acquiring the second inference result, the inference result acquisition unit 14 transmits the second inference result to the display control unit 15 and proceeds to the next step.

[0041] In step S45, the display control unit 15 causes the display device 25 to display the inference result for the medical data to be inferred. The inference result displayed on the display device 25 is an inference result obtained by performing a series of inference processes (first inference process and second inference process) by the first information processing device 2 and the second information processing device 3 on the medical data to be inferred.

[0042] The display control unit 15 may display the medical data to be inferred and the inference result in association with each other, or may display information on the inference model used in the inference.

[0043] Therefore, the information processing system 1 can perform inference while protecting the privacy of the medical data that is the subject of inference and ensuring the confidentiality of the inference model that performs inference on the medical data that is the subject of inference.

[0044] [Second embodiment] In the first embodiment, an information processing system has been described that is composed of a first information processing device 2 having a first partial model composed of an input layer and a part of an intermediate layer of a trained inference model based on a neural network composed of an input layer, an intermediate layer and an output layer, which performs inference processing on medical data, and a second information processing device 3 having a second partial model composed of a layer of the trained inference model different from the first partial model.

[0045] In this embodiment, the first information processing device 2 further has a third inference unit that performs inference using a third partial model including an output layer, and the configuration of an information processing system that can output to the first information processing device side will be described with reference to Fig. 5. Also, the inference model in this embodiment will be described with reference to Fig. 6.

[0046] In this section, differences from the first embodiment will be particularly described, and overlapping parts will be omitted as appropriate.

[0047] As in the first embodiment, the information processing system 1 in this embodiment includes a first information processing device 2, a second information processing device 3, and a network 4 that communicatively connects the information processing devices. The information processing system 1 is also configured from three partial models.

[0048] The first information processing device 2 on the user side has a trained inference model based on a neural network composed of an input layer, an intermediate layer and an output layer, which performs inference processing on medical data. The trained inference model has a first partial model composed of an input layer and a part of the intermediate layers, and a third partial model composed of a part of the intermediate layers and an output layer.

[0049] On the other hand, the second information processing device 3 on the administrator side of the inference model has a second partial model consisting of at least a part of the intermediate layer between the intermediate layer of the first partial model and the intermediate layer of the third partial model among the intermediate layers of the inference model. The number of partial models is variable, and it is sufficient that a partial model having an input layer and a partial model having an output layer are provided in the first information processing device 2 on the user side who has the authority to manage the medical data to be inferred. Also, the number of first information processing devices 2 is not important as long as they are information processing devices managed by the user side.

[0050] This information processing system 1 not only protects the privacy of the medical data that is the subject of inference while ensuring the confidentiality of the inference model for the medical data, but also protects privacy regarding the inference results, since the inference results are output by the user.

[0051] For example, in the case where an inference model performs image quality improvement, if the output layer is in an information processing device other than the user's side, it is difficult to protect the privacy of the medical data because an image with high image quality of the medical data to be inferred is generated in the second information processing device 3. Therefore, the first information processing device 2 has a partial model having an output layer, and inference is performed using the partial model, thereby protecting privacy regarding the output.

[0052] The information processing device 1 of this embodiment has at least a memory unit 10, an acquisition unit 11, a first inference unit 12, an output unit 13, an inference result acquisition unit 14, a display control unit 15, and further a third inference unit 51 that performs a third inference using the third partial model described above.

[0053] The information processing device 2 according to the present embodiment includes a storage unit 70 and a second inference unit 71. The second partial model used in the second inference unit 71 is configured from a network including an intermediate layer between the intermediate layer of the first partial model and the intermediate layer of the third partial model among the trained inference models.

[0054] The inference process in this embodiment will be described below with reference to FIG.

[0055] Up to step S42, the flow is the same as in the first embodiment, so the explanation will be omitted.

[0056] In step S73, the second information processing device 3 performs a second inference using the second partial model with the inference result by the first inference unit 12 as an input. The second partial model is composed of a network composed of an intermediate layer between the intermediate layer of the first partial model and the intermediate layer of the third partial model among the trained inference models. The second inference unit 71 transmits the output from the intermediate layer by the second partial model to the first information processing device as the result of the second inference, and proceeds to the next step.

[0057] In step S74, the third inference unit 51 in the first information processing device 2 performs a third inference using the third partial model with the second inference result as an input. The third inference unit 51 transmits the result of the third inference to the inference result acquisition unit 14 and proceeds to the next step.

[0058] In step S75, the inference result acquisition unit 14 acquires the third inference result as an inference result for the medical data that is the inference target, and transmits the acquired inference result to the display control unit 15.

[0059] Although explanation of step S45 will be omitted, the inference result displayed on the display device 25 is the inference result obtained by performing a series of inference processes (first inference process, second inference process, and third inference process) on the medical data to be inferred by the first information processing device 2 and the second information processing device 3.

[0060] This embodiment not only makes it possible to protect the privacy of the medical data that is the subject of inference while ensuring the confidentiality of the inference model for the medical data, but also makes it possible to perform inference processing while protecting privacy regarding the inference results.

[0061] Hereinafter, in the third and fourth embodiments, the learning process of the inference model in the present invention will be described. Note that the inference model used in the inference process described above is not limited to the inference model generated through the learning process of the third and fourth embodiments. In addition, the inference model in the following embodiments may be an inference model that has not been subjected to a learning process, or a trained inference model that has been subjected to a learning process.

[0062] [Third embodiment] In the third embodiment, the configuration for learning an information processing system 800 of the present invention will be described with reference to Fig. 8. The information processing system 800 is composed of a first information processing device 900 which is an information processing device on the user side of the inference model, a second information processing device 1000 which is an information processing device on the provider side of the inference model, and a network 1100 connecting the information processing devices. Also, with reference to Fig. 9, the network configuration of the inference model corresponding to each information processing device will be described.

[0063] The first information processing device 900 has a first partial model consisting of the input layer and at least a part of the intermediate layer among the inference models based on a neural network consisting of an input layer, an intermediate layer, and an output layer, which performs inference processing on medical data. The second information processing device 1000 has a second partial model having a second partial model consisting of a layer different from the layer constituting the first partial model among the inference models. In this embodiment, the second partial model is a partial model consisting of a part of the intermediate layer and an output layer. In this way, the first partial model including the input layer is provided in the first information processing device 900 on the user side, and the second partial model, which is a part of the inference model, is provided in the second information processing device 1000 on the provider side of the inference model, so that the learning process of the inference model can be performed while protecting the privacy of the medical data and ensuring the confidentiality of the inference model. Here, the first partial model may be configured as a public network, and the second partial model may be configured as a confidential network. The provider of the model can further increase the confidentiality of the inference model by making the second partial model a confidential network.

[0064] The first information processing device 900 has a storage unit 901 that stores teacher data and information on an inference model. It further includes a teacher data acquisition unit 902 that acquires teacher data from the storage unit 900, and a first learning unit 903 that learns a first partial model based on the acquired teacher data. The storage unit 900 may be configured by a storage device or the like managed by a user of the inference model. Furthermore, when the learning process of the first partial model is completed, the first learning unit 903 stores information on the learned first partial model in the storage unit 901.

[0065] The second information processing device 1000 has a memory unit 1001 that stores information on an inference model, and a second learning unit 1002 that learns a second partial model.

[0066] Here, the learning process performed by the learning unit refers to forward propagating training data to a partial model and updating the parameters of the partial model using error information acquired by the backpropagation method. The training data is composed of training data and correct answer labels, and the training data is, for example, medical image data, and the correct answer labels are information indicating an object reflected in the medical image data. The correct answer labels may be set as correct answer image data indicating what object is reflected for each pixel. The first information processing device 900 may be a second information processing device 1000 By transmitting model selection information to the other party, even if a plurality of first partial models and / or a plurality of second partial models exist, an appropriate model can be selected.

[0067] An example of a learning process of the information processing system 800 in this embodiment will be described below with reference to the flowchart in FIG.

[0068] In step S50, the teacher data acquisition unit 902 acquires teacher data in which learning data and a correct label are paired from the storage unit 901. The teacher data acquisition unit 902 transmits information on the learning data to the first learning unit 903, transmits the correct label to the second information processing device 1000, and then proceeds to the next step.

[0069] In step S51, the first learning unit 903 acquires the learning data transmitted from the teacher data acquisition unit 902 and information on the first partial model from the storage unit 901. Here, the first learning unit 903 may transmit information indicating the acquired first partial model to the second information processing device 1000.

[0070] In step S52, the second learning unit 1002 acquires information on the second partial model from the storage unit 1001 and information on the correct label from the teacher data acquisition unit 902.

[0071] In step S53, the first learning unit 903 inputs the learning data to the first partial model and forward propagates it, thereby performing a first learning process that is a part of the learning process. When the first learning process is completed, data generated by the first learning process, such as a tensor, is transmitted to the second learning unit 1002.

[0072] In step S54, the second learning unit 1002 inputs the parameters transmitted from the first learning unit 903 into the second partial model and forward propagates them, thereby carrying out a second learning process which is part of the learning process.

[0073] In step S55, the second learning unit 1002 compares the output of the second partial model by forward propagation using the second partial model including the output layer in the network configuration with the correct label, and obtains error information using a loss function. The second learning unit 1002 also determines whether the learning is completed. The second learning unit 1002 determines the end of the learning process based on whether the calculated error information is less than a predetermined value or whether the learning process has been performed a predetermined number of times. If the second learning unit 1002 determines that the learning process is ended, the flow ends. On the other hand, if it is determined that the learning process is to be continued, the process proceeds to step S56. Note that step S55 may be determined by the first learning unit 903 before the start of the first learning process.

[0074] In step S56, the second learning unit 1002 updates the parameters of the second partial model based on the error information calculated in step S55. Here, the parameters refer to, for example, weights and biases. Then, the error information is transmitted from the intermediate layer close to the output layer side to the input layer side by backpropagation (backpropagation method). The second learning unit 1002 transmits the error information from the intermediate layer close to the input layer side constituting the second partial model to the first learning unit 903, and then proceeds to the next step.

[0075] In step S57, the first learning unit 903 updates the parameters of the first partial model based on the error information transmitted from the second learning unit 1002. After updating the parameters of the first partial model, the process proceeds to step S53. As described in step S55, the first learning unit 903 may determine the end of the learning process at this timing.

[0076] The information processing system 800 of the present invention is configured in this way, and can learn an inference model while ensuring the privacy of medical data and the confidentiality of the inference model. The number of partial models is not limited to two, and the present invention is applicable as long as a partial model including an input layer is in an information processing device on the user side of the inference model. In addition, when there are multiple inference models, information for selecting a partial model corresponding to each of the multiple inference models may be transmitted from the first learning unit 903 to the second learning unit 1002. The selection of the partial model may be selected by the user, or may be selected by the information processing device according to the input data.

[0077] [Fourth embodiment] In the third embodiment, a learning process in an information processing system in which a first information processing device has a first partial model composed of an input layer and at least a part of an intermediate layer, and a second information processing device has a second partial model composed of a layer different from the first partial model has been described.

[0078] In this embodiment, the learning of an information processing system in which a first information processing device has a third partial model including at least a middle layer different from the first partial model and the second partial model, and an output layer will be described with reference to Fig. 11. Also, the network configuration of the inference model in this embodiment will be described with reference to Fig. 12. Note that the description of the overlapping parts with the third embodiment will be omitted as appropriate.

[0079] The information processing system 1200 in this embodiment is composed of a first information processing device 1300 which is an information processing device on the user side of the inference model, a second information processing device 1400 which is an information processing device on the provider side of the inference model, and a network 1100 connecting the information processing devices.

[0080] The first information processing device 1300 has a first partial model that includes the input layer and at least a part of the intermediate layer of an inference model based on a neural network that includes an input layer, an intermediate layer, and an output layer, and further has a third partial model that includes at least the output layer of the inference model.

[0081] The second information processing device 1400 has a second partial model consisting of at least a portion of the intermediate layers of the inference model.

[0082] According to the configuration of this embodiment, the first information processing device 1300 has a third partial model including an output layer in its configuration, so that the learning data and the correct answer label constituting the teacher data can be executed to learn the inference model without transmitting them to the second information processing device 1400. Furthermore, the provider of the inference model can ensure the confidentiality of the inference model by having a second partial model including at least a part of the intermediate layers constituting the inference model. In this embodiment, as long as the first information processing device 1300 has a partial model including an input layer and a partial model including an output layer, the number of partial models, the number of information processing devices, and the like can be appropriately designed.

[0083] The first information processing device 1300 has a storage unit 1301 that stores teacher data and information on an inference model. It also includes a teacher data acquisition unit 902 that acquires teacher data from the storage unit 1301, a first learning unit 1303 that learns a first partial model based on the acquired teacher data, and a third learning unit that learns a third partial model.

[0084] The second information processing device 1400 includes a memory unit 1001 that stores information on an inference model, and a second learning unit that learns a second partial model.

[0085] Here, the learning process refers to a series of processes that forward propagate the learning data constituting the teacher data to the partial model, and backpropagate (backpropagation method) the error information between the correct label and the output value by the output layer to update the parameters of the partial model. The teacher data is composed of the learning data and the correct label.

[0086] An example of a learning process of the information processing system 800 in this embodiment will be described below with reference to the flowchart in Fig. 13. Note that differences from Fig. 10 will be particularly described here.

[0087] In step S130, the third learning unit 1304 acquires information on the third partial model from the storage unit 1301 and information on the correct label from the teacher data acquisition unit 1302.

[0088] In step S132, the second learning unit 1402 inputs the parameters transmitted from the first learning unit 903 into the second partial model and forward propagates them, thereby carrying out a second learning process which is part of the learning process.

[0089] In step S133, the third learning unit 1304 inputs the parameters transmitted from the second learning unit 1402 into the third partial model and forward propagates them, thereby performing a third learning process which is a part of the learning process.

[0090] In step S134, the third learning unit 1304 compares the output of the third partial model by forward propagation using the third partial model including the output layer in the network configuration with the correct label, and obtains error information using a loss function. The third learning unit 1304 also determines whether the learning is completed. The third learning unit 1304 determines the completion of the learning process by determining whether the calculated error information is less than a predetermined value or whether the learning process has been performed a predetermined number of times. If the third learning unit 1304 determines that the learning process is completed, the flow ends. On the other hand, if the third learning unit 1304 determines that the learning process is to be continued, the process proceeds to step S53. The determination of the completion of the learning process in step S134 may be determined by the first learning unit 1303 before the start of the first learning process.

[0091] In step S135, the third learning unit 1304 updates the parameters of the third partial model based on the error information calculated in step S134. Here, the parameters refer to, for example, weights and biases. The third learning unit 1304 transmits the error information from the intermediate layer close to the output layer side to the input layer side by backpropagation. The third learning unit 1304 transmits the error information from the intermediate layer close to the input layer side constituting the third partial model to the second learning unit 1402, and then proceeds to the next step.

[0092] In step S136, the second learning unit 1402 updates the parameters of the second partial model based on the error information transmitted from the third learning unit 1304. The second learning unit 1402 transmits the error information from the intermediate layer closer to the output layer side to the input layer side by backpropagation, and transmits the output from the intermediate layer closer to the input layer side to the first learning unit 1303, and then proceeds to the next step.

[0093] In step S137, the first learning unit 1303 updates the parameters of the first partial model based on the error information transmitted from the second learning unit 1402. After updating the parameters of the first partial model, the process proceeds to step S53. As described in step S134, the first learning unit 1303 may determine the end of the learning process at this timing.

[0094] The information processing system 1200 of the present invention is configured in this way, and can perform learning processing of the inference model while ensuring the privacy of the medical data and the confidentiality of the inference model. Furthermore, there is no need to transmit the learning data and the correct answer label that constitute the teacher data from the information processing device on the user side of the inference model, and the confidentiality of the medical data can be further ensured.

[0095] The inference model learned by the third and fourth embodiments may be used as an inference model for performing the inference processing in the first and second embodiments. The third and fourth learning processes are also effective as a method for additional learning of the inference model for performing the inference processing.

[0096] [Variation 1] This modified example can be applied to the third and fourth embodiments. In the above-mentioned learning process, the learning process for updating the parameters of the partial models constituting the inference model has been described, but additional learning for the purpose of updating the parameters of only some of the partial models may be performed. For example, the parameters of the second partial model may be fixed and the parameters of the first partial model may be updated, or the parameters of the first partial model may be fixed and the parameters of the second partial model may be updated.

[0097] When the first learning unit 903 updates the parameters of the first partial model and the second learning unit 1002 does not update the parameters of the second partial model, the user of the inference model can perform learning processing that customizes the first partial model according to the data that the user wants to input, etc. For example, the first learning unit 903 performs additional learning on the first partial model that is specialized for the characteristics of the modality that acquires medical data and a specific shooting range, etc., while preventing unintended modification of the model for the provider of the model.

[0098] When the second learning unit 1002 updates the parameters of the second partial model and the first learning unit 903 does not update the parameters of the first partial model, it is expected that the accuracy and robustness of the second partial model under the management of the inference model provider can be improved.

[0099] Furthermore, a plurality of first partial models and a plurality of second partial models may be created by the above-mentioned additional learning, or a plurality of partial models may be appropriately combined and used.

[0100] [Variation 2] In the above-mentioned learning process, the case where the partial model including the output layer is in the first information processing device and the case where the partial model is in the second information processing device have been described.

[0101] In this modified example, a configuration is described in which, depending on the output of the inference model, it is set as to whether a partial model including an output layer is placed in an information processing device on the user's side of the inference model or on the provider's side of the inference model.

[0102] For example, when the output of the inference model is equivalent to the input learning data such as high resolution of the input data, the inference model including the output layer is placed on the information processing device on the user side of the inference model. On the other hand, when the output is an inference model that classifies or detects medical data, the inference model including the output layer is placed on the information processing device on the provider side of the inference model. By configuring a partial model according to the output of the inference model, it is possible to perform the learning process while balancing the machine resources and the confidentiality of the medical data. Note that configuring a partial model according to the output of the inference model is also effective in the inference process. For example, a first partial model that performs a first inference may be selected from a plurality of first models according to the medical data to be inferred, and a second classification model that performs a second inference may be selected from a plurality of second partial models according to the medical data to be inferred.

[0103] [Variation 3] In the above-mentioned learning process, the case where the learning process of the inference model is performed using the error backpropagation method has been described.

[0104] In this modified example, a case will be described in which the learning process for an inference model is performed using a learning method other than the backpropagation method.

[0105] For example, techniques such as Synthetic Gradient, which trains a model to estimate the gradient that will be obtained for each layer, Feedback Alignment, which uses a fixed random matrix when backpropagating errors, Target Prop, which propagates a target instead of an error, or any other technique are acceptable.

[0106] (Other Examples) The present invention can also be realized by executing the following process: That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the programs.

Claims

1. An information processing system comprising: a first information processing device operated by a provider of training data; and a second information processing device managed by a provider of an inference model capable of communicating with the first information processing device via a network, the information processing system performing a learning process for learning an inference model based on a multi-layer neural network, the inference model is configured by combining any one of a plurality of first partial models with a second partial model common to the plurality of first partial models, The first partial model is composed of an input layer and a part of an intermediate layer of the inference model, the second partial model is constructed in a different layer from the first partial model of the inference model, The first information processing device, A teacher data acquisition unit that acquires teacher data; a first learning unit that performs a first learning process by inputting the teacher data into the first partial model; The second information processing device, a second learning unit that performs a second learning process by inputting an output from the first learning process to the second partial model, The information processing system, wherein the first learning unit generates the plurality of first partial models.

2. The inference model further includes a third partial model including an output layer, The information processing system according to claim 1, characterized in that the first information processing device has a third learning unit that performs a third learning process by inputting an output from the second learning process into the third partial model.

3. The information processing system according to claim 2, characterized in that the third learning unit acquires correct labels constituting the teacher data from the teacher data acquisition unit, and acquires error information based on the correct labels and the output by the output layer.

4. 4. The information processing system according to claim 3, wherein the third learning unit updates parameters of a third partial model based on the error information by an error backpropagation method.

5. The second learning unit updates parameters of the second partial model based on error information transmitted from the third learning unit; and 5. The information processing system according to claim 4, wherein the first learning unit updates parameters of the first partial model based on error information transmitted from the second learning unit.

6. The second partial model includes an output layer of the inference model, The information processing system according to claim 1, characterized in that the second learning unit acquires a correct label related to the teacher data from the teacher data acquisition unit of the first information processing device, and acquires error information based on the correct label and the output from the output layer.

7. 7. The information processing system according to claim 6, wherein the second learning section updates parameters of the second partial model based on the error information.

8. The first partial model is a network for disclosure to providers of the training data, 8. The information processing system according to claim 1, wherein the second partial model is a network that is kept secret from a provider of the training data.

9. The information processing system according to any one of claims 1 to 8, characterized in that the second learning unit in the second information processing device generates a second partial model by additional learning with parameters of the first partial model fixed.

10. The information processing system according to any one of claims 1 to 9, characterized in that the first learning unit in the first information processing device generates the plurality of first partial models by additional learning with parameters of the second partial model fixed.

11. An information processing system according to any one of claims 1 to 10, characterized in that the second information processing device is an information processing device operated by the provider of the inference model.

12. 12. The information processing system according to claim 1, wherein the teaching data is medical data.

13. An information processing method for an information processing system comprising a first information processing device operated by a provider of training data, and a second information processing device managed by a provider of an inference model capable of communicating with the first information processing device via a network, the information processing system performing a learning process for learning an inference model based on a multi-layer neural network, comprising: the inference model is configured by combining any one of a plurality of first partial models with a second partial model common to the plurality of first partial models, the first partial model is configured with an input layer and a part of an intermediate layer of the inference model and is stored in the first information processing device; the second partial model is constructed in a layer different from that of the first partial model of the inference model and is stored in the second information processing device; The information processing method includes: A teacher data acquisition step of acquiring teacher data; a first learning step of performing a first learning process by inputting the teacher data into the first partial model; a second learning step of performing a second learning process by inputting an output from the first learning process to the second partial model; having An information processing method, comprising: generating the plurality of first partial models to be used in alternative combination with a second partial model learned in the second learning step.

14. A program for causing a computer to execute the information processing method according to claim 13.

Images