Information processing systems, information processing methods, information terminals, and programs

Abstract

To provide an information processing system capable of inferring by selecting at least one information processing device from a plurality of information processing devices.SOLUTION: An information processing system includes an information terminal 110 that acquires medical image data from a medical imaging device 110, a plurality of information processing devices 130, 132, 134, etc. each having an inference unit that performs inference on medical image data using a trained model, and a selection unit 406 that selects at least one information processing device from the plurality of information processing devices 130, 132, 134, etc. The information terminal 110 transmits the medical image data to the information processing device selected by the selection unit 406. The information processing device selected by the selection unit 406 performs inference on the transmitted medical image data to transmit an inference result to the information terminal 110.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , ,

[0005] , , ,

[0001] The present invention relates to an information processing system, an information processing method, an information terminal, and a program for performing inference using a learned model.

Background Art

[0002] A computer-aided diagnosis (CAD) system that analyzes medical data acquired by a medical imaging device and presents diagnostic support information to a doctor is known. The CAD system applies machine learning technology to, for example, medical image data among medical data and outputs diagnostic support information.

[0003] Patent Document 1 discloses that, in order to improve the reliability of analysis based on machine learning, a plurality of processed medical signals regarding a subject are acquired by executing at least one of different imaging methods and different signal processes, and inference is performed using a plurality of learned models for each of the plurality of processed medical signals.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document 1 does not assume a plurality of information processing devices having an inference unit that performs inference on medical image data. Therefore, a plurality of information processing devices cannot be used for performing inference.

[0006] Therefore, an object of the present invention is to provide an information processing system capable of selecting at least one information processing device from a plurality of information processing devices and performing inference.

Means for Solving the Problems

[0007] This invention The information processing system related to this The system comprises multiple information processing devices, each having an information terminal for acquiring medical image data from a medical imaging device and an inference unit for performing inference on the medical image data. death Each of the plurality of information processing devices is capable of inferring The first point regarding the diagnostic purpose The information terminal includes a storage unit for storing supplementary information including information, and the information terminal includes a selection unit for selecting at least one information processing device from the plurality of information processing devices, and the information terminal includes the supplementary information obtained from each of the plurality of information processing devices and the operator Regarding diagnostic purposes Based on the first instruction, among the plurality of information processing devices The first information corresponding to the diagnostic purpose specified by the first instruction is provided. The system presents at least one information processing device to the operator as a candidate for the information processing device, and the selection unit selects an information processing device that performs inference on the medical image data based on a second instruction from the operator for selecting the candidate for the information processing device, transmits the medical image data to the information processing device selected by the selection unit, the selected information processing device performs inference on the transmitted medical image data, and transmits the inference result to the information terminal. [Effects of the Invention]

[0008] According to the present invention, inference can be performed by selecting at least one information processing device from a plurality of information processing devices. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing the configuration of the information processing system of the present invention. [Figure 2] A diagram showing the configuration of the information processing device of the present invention. [Figure 3] A diagram showing the configuration of the storage unit in the information processing device of the present invention. [Figure 4] A diagram showing the configuration of the information terminal of the present invention. [Figure 5] A diagram showing one display configuration of the display unit of the present invention. [Figure 6] A diagram showing one display configuration of the display unit of the present invention. [Figure 7] A diagram illustrating the properties of multiple information processing devices of the present invention. [Figure 8] A diagram illustrating the operation of the inference phase of the present invention. [Modes for carrying out the invention]

[0010] Preferred embodiments of the present invention will be described below with reference to the attached drawings. [Examples]

[0011] Figure 1 shows the configuration of the information processing system of the present invention. The information processing system of the present invention consists of a medical imaging device 100 for acquiring medical data about a subject, an information terminal 110, a network 120, and a plurality of information processing devices 130, 132, and 134. The plurality of information processing devices may be four or more information processing devices. The information terminal 110 includes a function to select at least one information processing device from the plurality of information processing devices.

[0012] The information terminal 110 is connected to an operation unit 112 and a display unit 114. The operation unit 112 receives various instructions from the operator and transmits these instructions to the information terminal 110 and the medical imaging device 100. The operation unit 112 consists of, for example, a mouse, keyboard, buttons, panel switches, foot switches, trackballs, joysticks, etc. The display unit 114 displays a GUI for inputting various instructions from the operation unit 112, and also displays medical image data based on medical data acquired by the medical imaging device 100.

[0013] In Figure 1, the information terminal 110 is shown with a separate operation unit 112 and display unit 114. However, the information terminal 110 may have the functions of both the operation unit 112 and the display unit 114 built-in.

[0014] The medical imaging device 100 is a device that acquires medical data of a subject, such as an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or an ultrasonic diagnostic device.

[0015] The X-ray CT device includes an X-ray source and an X-ray detector. By rotating the X-ray source and the X-ray detector around the subject and irradiating the subject with X-rays from the X-ray source and projecting the data detected by the X-ray detector, CT image data is generated.

[0016] The MRI device generates MRI image data by generating a predetermined magnetic field for a subject placed in a static magnetic field and performing Fourier transform on the acquired data.

[0017] The ultrasonic diagnostic device transmits ultrasonic waves to a subject, receives the ultrasonic waves that are reflected waves from the subject, and generates ultrasonic image data.

[0018] The medical data (such as CT image data, MRI image data, ultrasonic image data, etc.) generated by the medical imaging device 100 is three-dimensional data (volume data) or two-dimensional data. The medical data is, for example, medical image data regarding a subject. The medical image data includes raw data. The medical image data may be moving image data composed of a plurality of frame data. Also, the medical data includes measurement data obtained by performing various measurements using the medical image data.

[0019] The medical imaging device 100 is connected to an information terminal 110. The information terminal 110 is a PC terminal, a mobile phone such as a smartphone, a notebook terminal, a tablet terminal, etc. The information terminal 110 can also set subject information and associate the medical data acquired from the medical imaging device 100 with the subject information. The information terminal 110 can also display various data such as the medical data and measurement data acquired from the medical imaging device 100.

[0020] Information terminal 110 and multiple information processing devices 130, 132, 134... are connected to network 120. Network 120 includes communication networks outside the hospital, such as wireless communication (Wi-Fi), the internet, wireless base stations, providers, and communication lines. Network 120 may also include an intranet, which is a communication network within the hospital. Information terminal 110 can connect to and communicate with multiple information processing devices 130, 132, 134... via network 120. Information terminal 110 can transmit medical data (including medical image data) to multiple information processing devices 130, 132, 134.... Multiple information processing devices 130, 132, 134... can transmit inference results obtained using the medical data (including medical image data) to information terminal 110.

[0021] Here, the information terminal 110 includes a selection unit that selects at least one information processing device from a plurality of information processing devices 130, 132, 134, etc. In this case, let's assume that information processing device 130 is selected. The information terminal 110 transmits medical image data to the information processing device 130 selected by the selection unit. The information processing device 130 performs inference on the transmitted medical image data and transmits the inference result to the information terminal 110.

[0022] Figure 2 shows the configuration of the information processing device 130 of the present invention. The other information processing devices 132, 134, etc. have the same configuration as the information processing device 130. Here, we will describe the information processing device 130.

[0023] The information processing device 130 includes a training data generation unit 200 that generates training data using medical image data, a learning unit 202 that performs learning about medical image data using the training data generated by the training data generation unit 200, a storage unit 204 that stores the trained model generated by the learning unit 202, and an inference unit 206 that performs inference using the trained model.

[0024] The components (functions) of the information processing device 130 are realized, for example, by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) executing a program (software) stored in memory.

[0025] The information processing device 130 has a processor and memory inside. The processor can execute each process of the information processing device 130 according to the program stored in memory, and can function as a training data generation unit 200, a learning unit 202, a storage unit 204, an inference unit 206, and so on.

[0026] The training data generation unit 200 is connected to the network 120 and can acquire medical data, including medical image data and measurement data. The training data generation unit 200 generates training data using medical image data. The training data generated by the training data generation unit 200 is determined according to the inference task and classification target performed by the neural network.

[0027] Examples of inference tasks that neural networks can perform include classification tasks, which classify medical image data into different classes; detection tasks, which determine what is present and at what location in medical image data; and segmentation tasks, which extract target regions from medical image data.

[0028] When training a neural network to perform a classification task, the training data generation unit 200 generates training data consisting of pairs of medical image data and correct labels that indicate the objects captured in the medical image data.

[0029] On the other hand, when performing the detection task using a neural network, the training data generation unit 200 generates training data consisting of pairs of medical image data and ground truth images to which the medical image data has been assigned a Region of Interest (ROI) indicating the location of the object captured in the medical image data and a ground truth label that identifies the object.

[0030] When the task performed by the neural network is segmentation, the training data generation unit 200 generates training data consisting of medical image data and ground truth images, which are pairs of medical image data with the positional information of the pixels of the object captured in the medical image data and the ground truth label that identifies the object.

[0031] For example, when training a neural network to perform the task of segmenting the presence, type, and region of a lesion from medical image data acquired from an information terminal 110, the training data generation unit 200 generates training data that pairs medical image data containing a lesion region with ground truth label information indicating the type of lesion, the position information of the pixels of the lesion, and the corresponding ground truth image.

[0032] Furthermore, the training data generation unit 200 may perform preprocessing of medical image data in accordance with the neural network being trained by the learning unit 202. For example, if the target of inference by the neural network is medical image data, the training data generation unit 200 may perform processing such as noise reduction, filtering, image cropping, and resolution change on the acquired medical image data. If the target of inference is natural language such as text, the training data generation unit 200 performs preprocessing of the data to be processed in accordance with the target of inference by the neural network and the task, such as performing morphological analysis and applying vector transformation techniques.

[0033] In Figure 2, the training data generation unit 200 is shown as being located inside the information processing device 130, but the information terminal 110 may also have the training data generation unit 200 located inside. In other words, the training data generation unit 200 may be included as part of the configuration of the information terminal 110. For example, after generating the training data as described above in the information terminal 110, the learning unit 202 of the information processing device 130 may perform training on the inference engine via the network 120.

[0034] The learning unit 202 is connected to the training data generation unit 200. The learning unit 202 generates a trained model by learning medical image data in association with training data using a neural network. Here, the trained model represents the parameters determined by performing the training process up to a predetermined standard and the model information corresponding to those parameters. The trained model may be used to train other models as a transfer learning, or further training processing may be performed on the trained model.

[0035] A neural network contains multiple layers. In particular, a type of deep learning technique called a CNN (Convolutional Neural Network), although not shown in the diagram, has multiple hidden layers between the input and output layers. These hidden layers include convolutional layers, pooling layers, upsampling layers, and synthesis layers. The convolutional layer performs convolution on the input data. In the convolutional layer, the input medical image data is convolved to extract features from the medical image data.

[0036] A pooling layer is a layer that processes input values ​​by decimating or combining them to reduce the number of output values ​​to less than the number of input values. An upsampling layer is a layer that processes input values ​​by duplicating them or adding interpolated values ​​to them to increase the number of output values ​​to more than the number of input values. A synthesis layer is a layer that takes input values ​​from multiple sources, such as the output values ​​of a certain layer or the pixel values ​​that make up medical image data, and processes them by concatenating or adding them together. The number of hidden layers can be changed as needed depending on the learning content.

[0037] The memory unit 204 is connected to the learning unit 202. The memory unit 204 stores a trained model that has been trained to extract, for example, the type and region of a lesion in medical image data. The trained model is generated using a neural network, for example, but in addition to CNNs, RNNs (Recurrent Neural Networks), and models derived from CNNs and RNNs, which are deep learning technologies among neural network technologies, other machine learning technologies such as support vector machines, logistic regression, and random forests may be used, or rule-based methods may be used.

[0038] The inference unit 206 is connected to the network 120 and can acquire medical data, including medical image data and measurement data. The inference unit 206 is connected to the storage unit 204 and can perform inference using a trained model stored in the storage unit 204.

[0039] The inference unit 206 performs inference on newly generated medical image data using a pre-trained model that has been trained to extract, for example, the type and region of a lesion in the medical image data. If there is a lesion in the newly generated medical image data, the inference unit 206 can output the type and region of the lesion.

[0040] Figure 3 shows the configuration of the memory unit 204 of the present invention. The memory unit 204 stores the trained model and associated information learned in the learning unit 202. Specifically, the memory unit 204 adds the first associated information 310 to the first trained model 300 learned in the learning unit 202 and stores the first trained model 300. The memory unit 204 adds the second associated information 312 to the second trained model 302 learned in the learning unit 202 and stores the second trained model 302. The memory unit 204 adds the third associated information 314 to the third trained model 304 learned in the learning unit 202 and stores the third trained model 304.

[0041] In Figure 3, three pre-trained models are shown being stored in the memory unit 204 along with their associated information. However, the memory unit 204 can also store four or more pre-trained models along with their associated information. Each of the multiple pre-trained models differs in one of the following: the inference task, the class to be classified, the model structure, or the training data. Each pre-trained model can be specified or identified based on the associated information attached to it. The inference unit 206 selects an appropriate pre-trained model based on the associated information in response to input from the information terminal 110, and uses the selected pre-trained model to perform inference processing on the data to be inferred acquired from the information terminal 110.

[0042] The supplementary information includes details such as the body parts that can be inferred using the trained model, the type of medical imaging device (type of medical image data), the diagnostic purpose (diagnostic items), and the type of training data. Further details of the supplementary information will be described later.

[0043] Figure 4 shows the configuration of the information terminal 110 of the present invention. The information terminal 110 has functions such as selecting an information processing device, performing preprocessing for inference processing, and sending and receiving information. The information terminal 110 does not perform inference processing, and the inference processing is performed by the information processing devices 130, 132, 134, etc.

[0044] The information terminal 110 includes an image acquisition unit 400 that acquires medical image data from the medical imaging device 100, a processing unit 402 that performs various processing on the medical image data, and a transmitting / receiving unit 404 that transmits the medical image data processed by the processing unit 402 to an external device such as an information processing device 130 via the network 120. The transmitting / receiving unit 404 receives information transmitted from an external device such as an information processing device 130 via the network 120. The display unit 114 displays a medical image based on the medical image data and information transmitted from an external device such as an information processing device 130. The processing unit 402 performs noise reduction processing, grayscale conversion processing, etc., on the medical image data acquired by the image acquisition unit 502.

[0045] Furthermore, the information terminal 110 includes a selection unit 406 that selects at least one information processing device from a plurality of information processing devices 130, 132, 134, etc. The selection unit 406 selects an information processing device to transmit medical image data from the information terminal 110 and perform inference. The transmitting / receiving unit 404 receives the information transmitted from the information processing device selected by the selection unit 406 via the network 120. In other words, the selection unit 406 selects an information processing device to perform inference on medical image data. If the plurality of information processing devices 130, 132, 134, etc. are located in different clouds and the cloud connection formats are different, the transmitting / receiving unit 404 changes the cloud connection format (physical connection type, logical connection type, etc.). In this way, the cloud connection format is changed and the transmitting / receiving unit 404 connects to the information processing device selected by the selection unit 406. A physical connection type is, for example, a format in which the router corresponding to the transmitting / receiving unit 404 and the public cloud are physically connected by a dedicated line. The logical connection type is a configuration in which the transmitting / receiving unit 404 is connected, for example, via a PDN (Packet Data Network) connection.

[0046] Figure 5 shows an example of the screen display configuration in the display unit 114 of the present invention. The display unit 114 displays a selection screen 504 for selecting an information processing device. The operator can select at least one information processing device from a plurality of information processing devices displayed on the selection screen 504. The selection screen 504 becomes a selection unit 406 for selecting an information processing device.

[0047] The display unit 114 displays selection screens 500 and 502 for selecting a diagnostic site and a diagnostic purpose corresponding to the diagnostic site. Here, the lungs, liver, heart, and kidneys are displayed as diagnostic sites. These are the sites that can be inferred by multiple information processing devices 130, 132, 134, etc. connected to the network 120. The information terminal 110 obtains supplementary information from the multiple information processing devices regarding the sites that each information processing device can infer. When the information terminal 110 performs (requests) inference on medical image data, it queries the multiple information processing devices 130, 132, 134, etc. connected to the network 120. The information terminal 110 then obtains the sites that the multiple information processing devices 130, 132, 134, etc. can infer from the multiple information processing devices 130, 132, 134, etc. Note that the information terminal 110 may pre-store supplementary information regarding the sites that each information processing device can infer. Furthermore, the information terminal 110 may acquire supplementary information from multiple information processing devices 130, 132, 134, etc., regarding the type of medical imaging device that each information processing device can infer, the diagnostic purpose (diagnostic items), the type of training data, the recognition rate, the number of inferences, satisfaction level, and so on.

[0048] The information terminal 110 acquires supplementary information from multiple information processing devices regarding the body parts that each information processing device can infer, and the display unit 114 displays the body part information. In other words, the display unit 114 displays the body parts that each information processing device can infer based on the supplementary information from each information processing device. Therefore, the operator can recognize that the lungs, liver, heart, and kidneys can be inferred, but other body parts such as the pancreas and head cannot be inferred.

[0049] The selection screen 500 displayed on the display unit 114 shows tags corresponding to the diagnostic area. The selection screen 502 displays icons corresponding to the diagnostic purpose (diagnostic item). The operator can select a diagnostic area by selecting the corresponding tag from the selection screen 500. The operator can then select a diagnostic purpose by selecting the corresponding icon from the selection screen 502.

[0050] The selection screen 502 displayed on the display unit 114 shows pneumonia, pulmonary nodules, lung cancer, and ALL as diagnostic objectives. These are the diagnostic objectives that can be inferred by multiple information processing devices 130, 132, 134, etc., connected to the network 120. The information terminal 110 has previously acquired supplementary information from the multiple information processing devices regarding the diagnostic objectives that each information processing device can infer. The information terminal 110 then acquires supplementary information from the multiple information processing devices regarding the diagnostic objectives (lesion information) that each information processing device can infer. The information terminal 110 may also store supplementary information regarding the diagnostic objectives (lesion information) that each information processing device can infer. The display unit 114 can display the diagnostic objectives that can be inferred based on the supplementary information from each information processing device. Therefore, the operator can recognize that pneumonia, pulmonary nodules, and lung cancer can be inferred, but other diagnostic objectives such as pneumothorax cannot be inferred.

[0051] By selecting an icon corresponding to the diagnostic objective, the operator can extract information processing devices that detect pneumonia, pulmonary nodules, or lung cancer. Alternatively, by selecting the ALL icon, the operator can extract information processing devices that can handle all types of lung diseases.

[0052] Here, the icon corresponding to lung cancer is selected. The information terminal 110 can extract the information processing device corresponding to lung cancer by referring to the supplementary information (diagnostic purpose) of multiple information processing devices (storage units). The selection screen 504 displays information processing devices that can infer lung cancer. Here, information processing devices 134, 136, and 138 are displayed. The display unit 114 can display multiple information processing devices 134, 136, and 138 corresponding to the diagnostic purpose of lung cancer. The operator selects at least one information processing device from the multiple information processing devices 134, 136, and 138. The operator can select an information processing device using the cursor 506.

[0053] If the information processing device 138 is selected in the selection unit 406 of the information terminal 110, the information terminal 110 transmits medical image data to the information processing device 138 selected in the selection unit 406. The information processing device 138 performs inference on the transmitted medical image data and transmits the inference result to the information terminal 110. At this time, the information processing device 138 performs inference on the medical image data transmitted from the information terminal 110 using a trained model that has been trained to extract lung cancer from the medical image data. If there is a lung cancer lesion in the medical image data transmitted from the information terminal 110, the information processing device 138 transmits the diagnosis name and region information related to lung cancer to the information terminal 110.

[0054] The display unit 114 displays a medical image 510 based on medical image data. The medical image 510 consists of multiple slice images of a CT image (three-dimensional volume data). The information processing device 138 performs inference on the medical image data transmitted from the information terminal 110 using a trained model corresponding to lung cancer. As shown in Figure 5, for example, if the inference unit of the information processing device 138 detects lung cancer in the CT image, the information processing device 138 transmits the diagnostic name and region information related to lung cancer to the information terminal 110. The display unit 114 displays the detection information 514 indicating that lung cancer has been detected. The display unit 114 also displays the region 512 in which lung cancer was detected.

[0055] The selection screen 504 displays multiple information processing devices 134, 136, and 138 capable of inferring lung cancer. The selection unit 406 of the information terminal 110 may select at least one information processing device based on the supplementary information of the multiple information processing devices 134, 136, and 138. Specifically, the selection unit 406 of the information terminal 110 can also automatically select an information processing device from information (parameters) based on the supplementary information of the multiple information processing devices 134, 136, and 138. For example, the selection unit 406 compares the recognition rates in the supplementary information of the multiple information processing devices 134, 136, and 138 and selects the information processing device 136 with the highest recognition rate. The selection unit 406 also compares the number of inferences in the supplementary information of the multiple information processing devices 134, 136, and 138 and selects the information processing device 136 with the most inferences. Furthermore, the information based on the supplementary information (e.g., recognition rate or number of inferences) can be arbitrarily selected by the operator.

[0056] Furthermore, inference can also be performed using multiple information processing devices. The operator selects multiple information processing devices 136, 138 from among multiple information processing devices 134, 136, and 138 using the cursor 506. The information terminal 110 transmits medical image data to the multiple information processing devices 136, 138 selected by the selection unit 406. The multiple information processing devices 136, 138 selected by the selection unit 406 each perform inference on the transmitted medical image data using a trained model. The multiple information processing devices 136, 138 each transmit multiple inference results, which were performed on the medical image data, to the information terminal 110. The information terminal 110 integrates the multiple inference results transmitted from the multiple information processing devices 136, 138. The display unit 114 displays the integrated inference results along with the medical image data.

[0057] Figure 6 shows an example of the screen display configuration in the display unit 114 of the present invention. The difference from Figure 5 is that pneumonia is selected on the selection screen 502.

[0058] The information terminal 110 can extract an information processing device corresponding to pneumonia by referring to the supplementary information (diagnostic purpose) of multiple information processing devices (storage units). The selection screen 504 displays information processing devices that can infer pneumonia. Here, information processing devices 134 and 138 are displayed. The display unit 114 can display multiple information processing devices 134 and 138 corresponding to the diagnostic purpose of pneumonia. The operator selects at least one information processing device from the multiple information processing devices 134 and 138. The operator can select an information processing device using the cursor 506.

[0059] If the information processing device 134 is selected in the selection unit 406 of the information terminal 110, the information terminal 110 transmits medical image data to the information processing device 134 selected in the selection unit 406. The information processing device 134 performs inference on the transmitted medical image data and transmits the inference result to the information terminal 110. At this time, the information processing device 134 performs inference on the medical image data transmitted from the information terminal 110 using a trained model that has been trained to extract pneumonia from the medical image data. If there are pneumonia lesions in the medical image data transmitted from the information terminal 110, the information processing device 138 transmits the diagnosis name and region information related to pneumonia to the information terminal 110.

[0060] The display unit 114 displays a medical image 610 based on medical image data. The medical image 610 consists of multiple slice images of a CT image (three-dimensional volume data). The information processing device 134 performs inference on the medical image data transmitted from the information terminal 110 using a trained model corresponding to pneumonia. As shown in Figure 6, for example, if the inference unit of the information processing device 138 detects pneumonia in the CT image, the information processing device 138 transmits the diagnostic name and region information related to pneumonia to the information terminal 110. The display unit 114 displays the detection information 614 indicating that pneumonia has been detected. The display unit 114 also displays the region 612 in which pneumonia was detected.

[0061] Here, the selection screen 504 shown in Figure 6 displays an information processing device capable of inferring pneumonia. Here, information processing devices 134 and 138, and information regarding the properties of the information processing devices 134 and 138 are displayed. The selection screen 504 shows an information processing device capable of inferring pneumonia. Here, information processing devices 134 and 138, and information regarding the properties of the information processing devices 134 and 138 are displayed. Multiple information processing devices 134 and 138 capable of inferring pneumonia are displayed on the selection screen 504. The selection unit 406 of the information terminal 110 may select an information processing device based on the supplementary information (e.g., recognition rate) of the multiple information processing devices 134 and 138.

[0062] Figure 7 shows the properties of multiple information processing devices 130-138. The properties of the multiple information processing devices 130-138 include the body part, diagnostic purpose, medical imaging device, type of training data, recognition rate, number of inferences (actual results), and operator satisfaction. The number of inferences and recognition rate are statistical information related to the inference of each information processing device.

[0063] The diagnostic sites for the information processing device 130 are the liver and heart, and the diagnostic purpose is to obtain lesion information for diagnosing conditions such as hepatitis, angina pectoris, and myocardial infarction. The medical imaging devices that the information processing device 130 can process are CT and MRI medical image data. The type of training data is A. The recognition rate of lesions in hepatitis, angina pectoris, and myocardial infarction is 82%. The number of inferences is 5000. The satisfaction level is 2 stars.

[0064] Thus, the information terminal 110 stores supplementary information corresponding to the properties of multiple information processing devices 130-138 connected to the network 120. The multiple information processing devices 130-138 are information processing devices that can be used for inference with medical image data from the information terminal 110. The information terminal 110 has previously acquired supplementary information related to the diagnostic purpose that each information processing device can infer from the multiple information processing devices.

[0065] The display unit 114 displays supplementary information corresponding to the multiple information processing devices 130 to 138, allowing the operator to select at least one information processing device from among the multiple information processing devices 130 to 138.

[0066] Since the information terminal 110 stores supplementary information corresponding to the diagnostic site and diagnostic purpose in multiple information processing devices 130 to 138, the operator can select the appropriate information processing device according to the diagnostic site and diagnostic purpose, as shown in Figures 5 and 6.

[0067] Since the information terminal 110 stores supplementary information corresponding to the medical imaging device in multiple information processing devices 130 to 138, the operator can select an information processing device from the multiple information processing devices 130 to 138 according to the medical imaging device.

[0068] For example, since the information terminal 110 stores supplementary information corresponding to the recognition rate in multiple information processing devices 130 to 138, the operator can select the information processing device with the highest recognition rate from among the multiple information processing devices 130 to 138.

[0069] Furthermore, since the information terminal 110 stores supplementary information corresponding to the number of inferences (actual results) in multiple information processing devices 130 to 138, the operator can select the information processing device with the highest number of inferences (actual results) from among the multiple information processing devices 130 to 138.

[0070] Furthermore, since the information terminal 110 stores supplementary information corresponding to the operator's satisfaction level in multiple information processing devices 130 to 138, the operator can select an information processing device that provides the highest level of operator satisfaction from among the multiple information processing devices 130 to 138.

[0071] The operation of the inference phase of the present invention will be explained using Figure 8.

[0072] S800: The medical imaging device 100 takes images of the subject and acquires medical image data. The information terminal 110 (image acquisition unit 400) acquires medical image data from the medical imaging device 100.

[0073] S802: The information terminal 110 acquires the diagnostic site and diagnostic purpose. The operator selects the diagnostic site from the lungs, liver, heart, kidneys, etc., via the operation unit 112. The diagnostic purpose is the disease name corresponding to the diagnostic site, and is a diagnostic purpose (disease name) that can be inferred by multiple information processing devices 130, 132, 134, etc. The operator selects the diagnostic purpose from the lung cancer, pneumonia, etc., via the operation unit 112. The information terminal 110 acquires the diagnostic site and diagnostic purpose from the selection information input from the operation unit 112.

[0074] S804: Select whether the information terminal 110 is connected to multiple information processing devices via the network 120. If the information terminal 110 is not connected to multiple information processing devices (i.e., it is connected to one information processing device), proceed to S806. If the information terminal 110 is connected to multiple information processing devices, proceed to S808.

[0075] S806: The information terminal 110 transmits medical image data to one information processing device connected to the network 120. The information processing device performs inference on the medical image data using a trained model. The information processing device transmits the inference results obtained from the inference on the medical image data to the information terminal 110. The display unit 114 displays the medical image data along with the inference results obtained from the inference on the medical image data.

[0076] S808: The information terminal 110 selects at least one information processing device from a plurality of information processing devices connected to the network 120. The operator may manually select the information processing device via the operation unit 112, or the information terminal 110 may automatically select the information processing device based on information (parameters) based on the information processing device's associated information and statistical information based on inference.

[0077] S810: The information terminal 110 is connected to the network 120 and transmits medical image data to the selected information processing device. The selected information processing device performs inference on the medical image data using a trained model. The selected information processing device transmits the inference results obtained from the inference on the medical image data to the information terminal 110. The display unit 114 displays the medical image data along with the inference results obtained from the inference on the medical image data.

[0078] As described above, the information processing system of the present invention comprises an information terminal 110 that acquires medical image data from a medical imaging device 110, a plurality of information processing devices 130, 132, 134... each having an inference unit that performs inference on the medical image data using a trained model, and a selection unit 406 that selects at least one information processing device from the plurality of information processing devices 130, 132, 134... The information terminal 110 transmits the medical image data to the information processing device selected by the selection unit 406, the information processing device selected by the selection unit 206 performs inference on the transmitted medical image data and transmits the inference result to the information terminal 110.

[0079] Furthermore, the information terminal 110 of the present invention includes a selection unit that selects at least one information processing device from a plurality of information processing devices 130, 132, 134, etc., each having an inference unit that performs inference on medical image data, and a transmitting / receiving unit 404 that transmits medical image data to the information processing device selected by the selection unit 406 and receives the inference results obtained by the information processing device.

[0080] Therefore, it is possible to select at least one information processing device from multiple information processing devices and perform inference. [Examples]

[0081] Next, we will describe Example 2. The difference from Example 1 is that the selection unit 406 selects at least one information processing device from among multiple information processing devices based on past inference information (inference history) inferred by the multiple information processing devices.

[0082] Specifically, the information terminal 110 stores past inference information (inference history) inferred by multiple information processing devices. The selection unit 406 obtains actual inference information from the past inference information (inference history) inferred by multiple information processing devices and selects at least one information processing device.

[0083] For example, the selection unit 406 selects at least one information processing device from among multiple information processing devices based on the number of past inferences made by the multiple information processing devices.

[0084] Specifically, the selection unit 406 compares the number of past inferences made by multiple information processing devices and selects the information processing device with the highest number of inferences.

[0085] Alternatively, the selection unit 406 can compare the number of past inferences made by multiple information processing devices and select the information processing devices in descending order of the number of inferences. The selection unit 406 may also select three information processing devices in descending order of the number of inferences. The information terminal 110 transmits medical image data to the multiple information processing devices selected by the selection unit 406. The multiple information processing devices perform inferences on the medical image data using trained models. The multiple information processing devices each transmit the inference results they have obtained on the medical image data to the information terminal 110. The information terminal 110 integrates the inference results transmitted from the multiple information processing devices. The display unit 114 displays the integrated inference results along with the medical image data.

[0086] Furthermore, the selection unit 406 can also select at least one information processing device from among multiple information processing devices based on the type of medical image data acquired from the medical imaging device 100 and past inference information inferred by the multiple information processing devices. Specifically, if the type of medical image data is CT image data, and the target image inferred by the multiple information processing devices includes CT image data, then that information processing device is selected.

[0087] The selection unit 406 selects the at least one information processing device that matches the type of medical image data obtained from the medical imaging device 100, if the type of medical image data inferred by at least one of the multiple information processing devices matches.

[0088] As described above, according to this embodiment, the selection unit 406 can select at least one information processing device from among multiple information processing devices based on past inference information (inference history) inferred by the multiple information processing devices. Therefore, the operator can select the information processing device used in the past inference information (inference history) and perform inference.

[0089] A computer program that implements the functions of the above embodiment can be supplied to a computer via a network or storage medium (not shown), and the computer program can be executed. This computer program is used to cause the computer to execute the information processing method described above. In other words, the computer program is a program that enables the computer to implement the functions of an information processing device. The storage medium stores the computer program. [Explanation of symbols]

[0090] 100 Medical imaging devices 110 Information terminal 112 Operation section 114 Display section 120 Networks 130 Information Processing Devices 132 Information Processing Devices 134 Information Processing Device 400 Image acquisition unit 402 Processing Unit 404 Transmitter / Receiver 406 Selection Section

Claims

1. An information terminal that acquires medical image data from a medical imaging device, An information processing system comprising: a plurality of information processing devices, each having an inference unit that performs inference on the aforementioned medical image data, Each of the plurality of information processing devices includes a storage unit that stores ancillary information, including first information relating to a diagnostic purpose that can be inferred by each of the plurality of information processing devices. The information terminal includes a selection unit that selects at least one information processing device from the plurality of information processing devices. Based on the supplementary information obtained from each of the plurality of information processing devices and a first instruction from the operator regarding the diagnostic purpose, the information terminal presents to the operator as a candidate information processing device at least one information processing device from among the plurality of information processing devices to which the first information corresponding to the diagnostic purpose specified by the first instruction is attached. The selection unit selects an information processing device that performs inference on the medical image data based on a second instruction from the operator for selecting a candidate for the information processing device. An information processing system characterized by transmitting the medical image data to an information processing device selected by the selection unit, the information processing device selected by the selection unit performing inference on the transmitted medical image data, and transmitting the inference result to the information terminal.

2. The information processing system according to claim 1, characterized in that the information terminal has the selection unit.

3. The information processing system according to claim 1, characterized in that the information terminal is connected to the plurality of information processing devices via a network.

4. The information processing system according to claim 1, characterized in that the information terminal comprises a transmitting and receiving unit that transmits the medical image data to the information processing device selected by the selection unit and receives the inference result obtained by the information processing device from the medical image data.

5. The information processing system according to claim 1, characterized in that the aforementioned supplementary information includes second information relating to a diagnostic site that each of the plurality of information processing devices can infer.

6. The information processing system according to claim 5, characterized in that the information terminal stores the second information in advance.

7. The information processing system according to claim 1, characterized in that the information terminal stores the first information in advance.

8. The information processing system according to any one of claims 1 to 7, characterized in that the first instruction is an instruction for selecting a diagnostic objective from a list of candidate diagnostic objectives.

9. The information processing system according to claim 8, characterized in that the information terminal presents candidates for the diagnostic purpose to the operator based on the first information.

10. The information processing system according to claim 8 or 9, characterized in that the candidate for the diagnostic objective is determined according to the diagnostic site selected by the operator.

11. The information processing system according to any one of claims 1 to 10, characterized in that the diagnostic objective is lesion information in the diagnosis.

12. The information processing system according to claim 1, characterized in that the supplementary information includes information relating to the performance of each of the plurality of information processing devices, and the information terminal presents the operator with information relating to the performance of each of the candidate information processing devices, along with the candidate information processing devices.

13. An information processing method used in an information processing system having an information terminal for acquiring medical image data from a medical imaging device and a plurality of information processing devices each having an inference unit for performing inference on the medical image data, The steps include acquiring medical image data from a medical imaging device, A step of presenting to the operator, as a candidate for an information processing device, at least one information processing device among the plurality of information processing devices to which the first information corresponding to the diagnostic objective specified by the first instruction is attached, based on supplementary information obtained from each of the plurality of information processing devices, which includes first information relating to a diagnostic objective that each of the plurality of information processing devices can infer, and first instructions from the operator regarding the diagnostic objective. A step of selecting an information processing device that performs inference on the medical image data based on a second instruction from the operator for selecting a candidate for the information processing device, An information processing method comprising the steps of transmitting the medical image data to the selected information processing device, the selected information processing device performing inference on the transmitted medical image data, and transmitting the inference result to the information terminal.

14. A program for causing a computer to execute the information processing method of claim 13.

15. An information terminal used in an information processing system comprising an information terminal that acquires medical image data from a medical imaging device, and a plurality of information processing devices each having an inference unit that performs inference on the medical image data, A selection unit that selects at least one information processing device from the plurality of information processing devices, The system includes a transmitting and receiving unit that transmits the medical image data to the information processing device selected by the selection unit and receives the inference results obtained by the information processing device from the medical image data, Based on the supplementary information obtained from each of the plurality of information processing devices, which includes first information relating to a diagnostic purpose that each of the plurality of information processing devices can infer, and the first instruction from the operator regarding the diagnostic purpose, the information terminal presents to the operator as a candidate information processing device at least one information processing device from the plurality of information processing devices to which the first information corresponding to the diagnostic purpose specified by the first instruction is attached. The selection unit is characterized by selecting an information processing device that performs inference on the medical image data based on a second instruction from the operator for selecting a candidate for the information processing device.

Images