Information processing device and program

Abstract

To provide an information processor and a program that can support the determination of the correctness of the result of an inference by a learned model.SOLUTION: The information processor includes an acquisition unit and a determination unit. The acquisition unit acquires the result of an inference performed by an inference unit for performing an inference on the basis of at least one piece of input information and first basis information showing the result of the inference performed by the inference unit. The determination unit determines the correctness of the result of the inference on the basis of the first basis information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to an information processing device and a program. [Background technology]

[0002] Conventionally, in medical image diagnostic devices, a technology has been known that uses a trained model created by machine learning to support positioning, setting of imaging parameters, diagnosis, etc. For example, based on medical information such as medical images, the probability of a specific diagnosis (such as a disease name) is inferred and presented together with the basis for the inference. However, in this case, whether the inference result is valid or not is determined by a human. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2013-48798 [Non-patent literature]

[0004] [Non-Patent Document 1] Christopher M. Bishop, "Pattern recognition and machine learning", (USA), 1st edition, Springer, 2006, pp. 225-290 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to provide an information processing device and a program that can support the determination of the validity of inference results obtained by a trained model. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] According to an embodiment, an information processing device includes an acquisition unit and a determination unit. The acquisition unit acquires an inference result from an inference unit that performs inference based on one or more pieces of input information, and first basis information that indicates the basis for the inference result. The determination unit determines the validity of the inference result based on the first basis information. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical image processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a positioning image according to the embodiment. [Figure 3] FIG. 3 is a diagram schematically illustrating an example of an inference result using a trained model. [Figure 4] FIG. 4 is a diagram illustrating an example of an inference process using a trained model. [Figure 5] FIG. 5 is a diagram illustrating an example of the operation of the interpretation function according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the operation of the determination function according to the embodiment. [Figure 7] FIG. 7 shows an example of a calculation result of the ratio of background pixels to the region of interest according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a process for determining whether or not an attention area other than the most attention area matches knowledge information according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the operation of the correction function according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of an inference process using a trained model. [Figure 11] FIG. 11 is a diagram illustrating an example of the operation of the interpretation function according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of the operation of the determination function according to the embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of processing executed by the medical image processing apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of an information processing device and a program will be described in detail with reference to the drawings. Note that the information processing device and program according to the present application are not limited to the embodiments shown below. Furthermore, the embodiments can be combined with other embodiments or conventional techniques as long as there is no contradiction in the processing content.

[0009] (First embodiment) 1 is a block diagram showing an example of the configuration of a medical information processing device 3 according to an embodiment. The medical information processing device 3 is an example of an information processing device. For example, as shown in FIG. 1, the medical information processing device 3 according to the embodiment is included in a medical information processing system 100 that is communicably connected to a medical image diagnostic device 1 and a medical image storage device 2 via a network 200.

[0010] Here, the devices included in the medical information processing system 100 are capable of communicating with each other directly or indirectly, for example, via an in-hospital LAN (Local Area Network) installed in the hospital. Note that the medical information processing system 100 shown in Fig. 1 may be communicably connected with devices other than those shown.

[0011] For example, the medical information processing system 100 may include various systems such as a Hospital Information System (HIS), a Radiology Information System (RIS), a diagnostic report system, a Picture Archiving and Communication System (PACS), and a Laboratory Information System (LIS).

[0012] A medical image diagnostic device 1 captures an image of a subject to collect medical images. Then, the medical image diagnostic device 1 transmits the collected medical images to a medical image storage device 2 and a medical information processing device 3. For example, the medical image diagnostic device 1 is an MRI (Magnetic Resonance Imaging) device, an X-ray diagnostic device, an X-ray CT (Computed Tomography) device, an ultrasound diagnostic device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission Computed Tomography) device, or the like.

[0013] The medical image storage device 2 stores various medical images related to subjects. Specifically, the medical image storage device 2 acquires medical images from the medical image diagnostic device 1 via the network 200, and stores the medical images in a memory circuit within the device.

[0014] For example, the medical image storage device 2 is realized by computer equipment such as a server, a workstation, etc. Also, for example, the medical image storage device 2 is realized by a PACS (Picture Archiving and Communication System) or the like, and stores medical images in a format that complies with DICOM (Digital Imaging and Communications in Medicine).

[0015] The medical information processing device 3 acquires various types of information from the medical image diagnostic device 1 and the medical image storage device 2, and performs various types of information processing using the acquired information. For example, the medical information processing device 3 is realized by computer equipment such as a server, a workstation, a personal computer, or a tablet terminal. Note that the medical information processing device 3 may also be a console device or the like that controls the medical image diagnostic device 1.

[0016] As shown in FIG. 1, the medical information processing device 3 includes a communication interface 31, a memory circuitry 32, an input interface 33, a display , and a processing circuitry .

[0017] The communication interface 31 is connected to the processing circuitry 35 and controls communication between each device in the medical information processing system 100. Specifically, the communication interface 31 receives various types of information from each device and outputs the received information to the processing circuitry 35. For example, the communication interface 31 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0018] The memory circuitry 32 is connected to the processing circuitry 35 and stores various types of data. For example, the memory circuitry 32 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0019] Specifically, the memory circuitry 32 stores various programs that the processing circuitry 35 reads and executes to realize various functions.

[0020] The memory circuitry 32 also stores various information received from the medical image diagnostic device 1 and the medical image storage device 2, information input via the input interface 33, and processing results of the medical information processing device 3. For example, as shown in FIG. 1 , the memory circuitry 32 stores a trained model 321, a knowledge DB 322, an interpretation table 323, a judgment table 324, and a correction table 325.

[0021] The trained model 321 is a trained model generated by machine learning using information acquired from the medical image diagnostic device 1, the medical image storage device 2, etc. as training data. The medical information processing device 3 executes inference processing using the trained model 321.

[0022] The trained model 321 may be generated by the processing circuitry 35 or by a device other than the medical information processing device 3. For example, the trained model 321 may be generated by an external device located outside the medical information processing system 100.

[0023] The knowledge DB 322 is a database that stores information including findings based on research results of clinical research, information on various guidelines, and simulations based on them. The knowledge DB 322 includes knowledge information that serves as a criterion for determining the validity of the inference results obtained by the trained model 321.

[0024] The interpretation table 323 holds information used to interpret basis information that indicates the basis for the inference results of the trained model 321. For example, the interpretation table 323 is a data table that associates each type of basis information with the processing content to be performed to interpret the basis information.

[0025] Note that the information used to interpret the basis information is not limited to a table, and may be, for example, a trained model that has been trained by machine learning (including deep learning) to output interpretation information obtained by interpreting the basis information when the basis information is input.

[0026] The judgment table 324 holds information used to judge the validity of the inference result by the trained model 321. For example, the judgment table 324 is a data table that associates the referenced knowledge information with each type of basis information or interpretation information. Here, the associated knowledge information serves as an index for judging the validity of the inference result.

[0027] The knowledge DB 322 may store knowledge information in association with the basis information or interpretation information, in which case the judgment table 324 may be unnecessary.

[0028] The correction table 325 holds information used to correct the inference result obtained by the trained model 321. For example, the correction table 325 is a data table that associates a set of grounds information, interpretation information, and knowledge information with a correction process for correcting the inference result related to the set.

[0029] The information used to correct an invalid inference result is not limited to a table, and may be, for example, a trained model that has been trained by machine learning to output a corrected inference result when an invalid inference result is input.

[0030] In addition, some or all of the trained model 321, knowledge DB 322, interpretation table 323, judgment table 324, and correction table 325 may be stored in another information processing device or the like that can be accessed by the medical information processing device 3.

[0031] The input interface 33 is connected to the processing circuitry 35 and receives various instructions and information from an operator. Specifically, the input interface 33 converts the input received from the operator into an electrical signal and outputs it to the processing circuitry 35.

[0032] For example, the input interface 33 may be realized by a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, and a voice input circuit.

[0033] In this specification, the input interface 33 is not limited to an interface having physical operation parts such as a mouse, a keyboard, etc. For example, an example of the input interface 33 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

[0034] The display 34 is connected to the processing circuit 35 and displays various information and images. Specifically, the display 34 converts the information and image data sent from the processing circuit 35 into an electrical signal for display and outputs it. For example, the display 34 is realized by a liquid crystal monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, etc.

[0035] The processing circuitry 35 controls the operation of the medical information processing device 3 in response to an input operation received from an operator via the input interface 33. For example, the processing circuitry 35 is realized by a processor. As shown in FIG. 1 , the processing circuitry 35 executes a control function 351, an inference function 352, an acquisition function 353, an interpretation function 354, a determination function 355, and a correction function 356.

[0036] Here, the control function 351 is an example of a presentation unit. The inference function 352 is an example of an inference unit. The acquisition function 353 is an example of an acquisition unit. The generation function 357 is an example of a generation unit. The interpretation function 354 and the judgment function 355 are examples of a judgment unit. The correction function 356 is an example of a correction unit.

[0037] The control function 351 controls the execution of processes in response to various requests input via the input interface 33. For example, the control function 351 controls the sending and receiving of medical images via the communication interface 31, the storage of various information in the memory circuitry 32, the display of information (for example, medical images, processing results by each function, etc.) on the display 34, etc.

[0038] For example, the control function 351 acquires data such as positioning image data and medical image data for determining the imaging position of the subject from the medical image diagnostic apparatus 1, and stores the data in the storage circuitry 32. Also, for example, the control function 351 controls the medical image diagnostic apparatus 1 to display a GUI for executing processing and the processing results of each function on the display 34.

[0039] The inference function 352 performs inference using the trained model 321. Specifically, the inference function 352 inputs input data to the trained model 321, causing the trained model 321 to execute inference processing.

[0040] For example, the inference function 352 inputs a positioning image (an example of input data) to the trained model 321. Next, the trained model 321 outputs the imaging position of the subject in the MRI apparatus in response to the input of the positioning image.

[0041] In this case, the imaging position is the inference result of the trained model 321. In this way, the inference function 352 can assist in the process of determining the imaging position of the subject in the MRI apparatus by having the trained model 321 output the imaging position based on the positioning image.

[0042] Furthermore, for example, the inference function 352 inputs user input information (an example of input data) such as information about the desired image quality and information about the imaging region for medical images captured by the MRI device to the trained model 321. Next, the trained model 321 outputs imaging parameters for determining imaging conditions for the subject in the MRI device in response to the input of the user input information.

[0043] In this case, the imaging parameters are the inference results of inference by the trained model 321. In this way, the inference function 352 can support the imaging parameter determination process by having the trained model 321 output the imaging parameters based on user input information. Note that information regarding the imaging region may be obtained from a RIS or the like.

[0044] Also, for example, the inference function 352 inputs medical image data (an example of input data) captured by an MRI device into the trained model 321. Next, the trained model 321 outputs a diagnosis indicating symptoms, etc. occurring in the subject in response to the input of the medical image data.

[0045] In this case, the diagnosis is the inference result of the trained model 321. In this way, the inference function 352 can provide diagnostic support by having the trained model 321 output the diagnosis based on the medical image data. Note that the above-mentioned processing related to the MRI device, such as positioning support, imaging parameter determination support, and diagnostic support, can also be performed in medical image diagnostic devices 1 other than MRI devices, such as X-ray CT devices.

[0046] The acquisition function 353 acquires basis information indicating the basis of the inference result. Specifically, the acquisition function 353 acquires the inference result output by the trained model 321 and the basis information indicating the basis of the inference result.

[0047] Here, the basis information is, for example, information representing candidates for inference results by the trained model 321 and an index indicating the likelihood of the candidates. The basis information may also be information representing feature amounts that contributed to deriving the candidates for inference results and an index indicating the degree to which the feature amounts contributed to deriving the candidates for inference results (hereinafter also referred to as the degree of contribution). The basis information may also be information combining the former and the latter. The candidates for inference results may include the final inference result.

[0048] The method for acquiring the basis information is not particularly limited. For example, when the trained model 321 outputs the basis information together with the inference result, the acquisition function 353 acquires the basis information acquired from the trained model 321. The acquisition function 353 may also acquire the basis information by performing an analysis on the output result (inference result) of the trained model 321, such as identifying the contributing feature amount. The acquisition function 353 may also acquire multiple pieces of basis information.

[0049] For example, when an inference is performed using the trained model 321 to provide positioning assistance in an MRI device, the acquisition function 353 acquires from the trained model 321 the imaging position derived as the inference result and the basis information indicating the basis for deriving this imaging position.

[0050] In this case, the basis information is, for example, a candidate imaging position identified based on a region of interest on a positioning image of the MRI apparatus, and the probability (an example of accuracy) that the candidate is the desired imaging position. Alternatively, the basis information in this case may be a candidate imaging position identified based on a feature of interest (e.g., shape) on a positioning image of the MRI apparatus, and the probability that the candidate is the desired imaging position, etc.

[0051] Furthermore, for example, when the trained model 321 performs inference to assist in determining imaging parameters in an MRI apparatus, the imaging parameters derived as the inference results and grounds information indicating the grounds on which the imaging parameters were derived are acquired from the trained model 321. In this case, the grounds information is, for example, the conditions that contributed to the derivation of the imaging parameters and the probability that the imaging parameters are imaging parameters for performing imaging with the desired image quality.

[0052] Furthermore, for example, when the trained model 321 performs inference for diagnostic support based on medical image data captured by an MRI device, the diagnosis derived as the inference result and evidence information indicating the basis for deriving this diagnosis are obtained from the trained model 321. In this case, the evidence information is, for example, the imaging findings that contributed to the derivation of the diagnosis and the probability that the diagnosis of the subject to be diagnosed is the diagnosis.

[0053] The interpretation function 354 performs an interpretation process to interpret the basis information. Here, the interpretation process is a process of converting the acquired basis information into a form that can be directly compared with the knowledge information, based on the basis information, when the acquired basis information cannot be directly compared with the knowledge information. Note that if the basis information is information that can be directly compared with the knowledge information, the interpretation process is not necessary.

[0054] For example, the interpretation function 354 refers to the interpretation table 323 and executes a process associated with the basis information acquired by the acquisition function 353. In other words, the interpretation function 354 executes a process for interpreting the basis information. The interpretation function 354 acquires the processing result of the process as interpretation information.

[0055] The determination function 355 determines whether the inference result by the inference function 352 is valid or not, based on the determination table 324. For example, the determination function 355 refers to the determination table 324 and identifies, from the knowledge DB 322, knowledge information associated with the basis information acquired by the acquisition function 353 or the interpretation information acquired by the interpretation function 354. Next, the determination function 355 determines the validity of the inference result based on the basis information or the interpretation information and the identified knowledge information.

[0056] If the basis information or interpretation information matches the identified knowledge information, the determination function 355 determines that the inference result is valid. On the other hand, if the basis information or interpretation information does not match the identified knowledge information, the determination function 355 determines that the inference result is invalid. Note that the determination function 355 may further determine the validity of the inference result corrected by the correction function 356 described below.

[0057] The correction function 356 corrects the inference result that is determined to be invalid by the judgment function 355, based on the correction table 325. For example, the correction function 356 corrects the inference result by referring to the correction table 325 and executing a process corresponding to the set of the basis information acquired by the acquisition function 353, the interpretation information acquired by the interpretation function 354, and the knowledge information identified by the judgment function 355. Next, the correction function 356 outputs the processing result of the process as the corrected inference result.

[0058] The generation function 357 generates visualized information that visualizes the processes executed by the inference function 352, the acquisition function 353, the interpretation function 354, the determination function 355, and the correction function 356. For example, the generation function 357 generates image data, graphs, text data, etc., which serve as visualized information of the basis information or the interpretation information, based on the basis information or the interpretation information. For example, the generated visualized information is presented to the user by being output to the display 34 by the control function 351.

[0059] This makes it easier for users to understand the basis on which the inference result was derived. For example, when inference is performed to assist positioning in an MRI device, visualized information that visualizes the basis information becomes image data of a region of interest map that represents the region of interest on the positioning image.

[0060] Furthermore, for example, the visualization information in this case may be a distribution of noted features, which represents the distribution of noted features in a graph such as a bar graph. Note that the visualization information may also be text data, which describes in sentences the process of deriving an inference result from evidence information, such as "The most noteworthy area with the highest level of attention on the positioning image was identified, and it was inferred that the center of gravity of that area was the shooting position."

[0061] For example, when inference is performed to assist in determining imaging parameters in an MRI device, the visualized information that visualizes the basis information will be text data or the like that explains the relationship between the conditions that contributed to the derivation of the imaging parameters and the derived imaging parameters.

[0062] For example, when inference is made for diagnostic support using an MRI device, the visualized information that visualizes the evidence information is text data or the like that explains that the imaging findings that contributed to the derivation of the diagnosis name are findings that are specifically observed in the case of that diagnosis name.

[0063] Below, with reference to Figures 2 to 9, we will explain each of the functions of the inference function 352, acquisition function 353, interpretation function 354, judgment function 355, correction function 356, and generation function 357, using as an example the case where the shooting position determination process is performed based on a positioning image, which is an example of medical image data.

[0064] FIG. 2 is a diagram showing an example of a positioning image. FIG. 2 shows a positioning image obtained by imaging the head of a subject using an MRI device. The trained model 321 used in this example derives the imaging position (cross section of the head to be imaged) from this positioning image as an inference result. The positioning image data includes information indicating the desired imaging position. For example, the positioning image data in FIG. 2 includes information indicating that the desired imaging position is a cross section of the head including the root of the nose.

[0065] For example, the desired shooting position may be determined by the control function 351 analyzing the positioning image data acquired by the medical image diagnostic apparatus 1. Also, for example, the control function 351 may receive input from a user regarding the desired shooting position and determine the desired shooting position in accordance with the user's input. In this case, the control function 351 performs control to add information indicating the determined desired shooting position to the positioning image data acquired from the medical image diagnostic apparatus 1.

[0066] Here, we will explain the trained model 321 used in this example. The trained model 321 is functionally configured to output the imaging position on the positioning image by inputting the positioning image and medical information related to the subject. The imaging position refers to a specific position (hereinafter also referred to as a feature point) on the body of the subject for specifying the cross section to be imaged.

[0067] Specifically, when a positioning image or the like is input, the trained model 321 outputs, as an inference result, the photographing position on the positioning image and the accuracy of the photographing position (an index showing the likelihood of inference) based on the feature amounts included in the positioning image, etc. Furthermore, for example, the trained model 321 outputs, as an inference result, the photographing position on the positioning image with the highest accuracy.

[0068] There is no particular restriction on the method for generating the trained model 321. For example, the trained model 321 can be generated by machine learning a positioning image, a shooting position on the positioning image, and the like as learning data in a learning device that generates a trained model.

[0069] The machine learning engine used for machine learning is not particularly limited, and known techniques can be used. For example, the neural network described in the well-known non-patent document "Pattern Recognition and Machine Learning" by Christopher M. Bishop (USA), 1st Edition, Springer, 2006, pp. 225-290 can be used as the machine learning engine.

[0070] In addition to the neural network described above, the machine learning engine may also use various algorithms such as deep learning, logistic regression analysis, nonlinear discriminant analysis, support vector machine (SVM), random forest, and naive Bayes.

[0071] The inference function 352 inputs the positioning image shown in FIG. 2 into the trained model 321 described above, causing the trained model 321 to execute inference processing.

[0072] Fig. 3 is a diagram schematically illustrating an example of an inference result by the trained model 321. Fig. 3 shows the inference result obtained by inputting the positioning image shown in Fig. 2 into the trained model 321. Fig. 3 shows that the position corresponding to the root of the nose on the positioning image in Fig. 2 has been output as the inference result. Here, the inference result means the photographing position with the highest accuracy.

[0073] FIG. 4 is a diagram schematically illustrating an example of inference processing using the trained model 321. FIG. 4 is a map of attention areas of the positioning image of FIG. 2. Here, the attention area map is an image that shows attention areas, which serve as ground information, on the positioning image. In FIG. 4, (1), (2), and (3) represent attention areas. The trained model 321 outputs the shooting position based on the attention areas. For example, the trained model 321 outputs the position of the center of gravity of the attention area as the shooting position.

[0074] FIG. 4 also shows that the attention level increases in the order of (1), (2), and (3). The attention level is a numerical value related to the accuracy of the shooting position. For example, the trained model 321 outputs the accuracy of the shooting position based on the proportion of the attention level of each attention area to the total value of the attention levels of all attention areas. In FIG. 4, the trained model 321 outputs the tip of the nose, which is the center of gravity of attention area (1) with the highest accuracy, as the inference result.

[0075] The interpretation function 354 executes interpretation processing based on, for example, information about the region of interest on the positioning image. Specifically, the interpretation function 354 refers to the interpretation table 323 and executes processing corresponding to the information about the region of interest on the positioning image.

[0076] As an example, the interpretation table 323 stores a process of "calculating the ratio of background pixels in the most attention area" in association with "information about the attention area on the positioning image." In this case, the interpretation function 354 calculates the ratio of background pixels in the most attention area. The result of this calculation becomes interpretation information.

[0077] FIG. 5 is a diagram illustrating an example of the operation of the interpretation function 354. "(1)" in FIG. 5 is a number that identifies the attention area on the attention area map. Also, "85%" in FIG. 5 represents the probability that attention area (1) includes the shooting position. "Proportion of background pixels to the attention area = 70%" in FIG. 5 represents interpretation information. Based on this interpretation information, the judgment function 355 judges the validity of the inference result that the shooting position is the tip of the nose.

[0078] Specifically, the determination function 355 refers to the determination table 324 and identifies a reference location in the knowledge DB 322 that corresponds to the "proportion of background pixels to the region of interest." In this example, the "proportion of background pixels to the region of interest" is stored in association with the "address in the knowledge DB 322 where the knowledge information that the proportion of background pixels in the region of interest that includes the position corresponding to the root of the nose to the region of interest = 30% is stored."

[0079] This allows the determination function 355 to refer to the knowledge DB 322 and identify that the knowledge information used to determine validity is "the proportion of background pixels in the region of interest that includes the position corresponding to the nose root to the region of interest = 30%." The determination function 355 determines whether the inference result is valid or not based on the knowledge information "the proportion of background pixels in the region of interest that includes the position corresponding to the nose root to the region of interest = 30%."

[0080] 6 is a diagram illustrating an example of the operation of the determination function 355. As shown in FIG. 6, the determination function 355 determines that the inference result is invalid because the knowledge information "the proportion of background pixels in the region of interest including the position corresponding to the nose root to the region of interest = 30%" does not match the interpretation information acquired as "the proportion of background pixels in the region of interest = 70%." In this case, the correction function 356 performs a correction process for the inference result.

[0081] Specifically, the correction function 356 refers to the correction table 325 and performs a correction process corresponding to the set of grounds information "information regarding the region of interest on the positioning image," interpretation information "proportion of background pixels in the region of interest = 70%," and knowledge information "proportion of background pixels in the region of interest including the position corresponding to the root of the nose in the region of interest = 30%."

[0082] In this example, a set of basis information "information regarding the area of ​​interest on the positioning image," interpretation information "proportion of background pixels to the area of ​​interest = 70%," and knowledge information "proportion of background pixels to the area of ​​interest including the position corresponding to the root of the nose = 30%" is stored in association with a correction process that "calculate the proportion of background pixels for each area other than the area of ​​greatest interest, and use the position of the center of gravity of the area that matches the knowledge information and has the highest probability of containing the shooting position as the shooting position."

[0083] The correction function 356 calculates the ratio of background pixels to attention areas (2) and (3), which are attention areas other than attention area (1), which is the most attention area in Figure 2. Figure 7 shows an example of the calculation result of the ratio of background pixels to the attention area.

[0084] Figure 7 shows that there is an 80% probability that attention area (2) on the attention area map in Figure 2 includes the shooting location, and that the proportion of background pixels in the attention area is 30%. Figure 7 also shows that there is a 20% probability that attention area (3) on the attention area map in Figure 2 includes the shooting location, and that the proportion of background pixels in the attention area is 30%.

[0085] Next, the correction function 356 cooperates with the judgment function 355 to determine whether the "proportion of background pixels in the region of interest = 30%" of the region of interest (2) and the "proportion of background pixels in the region of interest = 30%" of the region of interest (3) match the knowledge information. Specifically, the correction function 356 determines whether the "proportion of background pixels in the region of interest including the position corresponding to the root of the nose in the region of interest = 30%" matches the knowledge information, similar to the process of determining the validity of the inference result.

[0086] 8 is a diagram illustrating an example of a process for determining whether or not a region of interest other than the most important region matches knowledge information. As shown in FIG. 8, the calculation results of the ratio of background pixels to the region of interest for both region of interest (2) and region of interest (3) match the knowledge information "the ratio of background pixels in the region of interest including the position corresponding to the root of the nose to the region of interest = 30%."

[0087] Fig. 9 is a diagram illustrating an example of the operation of the correction function 356. As shown in Fig. 9, of the attention area (2) and the attention area (3) that match the knowledge information "the proportion of background pixels in the attention area including the position corresponding to the nose bridge to the attention area = 30%", the correction function 356 outputs information on the attention area (2) that is most likely to include the shooting position as the inference result of the inference regarding the shooting position after correction.

[0088] Based on the inference result output by the correction function 356, the generation function 357 generates, as visualization information, an image indicating the position of the center of gravity of the attention area (2) on the positioning image of Fig. 2. Note that the generation function 357 may also generate, as visualization information, the diagrams shown in Fig. 3 to Fig. 9. The generated visualization information is output as an image to the display 34 by, for example, the control function 351.

[0089] 9, invalid inference results are automatically corrected, but the correction function 356 may correct the inference results according to instructions from a user such as a doctor. In this case, the user may issue an instruction to correct the inference results based on the visualization information generated by the generation function 357. Being able to manually correct the inference results in this way is useful, for example, in situations where the user can easily think of a correction method from the generated visualization information.

[0090] The determination function 355 may be configured to re-determine the validity of the inference result corrected by the correction function 356. In this case, it is preferable to determine the validity based on knowledge information other than the knowledge information used to determine the validity of the inference result before correction. For example, when performing the inference to support the determination of the above-mentioned shooting position, it is preferable to determine the validity based on knowledge information related to the shooting position other than the knowledge information "the proportion of background pixels in the region of interest including the position corresponding to the nose bridge to the region of interest = 30%."

[0091] In this case, the control function 351 may output the final inference result to the display 34 or the like only when the corrected inference result is valid. This can improve the accuracy of the inference.

[0092] In the above description, the basis information is "information about a region of interest on the positioning image," but the form of the basis information is not limited to this. For example, the basis information may be "information about a feature of interest on the positioning image." Hereinafter, with reference to FIGS. 10 to 12, a process for determining a shooting position based on a positioning image when the basis information is "information about a feature of interest on the positioning image" will be described.

[0093] In this example, the trained model 321 also outputs the tip of the subject's nose as the imaging position. In this example, the trained model 321 outputs the imaging position on the positioning image based on "information on the feature of interest on the positioning image" rather than "information on the region of interest on the positioning image."

[0094] Fig. 10 is a diagram schematically illustrating an example of inference processing using the trained model 321. Fig. 10 is a distribution of noted features, which is a bar graph showing the distribution of noted features for the positioning image of Fig. 2. Fig. 10 also shows that for the positioning image of Fig. 2, the most noted feature is "convex," the second most noted feature is "concave," and the third most noted feature is "flat."

[0095] As shown in Fig. 10, the trained model 321 outputs the inference result that the imaging position is the tip of the subject's nose, based on the fact that the most notable feature in the positioning image of Fig. 2 is "convex." In order to determine the validity of this inference result, the interpretation function 354 first performs an interpretation process on the "information related to the notable feature on the positioning image."

[0096] The interpretation function 354 refers to the interpretation table 323 and executes processing corresponding to information relating to the noted feature on the positioning image. In this example, it is assumed that "information relating to the noted feature on the positioning image" and "extract the most noted feature" are stored in association with each other. The interpretation function 354 extracts the most noted feature on the positioning image shown in FIG. 2. The result of this extraction becomes interpretation information.

[0097] Fig. 11 is a diagram illustrating an example of the operation of the interpretation function 354. Fig. 11 shows that the most notable feature in the positioning image shown in Fig. 2 is "convex." Based on this interpretation information, the determination function 355 determines the validity of the inference result that the photographing position is the tip of the nose.

[0098] The judgment function 355 refers to the judgment table 324 and identifies the reference location in the knowledge DB 322 that corresponds to "extracted most noteworthy feature=convex". In this example, it is assumed that "extracted most noteworthy feature=convex" and "the address in the knowledge DB 322 where the knowledge information that the area around the position corresponding to the nose root is concave" are stored in association with each other.

[0099] This allows the determination function 355 to refer to the knowledge DB 322 and identify that the knowledge information used to determine validity is "the area around the position corresponding to the root of the nose is concave." The determination function 355 determines whether the inference result is valid based on the knowledge information "the area around the position corresponding to the root of the nose is concave."

[0100] 12 is a diagram illustrating an example of the operation of the determination function 355. As shown in FIG. 12, the determination function 355 determines that the inference result is invalid because the knowledge information "the area around the position corresponding to the nose root is concave" does not match the interpretation information obtained as "the most notable feature is extracted = convex." Since the inference result is invalid, the correction function 356 performs a correction process for the inference result. Note that if the inference result is valid, the control function 351 outputs information representing the inference result to the display 34.

[0101] The correction function 356 refers to the correction table 325 and executes a correction process corresponding to the set of grounds information "information regarding the feature of interest on the positioning image", interpretation information "extracting the most notable feature = convex", and knowledge information "the area near the position corresponding to the root of the nose is concave".

[0102] In this example, a set of basis information "information on the feature of interest," interpretation information "extracting the feature of interest most = convex," and knowledge information "the area near the position corresponding to the root of the nose is concave" is stored in association with a correction process of "applying a filter that removes convexity to the positioning image and performing the same process again."

[0103] The correction function 356 applies a filter that removes convex shapes to the positioning image, and inputs the positioning image after the filter has been applied to the trained model 321. Then, the correction function 356 outputs the inference result derived by the trained model 321 as the corrected inference result.

[0104] The generating function 357 generates, as visualized information, an image showing the position of the subject's nose bridge on the positioning image of Fig. 2 based on the inference result output by the correcting function 356. The generating function 357 may also generate, as visualized information, the diagrams shown in Fig. 10 to Fig. 12. The generated visualized information is output as an image to the display 34 by, for example, the control function 351.

[0105] The determination function 355 may determine the validity of the corrected inference result output by the correction function 356 based on the knowledge information "the area around the position corresponding to the nose root is concave." In this case, the control function 351 may output the final inference result to the display 34 or the like only when the corrected inference result is valid. This can improve the accuracy of the inference.

[0106] 2 to 9 and the processing based on the basis information described using Figures 10 to 12 may both be executed. In this case, only when the inference results from the two processes match, the control function 351 may output the inference result as the final inference result.

[0107] In addition to the processing based on evidence information described using Figures 2 to 9 and the processing based on evidence information described using Figures 10 to 12, processing based on different evidence information may be performed, and only if two or more of the inference results from the three processes match, the control function 351 may output the inference result as the final inference result.

[0108] 2 to 9 may be performed as a first processing, and then, as a second processing, the processing based on the basis information described with reference to Figures 10 to 12 may be performed. In this case, the order of performing the processing may be determined based on the contribution rate of each piece of basis information to the inference result, etc.

[0109] In this case, only when the inference result from the first processing and the modified result from the second processing match, the control function 351 may output the inference result as the final inference result. This makes it possible to judge the validity of the inference result from multiple different perspectives.

[0110] The generation function 357 may visualize the inference result and generate visualized information that visualizes information related to the processing executed by each of the inference function 352, the acquisition function 353, the interpretation function 354, the judgment function 355, and the correction function 356. The generated visualized information is output by the control function 351 to the display 34 or the like.

[0111] This allows the user to easily understand the basis on which the validity of the inference result was determined, and how the inference result was corrected if it was found to be invalid.

[0112] Next, a description will be given of the processing executed by the medical information processing device 3. Fig. 13 is a flowchart showing an example of the processing executed by the medical information processing device 3.

[0113] First, the inference function 352 inputs input data to the trained model 321 (step S1). For example, when performing a process for determining an imaging position, the inference function 352 inputs image data of a positioning image captured by an MRI device to the trained model 321. Next, the acquisition function 353 acquires the inference result output from the trained model 321 (step S2).

[0114] For example, when performing a process to determine a shooting position, the acquisition function 353 acquires, as an inference result of inference regarding the shooting position, the area that is most likely to include the shooting position from among the information on multiple areas output from the trained model 321. Note that the possibility that each area includes the shooting position is determined based on the probability that the area includes the shooting position, which is output from the trained model 321 together with the information on each area.

[0115] Next, the acquisition function 353 acquires grounds information that serves as the grounds for the inference result (step S3). For example, when performing the process of determining the shooting position, the acquisition function 353 acquires "information related to the region of interest on the positioning image" of the positioning image as grounds information.

[0116] Next, the interpretation function 354 performs interpretation processing on the basis information and acquires interpretation information (step S4). Note that if interpretation processing is not required, step S4 is omitted.

[0117] For example, when the acquisition function 353 acquires information about the region of interest on the positioning image as the basis information, the interpretation function 354 refers to the interpretation table 323 and performs interpretation processing corresponding to the "information about the region of interest on the positioning image." If the interpretation table 323 stores an association between "information about the region of interest on the positioning image" and "calculate the proportion of background pixels in the region of greatest interest," the interpretation function 354 calculates the proportion of background pixels in the region of greatest interest. The result of this calculation becomes the interpretation information.

[0118] Next, the determination function 355 determines whether or not the inference result obtained by the inference function 352 is valid based on the basis information obtained by the acquisition function 353 or the interpretation information obtained by the interpretation function 354 (step S5). For example, if the interpretation function 354 obtains the ratio of background pixels in the region of greatest interest as interpretation information, the interpretation function 354 refers to the determination table 324 and identifies knowledge information for determining the validity of the inference result, which corresponds to the "ratio of background pixels in the region of greatest interest."

[0119] If the "proportion of background pixels in the region of greatest interest" and the "address in the knowledge DB 322 where knowledge information on the proportion of background pixels in the region of interest including the shooting position is stored" are stored in association with each other in the judgment table 324, the judgment function 355 identifies the "proportion of background pixels in the region of interest including the shooting position" as knowledge information for judging the validity of the inference result. The judgment function 355 judges the validity of the inference result based on whether the knowledge information matches the interpretation information.

[0120] If it is determined that the inference result is valid (step S5: Yes), the generation function 357 generates visualization information representing the inference result output by the trained model 321. Then, the control function 351 outputs the generated visualization information to the display 34 and ends this process (step S6). On the other hand, if it is determined that the inference result is invalid (step S5: No), the correction function 356 performs a correction process for the invalid inference result (step S7).

[0121] For example, if the judgment function 355 determines that the inference result is invalid based on the basis information "information regarding the area of ​​interest on the positioning image," the interpretation information "the proportion of background pixels in the area of ​​greatest interest," and the knowledge information "the proportion of background pixels in the area of ​​interest including the shooting position," the judgment function 355 refers to the correction table 325 and performs correction processing corresponding to the basis information "information regarding the area of ​​interest on the positioning image," the interpretation information "the proportion of background pixels in the area of ​​greatest interest," and the knowledge information "the proportion of background pixels in the area of ​​interest including the shooting position."

[0122] In the correction table 325, when the basis information "information regarding the region of interest on the positioning image," the interpretation information "proportion of background pixels in the region of greatest interest," and the knowledge information "proportion of background pixels in the region of interest including the shooting position" are stored in correspondence with "the proportion of background pixels in each region other than the region of greatest interest is calculated, and the position of the center of gravity of the region that is most likely to include a shooting position that matches the knowledge information is taken as the shooting position," correction function 356 calculates the proportion of background pixels in each region other than the region of greatest interest.

[0123] The correction function 356 then outputs, as the corrected inference result, the position of the center of gravity of the region of interest that is most likely to include the shooting position among the regions of interest that match the knowledge information "the proportion of background pixels in the region of interest that includes the shooting position." Next, the generation function 357 generates visualization information that visualizes the inference result based on the output inference result.

[0124] Next, the control function 351 outputs visualized information that visualizes the generated inference results to the display 34, and ends this process (step S8).

[0125] As described above, the medical information processing device 3 of this embodiment performs inference using a trained model 321 generated using machine learning to support medical procedures such as determining the imaging position of a subject in an MRI device based on input data such as positioning image data, obtains evidence information that serves as the basis for the inference, and determines the validity of the inference based on the evidence information and knowledge information for determining the inference result stored in the knowledge DB 322.

[0126] This automatically determines the validity of the inference, eliminating the need for users to determine the validity of the inference results from a trained model themselves.

[0127] Furthermore, the medical image processing device 3 according to this embodiment determines the validity of the inference result based on multiple pieces of evidence information. This allows validity to be determined from multiple perspectives, which is expected to improve the accuracy of inference using a trained model.

[0128] Furthermore, when the medical information processing device 3 according to this embodiment determines that the inference result obtained by machine learning is invalid, it corrects the invalid inference result based on the basis information, interpretation information obtained by interpreting the basis information, and knowledge information. This allows the user to obtain the corrected inference result without checking the information that is the basis for the inference, even if the inference result is invalid.

[0129] Furthermore, the medical information processing device 3 according to this embodiment determines the validity of the corrected inference result based on evidence information different from that used to determine the validity before the correction and on knowledge information stored in the knowledge DB 322. By using evidence information different from that used to determine the validity before the correction, the validity of the corrected inference result can be determined from a different perspective than before the correction, which is expected to improve the accuracy of inference using the trained model.

[0130] The term "processor" used in the above explanation refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)).

[0131] The processor realizes its functions by reading and executing a program stored in the memory circuitry 32. Note that instead of storing a program in the memory circuitry 32, the program may be directly embedded in the circuitry of the processor. In this case, the processor realizes its functions by reading and executing the program embedded in the circuitry. Furthermore, the processor of this embodiment is not limited to being configured as a single circuit, and may be configured as a single processor by combining multiple independent circuits to realize its functions.

[0132] Here, the program executed by the processor (medical information processing program) is provided by being pre-installed in a ROM (Read Only Memory), a storage circuit, etc. Note that this program may also be provided by being recorded in a computer-readable storage medium such as a CD (Compact Disc)-ROM, a FD (Flexible Disk), a CD-R (Recordable), or a DVD (Digital Versatile Disc) in a format that can be installed or executed by these devices.

[0133] This program may also be provided or distributed by being stored on a computer connected to a network such as the Internet and downloaded via the network. For example, this program may be composed of modules including the above-mentioned functional units. In actual hardware, a CPU reads and executes the program from a storage medium such as a ROM, whereby each module is loaded into a main memory and generated on the main memory.

[0134] According to at least one of the embodiments described above, it is possible to assist in determining the validity of inference results obtained using a trained model.

[0135] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0136] 100 Medical Information Processing System 200 Network 1 Medical imaging diagnostic equipment 2 Medical image storage device 3 Medical information processing equipment 31 Communication Interface 32 Memory circuit 33 Input Interface 34 Display 35 Processing circuit 351 Control Functions 352 Inference Function 353 Acquisition Function 354 Interpretation Function 355 Judgment function 356 Correction Function 357 Generation function

Claims

1. an acquisition unit that acquires an inference result of an inference unit that performs inference based on one or more pieces of input information and first basis information that indicates the basis of the inference result; a determination unit that determines the validity of the inference result based on the first basis information, the determination unit determining the validity of the inference result based on the first basis information and the input information or knowledge information related to the inference result; a correction unit that corrects the inference result based on the knowledge information when the inference result is invalid; An information processing device comprising:

2. the determination unit determines the validity of the inference result corrected by the correction unit. The information processing device according to claim 1 .

3. the acquisition unit acquires second basis information different from the first basis information used by the determination unit to determine the validity before the correction by the correction unit; the determination unit determines the validity of the inference result based on the second basis information and the knowledge information related to the corrected inference result. The information processing device according to claim 2 .

4. a generation unit that generates visualized information that visualizes processing content related to the validity determination by the determination unit; a presentation unit that presents the visualized information to a user; Further comprising: The information processing device according to claim 1 .

5. the inference unit uses medical information including at least a medical image of a subject as the input information and infers information for supporting a medical procedure related to the subject. The information processing device according to claim 1 .

6. the inference unit uses a positioning image for determining an imaging position as the input information in the medical image diagnostic apparatus and infers information for supporting the determination of the imaging position. The information processing device according to claim 5 .

7. the inference unit infers information for assisting in determining imaging parameters for capturing a medical image using information on desired image quality as the input information in the medical image diagnostic apparatus; The information processing device according to claim 5 .

8. the inference unit infers information for supporting a diagnostic procedure related to a subject using a medical image captured by the medical image diagnostic device as the input information in the medical image diagnostic device; The information processing device according to claim 5 .

9. On the computer, an acquisition step of acquiring an inference result of an inference unit that performs inference based on one or more pieces of input information and first basis information that indicates the basis of the inference result; a determination step of determining the validity of the inference result based on the first basis information, the determination step determining the validity of the inference result based on the first basis information and the input information or knowledge information related to the inference result; a correcting step of correcting the inference result based on the knowledge information if the inference result is invalid; A program that executes the following.

Images