Medical information processing device, medical information processing method, and program

Abstract

To confirm clinical validity of grounds for an inference made by a machine learning model.SOLUTION: A medical information processing device according to an embodiment includes an acquisition unit, an inference grounds determination unit, a concept reflection degree calculation unit, and a dependency determination unit. The acquisition unit acquires a machine learning model and learning data used to train the machine learning model. The inference grounds determination unit determines each of inference grounds of the learning data and generates a visualized result of the inference grounds using the machine learning model. The concept reflection degree calculation unit determines a concept seriously considered during an inference made by the machine learning model on the basis of the visualized result of the inference grounds and calculates each concept reflection degree of the learning data with respect to a concept. The dependency determination unit generates visualized information on the dependency between the concept and a user-interpretable feature of the learning data on the basis of the concept reflection degree and the user-interpretable feature.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus, a medical information processing method, and a program.

Background Art

[0002] In the learning of a machine learning model, learning may be performed based on artifacts or biases (hereinafter referred to as "confounding factors") that are irrelevant to class classification. When learning is performed using such confounding factors as clues, inferences will be made based on false correlation relationships, resulting in a model with low accuracy for unknown data of types that were not included in the learning data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As a method of learning without being affected by confounding factors, a method has been proposed in which, from medical images, images with a high degree of similarity to an image specified by a user are extracted, and the images suitable as learning data are selected. However, in this method, since it is necessary to exclude images including confounding factors, when applied to a small amount of learning data, the number of data will be excessively reduced, and there is a risk of obtaining a model with low accuracy for unknown data.

[0006] Also, as a method of robustly learning with respect to confounding factors without reducing learning data, a method has been proposed in which the model is updated so as to be a basis for inference that emphasizes clinically appropriate information (hereinafter referred to as "clinical concept") without emphasizing information regarding confounding factors. However, in this method, it is necessary for doctors to manually label the clinical concepts for each learning data, which places a heavy burden on doctors. As an alternative method, it is conceivable to identify clinical concepts without a teacher by clustering the basis for inference and perform labeling. However, it is not easy to confirm whether the basis for inference of the learned machine learning model matches the clinical concept. As a result, there is a risk that learning will be performed while emphasizing incorrect clinical concepts.

[0007] The problem to be solved by the embodiments disclosed in this specification and the drawings is to enable confirmation of the clinical validity of the basis for inference of a machine learning model. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. It is also possible to position the problems corresponding to the respective effects of each configuration shown in the embodiments described later as other problems.

Means for Solving the Problem

[0008] The medical information processing apparatus according to the embodiment includes an acquisition unit, an inference basis determination unit, a concept reflection degree calculation unit, and a dependency relationship determination unit. The acquisition unit acquires a machine learning model and learning data used for learning the machine learning model. The inference basis determination unit determines the inference basis of each of the learning data using the machine learning model and generates an inference basis visualization result. The concept reflection degree calculation unit determines the concepts emphasized by the machine learning model during inference based on the inference basis visualization result, and calculates the concept reflection degree of each of the learning data regarding the concepts. The dependency relationship determination unit generates visualization information on the dependency relationship between the concepts and the user-interpretable features based on the concept reflection degree and the user-interpretable features in the learning data.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0010] Hereinafter, a medical information processing apparatus, a medical information processing method, and a program according to the embodiment will be described with reference to the drawings. The medical information processing apparatus according to the embodiment provides information indicating the relationship (dependency relationship) between the information (hereinafter referred to as "concept") that the machine learning model emphasizes during inference and the features calculated from the learning data, thereby enabling the confirmation of the clinical validity of the inference basis of the machine learning model. Furthermore, by updating the machine learning model so as to conform to the relationship edited according to the instructions of the operator (such as doctors), a machine learning model that can operate based on the inference basis corresponding to the intuition (clinical knowledge) of the doctor can be generated, and thus the accuracy of the machine learning model can be improved.

[0011] [Configuration of Medical Information Processing Apparatus] FIG. 1 is a functional block diagram showing an example of the configuration of the medical information processing apparatus 1 according to the embodiment. The medical information processing apparatus 1 is communicably connected to at least one or more terminal devices D via a communication network NW. The communication network NW means the entire information communication network using telecommunication technologies. For example, the communication network NW includes wireless / wired LANs such as a hospital backbone LAN (Local Area Network), the Internet, as well as a telephone communication line network, an optical fiber communication network, a cable communication network, and a satellite communication network. The medical information processing apparatus 1 is an example of a "medical information processing apparatus".

[0012] The terminal device D is operated by, for example, a doctor U (user) who checks various information regarding the machine learning model and controls the learning of the machine learning model. The terminal device D is, for example, a personal computer, a mobile terminal such as a tablet or a smartphone, or the like. The terminal device D includes a display function such as a liquid crystal display for displaying various information, and an input function which is an input interface for receiving various input operations by the doctor U. The input interface is realized by, for example, a mouse, a keyboard, a touch panel, a trackball, a switch, a button, a joystick, a camera, an infrared sensor, a microphone, or the like. Note that in this specification, the input interface is not limited to those having physical operation components such as a mouse and a keyboard. For example, an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the device and outputs this electric signal to the control circuit is also included in the examples of the input interface. The terminal device D is an example of a "terminal device".

[0013] The medical information processing device 1 includes, for example, a communication interface 10, a processing circuit 20, and a memory 30. The communication interface 10 communicates with an external device via a communication network NW. The communication interface 10 includes, for example, a NIC (Network Interface Card) and an antenna for wireless communication. The memory 30 stores a machine learning model M, learning clinical data TD (learning data), a feature filter bank FB, and the like. The memory 30 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like. The machine learning model M, the learning clinical data TD, and the feature filter bank FB may be stored in an external memory that the medical information processing device 1 can communicate with (or in addition to the memory 30). The external memory is controlled by, for example, a cloud server that manages the external memory and accepts read / write requests.

[0014] The machine learning model M is a model trained to perform desired classification processing or the like according to the purpose. The machine learning model M is, for example, a model that performs classification processing of skin diseases, classification processing of cell images, and the like. The machine learning model M is generated using any machine learning method such as a neural network, a support vector machine, or a decision tree. Neural networks include, for example, convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, and the like.

[0015] The training clinical data TD is the training data used in the training stage of the machine learning model M. The training clinical data TD is image data or non-image data. Image data includes, for example, MR (Magnetic Resonance) images, CT (Computed Tomography) images, images taken by imaging devices such as cameras and microscopes (e.g., skin images, cell images), and the like. Non-image data includes, for example, result data of specimen tests, vital data measured by electrocardiographs, pulse meters, and the like, and attribute information of the subject.

[0016] The processing circuit 20 controls the overall operation of the medical information processing apparatus 1. The processing circuit 20 executes, for example, an acquisition function 201, an inference basis determination function 203, a concept reflectivity calculation function 205, a dependency determination function 207, an editing function 209, a model update function 211, and a display control function 213. These functions are realized, for example, by a hardware processor (computer) executing a program (software) stored in the memory 30. The hardware processor means, for example, a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an SOC (System On Chip), an application-specific integrated circuit (ASIC), a programmable logic device (for example, a simple programmable logic device (SPLD) or a complex programmable logic device (CPLD), a field programmable gate array (FPGA)). Instead of storing the program in the memory 30, it may be configured to directly incorporate the program into the circuit of the hardware processor. In this case, the hardware processor realizes the function by reading and executing the program incorporated in the circuit. The hardware processor is not limited to being configured as a single circuit, and may be configured as one hardware processor by combining a plurality of independent circuits to realize each function. Also, a plurality of components may be integrated into one hardware processor to realize each function.

[0017] The acquisition function 201 acquires the machine learning model M stored in the memory 30, the learning clinical data TD used for learning the machine learning model M, various information transmitted from the terminal device D, and the like. The acquisition function 201 is an example of an "acquisition unit".

[0018] The inference basis determination function 203 uses the machine learning model M to determine the inference basis of each piece of learning clinical data TD and generates an inference basis visualization result. The inference basis determination function 203 generates, for example, an inference basis map. For the learning clinical data TD that is image data, the inference basis determination function 203 determines the inference basis and generates an inference basis visualization result (inference basis map) using, for example, the Grad-CAM method. Grad-CAM is a method for visualizing what the machine learning model M composed of a CNN focuses on in the image data (i.e., the inference basis). Also, for the learning clinical data TD that is non-image data, the inference basis determination function 203 determines the inference basis and generates an inference basis visualization result using the SHAP (Shapley Additive Explanations) method. SHAP is a method for calculating the contribution degree of each feature amount to the inference result of the machine learning model (the influence of the increase or decrease of the value of the feature amount variable on the inference result). The inference basis determination function 203 is an example of an "inference basis determination unit".

[0019] The concept reflection degree calculation function 205 determines the concepts that the machine learning model M emphasized during inference based on the inference basis visualization result, and calculates the concept reflection degree of each piece of learning clinical data TD related to the concept. As a first step, the concept reflection degree calculation function 205 extracts the main ones of the inference basis visualization results. For example, the concept reflection degree calculation function 205 clusters the inference basis maps based on the similarity of the appearance of the inference basis maps, and calculates the centroid of each cluster. Clustering includes, for example, methods such as k-means and hierarchical clustering. Similarity includes, for example, the cosine similarity of the Fourier transform result (in the case of image data), the cosine similarity (in the case of non-image data), etc. As a second step, the concept reflection degree calculation function 205 calculates the concept reflection degree of each piece of learning clinical data TD. For example, the concept reflection degree calculation function 205 calculates the concept reflection degree of the main concept for each piece of learning clinical data TD based on the distance from the cluster centroid of the clustered concepts. The concept reflection degree of the learning clinical data TD is calculated such that the closer the distance from the cluster centroid of the clustered concepts, the greater the concept reflection degree, and the farther the distance, the smaller the concept reflection degree. The concept reflection degree calculation function 205 is an example of a "concept reflection degree calculation unit".

[0020] The dependency relationship determination function 207 generates visualization information on the dependency relationship between the concept and the interpretation support feature based on the concept reflection degree and the features interpretable by a doctor (user) in the learning clinical data TD (hereinafter referred to as "interpretation support features"). The dependency relationship determination function 207, for example, uses the calculated concept reflection degree for each piece of learning clinical data TD as the target variable, and the predefined interpretation support features calculated from each of the learning clinical data TD as the explanatory variables, determines their dependency relationship, and generates a dependency relationship graph, which is visualization information, by graphical modeling. The dependency relationship determination function 207 is an example of a "dependency relationship determination unit".

[0021] Graphical modeling includes, for example, Markov random fields (MRFs) that use undirected graphs to represent correlation relationships, Bayesian networks that use directed graphs to represent causal relationships, and the like. The interpretation support features calculated from the learning clinical data TD are those obtained by converting features associated with a certain concept into information that is easy for doctors to interpret. When the learning clinical data TD is image data, the interpretation support features include, for example, at least one of features related to color (red component, brown component), features related to texture (uniformity, appearance frequency of elliptical or circular regions), and features related to shape (complexity of the contour with respect to the binarized result, circularity). When the learning clinical data TD is non-image data, the interpretation support features include, for example, features calculated based on predetermined clinical guidelines. Features calculated based on clinical guidelines include, for example, the APACHE (acute physiology and chronic health evaluation) score (a score that assesses the severity of ICU admitted patients from values such as respiration, circulation, and blood test values). Functions for expressing each interpretation support feature are predefined and registered in the feature filter bank FB stored in the memory 30. In addition, the doctor U may be able to create and set features by himself / herself.

[0022] The editing function 209 edits the visualization information (dependency graph) generated by the dependency determination function 207 according to the instructions of the doctor U (user) so as to conform to the doctor's intuition (clinical knowledge). Editing includes, for example, node addition, node deletion, edge editing, etc. Node addition means adding an interpretation support feature that conforms to the doctor's intuition, selected from the interpretation support features registered in the feature filter bank FB, to the dependency graph. Node deletion means deleting an interpretation support feature that does not conform to the doctor's intuition from the dependency graph. Edge editing means defining the strength of the dependency relationship in the dependency graph. In the case of features that are difficult to calculate with existing features, you may create them on your own by utilizing machine learning methods. By learning with deep distance learning so as to distinguish between the target image and the non-target image, a feature extractor that can quantify the reflectance of the target image is learned. After learning, it is newly added to the feature filter bank FB. For example, when it is desired to intentionally separate similar concepts, it can be handled by node addition or edge editing. For example, when emphasizing erythema and not emphasizing telangiectasia, by setting a positive correlation for the circular shape and a negative correlation for the complexity of the contour, among the concepts having the same red feature, only erythema can be emphasized. The editing function 209 is an example of an "editing section".

[0023] The model update function 211 performs additional learning on the machine learning model M and updates the machine learning model M based on the visualization information (dependency graph edited to conform to the doctor's intuition) edited by the editing function 209. The model update function 211 performs additional learning on the machine learning model M so that the dependency relationship derived from the machine learning model M matches the dependency relationship corresponding to the edited visualization information. For example, the loss function is defined as the sum of the classification error and the error between the dependency relationship derived from the model and the dependency relationship defined by the doctor, and additional learning is performed to change the weights of the machine learning model M so that this loss function becomes smaller. The model update function 211 is an example of a "model update section".

[0024] The display control function 213 causes a display device of the terminal device D to display visualization information (dependency graph) generated by the dependency determination function 207, GUI (Graphical User Interface) images, etc. for receiving various input operations by the doctor U such as an edit instruction for the machine learning model M. The display control function 213 is an example of a "display control unit".

[0025] [Processing Flow] Next, the processing flow of the medical information processing apparatus 1 will be described. FIG. 2 is a flowchart showing an example of the processing of the medical information processing apparatus 1 according to the embodiment. The processing shown in FIG. 2 is executed, for example, when clinically appropriate information is emphasized for a learned machine learning model M (for example, a skin cancer classification model) before product development and operation after the proof of concept (PoC) of the machine learning model M. The processing shown in FIG. 2 is started, for example, when the doctor U inputs an instruction to start via the input interface of the terminal device D. Hereinafter, the case where the machine learning model M is a skin cancer classification model and the learning clinical data TD is image data will be described as an example.

[0026] First, the acquisition function 201 acquires the machine learning model M to be evaluated and the learning clinical data TD used for learning this machine learning model M from the memory 30 (step S101). This learning clinical data TD is, for example, a mixture of learning data with correct labels of "cancer present" or "cancer absent" for skin cancer classification.

[0027] Next, the inference basis determination function 203 determines the inference basis of each of the learning clinical data TD using the machine learning model M and generates an inference basis map (step S103). The inference basis determination function 203 generates an inference basis map for each of the learning clinical data TD using, for example, the Grad-CAM method. For example, when the learning clinical data TD is 100 pieces of image data, the inference basis determination function 203 generates 100 inference basis maps.

[0028] Next, the concept reflection degree calculation function 205 clusters the inference basis maps according to the similarity of the appearances of the inference basis maps, and calculates the centroid of each cluster (step S105). For example, the concept reflection degree calculation function 205 clusters 100 inference basis maps by using a method such as k-means or hierarchical clustering based on the cosine similarity of the Fourier transform results of each of the 100 inference basis maps.

[0029] FIG. 3 is a diagram showing an example of a clustering result CR based on the inference basis map according to the embodiment. In FIG. 3, as representative concepts, an example in which a first concept C1 (centroid P1), a second concept C2 (centroid P2), and a third concept C3 (centroid P3) are calculated as main concepts is shown.

[0030] Next, the concept reflection degree calculation function 205 calculates the concept reflection degree of each of the learning clinical data TD (step S107). For example, the concept reflection degree calculation function 205 calculates the main concept reflection degree for each learning clinical data TD from the distance from the centroid of each cluster (concept). The concept reflection degree of the learning clinical data TD is calculated such that the closer it is to the centroid of each cluster, the greater the concept reflection degree, and the farther it is from the centroid, the smaller the concept reflection degree.

[0031] FIG. 4 is a diagram showing an example of the concept reflection degree RG for each learning clinical data according to the embodiment. In FIG. 4, the first concept C1, the second concept C2, and the third concept C3 calculated in FIG. 3 are defined on three axes, and the concept reflection degree RG for each of the learning clinical data TD is shown on the graph. On this graph, for example, a certain piece of learning clinical data TD1, which is one piece of image data, is closest to the centroid P3 of the third concept C3, and is farther away in the order of the centroid P2 of the second concept C2 and the centroid P1 of the first concept C1. That is, the magnitudes of the concept reflection degrees of the learning clinical data TD1 are in the order of the third concept, the second concept, and the first concept.

[0032] Returning to FIG. 2, next, the dependency determination function 207 determines the dependency between each of the representative concepts determined from the learning clinical data TD and the features (interpretation support features) extracted from the learning clinical data TD, and generates a dependency graph (step S109).

[0033] FIG. 5 is a diagram showing an example of a dependency graph DG according to the embodiment. In FIG. 5, the dependency between the third concept C3 (node destination) and each of the interpretation support features “circularity”, “uniformity”, and “red component” (node sources) extracted from the learning clinical data TD is expressed by a Markov random field (MRF). A numerical value (correlation coefficient) indicating the strength of the connection between the node destination and the node sources is attached to the edge connecting them. In the example of FIG. 5, it is shown that the dependency with the “red component” is the highest for the third concept.

[0034] Returning to FIG. 2, next, the display control function 213 causes the terminal device D to display a screen showing the clustering result and the dependency graph (step S111).

[0035] FIG. 6 is a diagram showing an example of a first screen PG1 including the clustering result CR and the dependency graph DG displayed on the terminal device D according to the embodiment. By checking the first screen PG1 displayed on the terminal device D, the doctor U can grasp the dependency between the concepts emphasized in the inference in the machine learning model M and the interpretation support features associated with each concept, and confirm whether the concept matches the intuition of the doctor U.

[0036] Returning to FIG. 2, next, the acquisition function 201 determines whether the dependency matches the intuition of the doctor U (whether an evaluation completion instruction has been received from the terminal device D) (step S113). When the doctor U who has checked the first screen PG1 displayed on the terminal device D thinks that the concept matches the intuition of the doctor U (that is, the concept of the machine learning model M is appropriate and additional learning is not required) and presses the evaluation completion button B5, the acquisition function 201 acquires the evaluation completion instruction (step S113: YES), and the processing of this flowchart ends.

[0037] On the one hand, when the doctor U who has confirmed the first screen PG1 displayed on the terminal device D believes that the concept does not match the intuition of the doctor U (that is, the concept of the machine learning model M is inappropriate and additional learning is required), and does not press the evaluation completion button B5, and when the acquisition function 201 receives an editing instruction without obtaining an evaluation completion instruction (step S113: NO), the editing function 209 edits the dependency relationship according to the received editing instruction (step S115). Editing includes, for example, node addition, node deletion, edge editing, etc.

[0038] (Node addition) FIG. 7 is a diagram showing an example of screen display when performing an editing process for adding a node according to an embodiment. When the doctor U who has confirmed the first screen PG1 displayed on the terminal device D operates the input interface and presses the node addition button B1, a second screen PG2 for specifying an interpretation support feature to be added as a node is displayed. The second screen PG2 lists the interpretation support features defined and registered in advance in the feature filter bank FB stored in the memory 30, and the doctor U can select and add a desired interpretation support feature. In the example of FIG. 7, the feature "red component" is selected, the dependency graph DG of the first screen PG1 is edited, and the node NN of "red component" is added. When the doctor U operates the input interface and presses the self-made feature addition button B6 provided on the second screen PG2, a third screen PG3 for self-making and adding an interpretation support feature as shown in FIG. 8 is displayed. The doctor U can newly register the definition of the interpretation support feature in the feature filter bank FB and add it to the dependency graph DG by inputting information about the interpretation support feature to be added on this third screen PG3.

[0039] (Node deletion) FIG. 9 is a diagram showing an example of a screen display when performing an editing process for node deletion according to an embodiment. When doctor U who has confirmed the first screen PG1 displayed on the terminal device D operates the input interface and presses the node deletion button B2, a fourth screen PG4 for specifying a node to be deleted is displayed. On the fourth screen PG4, the interpretation support features set as nodes in the dependency graph DG of the first screen PG1 are listed, and doctor U can select and delete the interpretation support feature to be deleted. In the example of FIG. 9, the feature "uniformity" is selected, the dependency graph DG of the first screen PG1 is edited, and the node DN of "uniformity" is deleted.

[0040] (Edge editing) FIG. 10 is a diagram showing an example of a screen display when performing an editing process for edge editing according to an embodiment. When doctor U who has confirmed the first screen PG1 displayed on the terminal device D operates the input interface and presses the edge editing button B3, a fifth screen PG5 for specifying an edge to be edited is displayed. On the fifth screen PG5, the edges indicated by the node source and the node destination set in the dependency graph DG of the first screen PG1 are listed, and doctor U can edit the edge by specifying the numerical value of the correlation coefficient of the edge to be edited (for example, moving the slider). In the example of FIG. 10, the edge with the node source being "red component" is selected and the numerical value of the correlation coefficient is changed to "0.7", and the dependency graph DG of the first screen PG1 is edited and the numerical value of the correlation coefficient of the edge E is changed to "0.7".

[0041] Returning to FIG. 2, after the dependency is edited by the editing function 209, when doctor U operates the input interface and presses the model update button B4 set on the first screen PG1, the model update function 211 performs additional learning of the machine learning model M based on the dependency edited by the editing function 209 (the dependency edited to match the doctor's intuition), and updates the machine learning model M (step S117).

[0042] FIG. 11 is a diagram showing an example of a sixth screen PG6 displayed during the update of the machine learning model according to the embodiment. On this sixth screen PG6, a display W1 indicating that additional learning is in progress is displayed. After the additional learning is completed, when the cursor is placed on each concept displayed in the clustering result CR included in the sixth screen PG6, the original image OG and the inference basis map EM are enlarged and displayed.

[0043] FIG. 12 is a diagram for explaining an example of the flow of the model update process according to the embodiment. The model update function 211 performs additional learning of the machine learning model M so that the dependency relationship derived from the machine learning model M matches the dependency relationship edited to match the intuition of the doctor. For example, the loss function is defined as the sum of the classification error and the error between the dependency relationship derived from the model and the dependency relationship defined by the doctor, and additional learning is performed to change the weights of the machine learning model M so that this loss function becomes small. As an example of the error, the dependency relationship between the concept and the feature based on the output of the machine learning model M (dependency relationship derived from the model, graph ML ) and the dependency relationship between the concept and the feature defined by the doctor (dependency relationship derived from the doctor's definition, graph User ) is used. The definitions of the various expressions shown in FIG. 12 are as follows. As an example of the method for calculating the similarity, the graph edit distance is used.

[0044]

Number

[0045]

Number

[0046]

Number

[0047]

Number

[0048] Return to FIG. 2. After the update of the machine learning model M is completed, return to step S103. Using the updated machine learning model M, the inference basis of each piece of learning clinical data TD is determined again to generate an inference basis map, and subsequent processing is repeated. Finally, when an evaluation completion instruction is received (step S113: YES), the processing of this flowchart ends.

[0049] According to the embodiment described above, by providing information indicating the relationship (dependency) between the information (concepts) emphasized by the machine learning model during inference and the features calculated from the learning data, it is possible to confirm the clinical validity of the inference basis of the machine learning model. Furthermore, by updating the machine learning model so as to conform to the dependency relationship edited according to the instructions of the operator (physician), it is possible to generate a machine learning model that can operate with an inference basis corresponding to the intuition (clinical knowledge) of the physician, and thus improve the accuracy of the machine learning model. As a result, when a physician uses a machine learning model for clinical decision support, it is possible to prevent referring to the inference results of a model affected by confounding factors.

[0050] Note that each function of the medical information processing apparatus 1 described in the above embodiment may be realized by installing an application in the terminal device D. In this case, the terminal device D is an example of a "medical information processing apparatus".

[0051] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and its equivalent scope.

Description of Reference Numerals

[0052] 1 Medical information processing apparatus 10 Communication Interface 20 Processing Circuit 30 Memory 201 Acquisition Function 203 Inference Basis Judgment Function 205 Concept Reflection Degree Calculation Function 207 Dependency Judgment Function 209 Editing Function 211 Model Update Function 213 Display Control Function D Terminal Device D

Claims

1. An acquisition unit that acquires a machine learning model and learning data used for learning the machine learning model; An inference basis determination unit that uses the machine learning model to determine the inference basis of each of the learning data and generates an inference basis visualization result; A concept reflectivity calculation unit that determines a concept emphasized by the machine learning model during inference based on the inference basis visualization result and calculates the concept reflectivity of each of the learning data regarding the concept; A dependency determination unit that generates visualization information on the dependency relationship between the concept and the user-interpretable features based on the concept reflectivity and the user-interpretable features in the learning data; A medical information processing device comprising the above.

2. An editing unit that edits the visualization information according to the user's instruction; A model update unit that updates the machine learning model based on the edited visualization information; The medical information processing device according to claim 1, further comprising the above.

3. The dependency determination unit generates a dependency graph, which is the visualization information, by graphical modeling. The medical information processing device according to claim 1 or 2.

4. The medical information processing device according to claim 1 or 2, further comprising a display control unit that causes the visualization information to be displayed on a display device.

5. The model update unit performs additional learning on the machine learning model so that the dependency relationship derived from the machine learning model matches the dependency relationship corresponding to the edited visualization information. The medical information processing device according to claim 2.

6. The concept reflectivity calculation unit determines the concept by clustering the inference basis visualization results based on the similarity of the inference basis visualization results, and calculates the concept reflectivity of each of the learning data based on the distance from the cluster center of the clustered concepts. The medical information processing device according to claim 1 or 2.

7. The concept reflectivity calculation unit calculates the concept reflectivity such that the closer the distance from the cluster center of the clustered concepts, the greater the concept reflectivity, and the farther the distance, the smaller the concept reflectivity. The medical information processing device according to claim 6.

8. When the learning data is image data, the user-interpretable features include at least one of features related to color, texture, and shape. The medical information processing device according to claim 1 or 2.

9. [[ID ​ ​ ​ When the learning data is non-image data, the interpretable features for the user include features calculated based on predetermined guidelines. The medical information processing apparatus according to claim 1 or 2.

10. The editing unit adds the interpretable feature specified by the user to the visualization information. The medical information processing apparatus according to claim 2.

11. The editing unit deletes the interpretable feature specified by the user from the visualization information. The medical information processing apparatus according to claim 2.

12. The editing unit newly adds a definition of the interpretable feature based on an instruction of the user. The medical information processing apparatus according to claim 2.

13. The interpretable features for the user are predefined. The medical information processing apparatus according to claim 1 or 2.

14. A computer acquires a machine learning model and learning data used for learning the machine learning model, uses the machine learning model to determine the inference basis for each of the learning data and generates an inference basis visualization result, based on the inference basis visualization result, determines a concept emphasized by the machine learning model during inference, and calculates the concept reflection degree of each of the learning data related to the concept, generates visualization information on the dependency relationship between the concept and the interpretable features for the user based on the concept reflection degree and the interpretable features for the user in the learning data. Medical information processing method.

15. Cause a computer to acquire a machine learning model and learning data used for learning the machine learning model, use the machine learning model to determine the inference basis for each of the learning data and generate an inference basis visualization result, based on the inference basis visualization result, determine a concept emphasized by the machine learning model during inference and calculate the concept reflection degree of each of the learning data related to the concept, generate visualization information on the dependency relationship between the concept and the interpretable features for the user based on the concept reflection degree and the interpretable features for the user in the learning data. Program.

Images