Medical information processing apparatus and medical information processing method

Abstract

To improve the convenience of a simulation technique for predicting conditions after treatment.SOLUTION: A medical information processing apparatus comprises: a first acquisition function that acquires first medical information related to a medical image obtained by imaging a target organ being a treatment target of a subject at a first time phase; a second acquisition function that acquires second medical information related to a target organ imaged at a second time phase different from the first time phase; an estimation function that obtains third medical information being the second time phase estimated from the first medical information on the basis of a learned model that has machine-learned a relation among the first medical information, preset first simulation parameters, and second medical information derived from the first medical information and the simulation parameters; and an optimization function that identifies second simulation parameters obtained by optimizing the first simulation parameters for the subject on the basis of third medical information repeatedly obtained with the estimation function and a first index indicating a difference between the second medical information and the third medical information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus and a medical information processing method.

[0002] Various treatment methods have been proposed for treating heart diseases. For example, a treatment method using a cardiac valve repair device using a catheter is known as a treatment method for diseases related to heart valves such as the mitral valve.

[0003] Known cardiac valve repair devices include, for example, mitral valve repair devices used to treat secondary mitral regurgitation (MR). Mitral valve repair devices are used in a procedure called Edge-to-Edge Repair, which increases the coaptation area by grasping the tips of the anterior and posterior leaflets of the mitral valve.

[0004] For example, when treatment is performed using a mitral valve repair device, a simulation may be performed before the actual treatment begins to predict post-treatment characteristics (e.g., valve area, regurgitation volume, etc.) for multiple treatment conditions (e.g., multiple gripping positions of the mitral valve repair device).

[0005] Conventionally, in the above-mentioned simulations, parameter optimization (hereinafter also referred to as personalization) for the purpose of estimating patient-specific parameters has been used to improve the accuracy of the simulations.

[0006] However, it is known that the above-mentioned methods generally require a large amount of computational time. For this reason, for example, personalization may not be applicable to patients who do not have time before surgery. It is also known that personalization generally requires a large amount of computational resources. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2022-73363 Summary of the Invention [Problem to be solved by the invention]

[0008] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the convenience of simulation technology for predicting post-treatment conditions. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0009] A medical information processing apparatus according to an embodiment includes a first acquisition unit, a second acquisition unit, an estimation unit, and an optimization unit. The first acquisition unit acquires first medical information related to medical images of a target organ to be treated in a subject at a first timing. The second acquisition unit acquires second medical information related to medical images of the target organ in the subject at a second timing different from the first timing. The estimation unit acquires third medical information, which is second medical information estimated from the first medical information, based on the first medical information, first simulation parameters used in a simulation process related to a predetermined target organ, and a trained model that has learned, using machine learning technology, the relationship between the first medical information, the simulation parameters, and the second medical information derived from the first medical information and the simulation parameters. The optimization unit identifies second simulation parameters that optimize the first simulation parameters for the subject, based on the third medical information repeatedly acquired by the estimation unit and an index indicating the difference between the second medical information and the third medical information. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a device placement simulation performed by the medical information processing apparatus according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a shape estimation model according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram showing an example of a method for generating a shape estimation model according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of the operation of the medical image processing apparatus according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of processing executed by the medical image processing apparatus according to the first embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the second embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the operation of the medical image processing apparatus according to the second embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of processing executed by the medical image processing apparatus according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the second modification. [Figure 11] FIG. 11 is a diagram illustrating an example of a distribution estimation method according to the second modification. [Figure 12] FIG. 12 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the third modification. [Figure 13] FIG. 13 is a diagram illustrating an example of a distribution estimation method different from the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a medical information processing apparatus and a medical information processing method will be described in detail with reference to the drawings.

[0012] (First embodiment) FIG. 1 is a diagram showing an example of the configuration of a medical information processing apparatus 100 according to the first embodiment.

[0013] 1, a medical information processing apparatus 100 according to the first embodiment is connected to a medical image diagnostic apparatus 30 and a medical image storage apparatus 50 via a network 10 so as to be able to communicate with each other. For example, the medical information processing apparatus 100 is installed in a medical setting such as a hospital or a clinic, and is used by an operator such as a doctor.

[0014] The medical image diagnostic device 30 acquires various medical images of a subject based on data collected from the subject. For the sake of specificity, the medical image diagnostic device 30 will be described as an X-ray CT (Computed Tomography) device, but the medical image diagnostic device 30 is not limited to this.

[0015] The medical image storage device 50 stores various medical images acquired by the medical image diagnostic device 30. For example, the medical image storage device 50 is realized by computer equipment such as a server or workstation equipped with a PACS (Picture Archiving Communication System) or the like.

[0016] The medical information processing device 100 acquires medical information about a subject from the medical image diagnostic device 30 and the medical image storage device 50 via the network 10, and executes simulation processing about the subject based on the acquired medical information. For example, the medical information processing device 100 is realized by computer equipment such as a server, a workstation, a personal computer, or a tablet terminal.

[0017] Specifically, the medical information processing device 100 includes a network (NW) interface 110, a memory 120, an input interface 130, a display 140, and a processing circuit 150.

[0018] The NW interface 110 controls the transmission and communication of various data sent and received between the medical information processing device 100 and other devices connected via the network 10 .

[0019] Specifically, the NW interface 110 is connected to the processing circuitry 150, and outputs medical images received from the medical image diagnostic device 30 or the medical image storage device 50 to the processing circuitry 150. For example, the NW interface 110 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0020] The memory 120 stores various data, various programs, etc. Specifically, the memory 120 is connected to the processing circuitry 150, and stores input medical images or outputs stored medical images to the processing circuitry 150 in response to commands sent from the processing circuitry 150. For example, the memory 120 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, etc.

[0021] For example, the memory 120 stores a shape estimation model 121. The shape estimation model 121 is a trained model generated by machine learning (including deep learning) using an organ model representing a target organ to be treated in a subject and simulation parameters for executing a simulation related to the target organ as training data. The shape estimation model 121 will be described later.

[0022] The input interface 130 receives input operations of various instructions and various information from an operator. Specifically, the input interface 130 is connected to the processing circuitry 150, converts the input operations received from the operator into electrical signals, and outputs the electrical signals to the processing circuitry 150.

[0023] For example, the input interface 130 may be realized by a trackball for setting a region of interest (ROI), a switch button, a mouse, a keyboard, a touchpad for performing input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, and a voice input circuit.

[0024] In this specification, the input interface 130 is not limited to an interface having physical operation parts such as a mouse, a keyboard, etc. For example, an example of the input interface 130 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

[0025] The display 140 displays various types of information and various types of data. Specifically, the display 140 is connected to the processing circuit 150 and displays various types of information and various types of data output from the processing circuit 150. For example, the display 140 is realized by a liquid crystal monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, or the like.

[0026] The processing circuitry 150 controls the components of the medical information processing device 100 in response to input operations received from an operator via the input interface 130. For example, the processing circuitry 150 stores medical images output from the NW interface 110 in the memory 120. Furthermore, for example, the processing circuitry 150 reads out the medical images from the memory 120 and displays them on the display 140.

[0027] The above has described the configuration of the medical information processing device 100 according to the first embodiment. With this configuration, the medical information processing device 100 according to the first embodiment executes a mitral valve simulation as a simulation process related to a subject, with the aim of supporting a treatment plan for a patient with mitral valve disease.

[0028] For example, the medical information processing device 100 executes a device placement simulation, which is a simulation of a treatment in which a specific device such as MitraClip (registered trademark) or PASCAL (Edwards Lifesciences) is placed in the mitral valve, as a mitral valve simulation. For example, in the device placement simulation, the shape and fluid information after device placement are predicted by simulation based on the type and placement position of the device input by the operator.

[0029] FIG. 2 is a diagram showing an example of a device placement simulation performed by the medical information processing apparatus 100 according to the first embodiment.

[0030] For example, when performing a device placement simulation using MitraClip (registered trademark), the medical information processing device 100 displays an image representing the shape of the heart near the mitral valve on the display 140, as shown in Fig. 2A. The medical information processing device 100 then receives, via the input interface 130, instructions from the operator to select the MitraClip (registered trademark) to be used in treatment and to set the placement position of the MitraClip (registered trademark) on the image.

[0031] Then, as shown in Figure 2 (B), the medical information processing device 100 predicts the shape and fluid information around the mitral valve after treatment through simulation based on the selected type of MitraClip (registered trademark) and the placement position of the MitraClip (registered trademark), and displays the predicted results on the display 140.

[0032] Generally, such mitral valve simulations are known to take a long time to process. Specifically, in mitral valve simulations, a large proportion of the time is spent estimating (personalizing) patient-specific simulation parameters used in the simulation. Therefore, there is a demand for reducing the time required to estimate simulation parameters used in simulation processing such as mitral valve simulation.

[0033] Therefore, the medical information processing apparatus 100 according to the first embodiment is configured to be able to reduce the time required to estimate simulation parameters used in simulation processing such as mitral valve simulation.

[0034] The medical information processing apparatus 100 according to the first embodiment will be described in detail below.

[0035] In this embodiment, as an example, a case will be described in which a mitral valve mesh (organ model) for a patient with mitral regurgitation is used to simulate the shape of the mitral valve after Edge to Edge Repair (e.g., the valve orifice area during cardiac systole).

[0036] In the medical image processing device 100 according to the first embodiment, the processing circuitry 150 has a learning function 151, a first acquisition function 152, a second acquisition function 153, an estimation function 154, a calculation function 155, an optimization function 156, and a simulation function 157. Here, the first acquisition function 152 is an example of a first acquisition unit. The second acquisition function 153 is an example of a second acquisition unit. The estimation function 154 is an example of an estimation unit. The optimization function 156 is an example of an optimization unit.

[0037] In this embodiment, the processing circuitry 150 is realized by, for example, a processor. In this case, the above-described processing functions are stored in the memory 120 in the form of programs executable by a computer. The processing circuitry 150 then reads and executes the programs stored in the memory 120 to realize the functions corresponding to the programs. In other words, the processing circuitry 150 has the processing functions shown in FIG. 1 when the programs are read.

[0038] The learning function 151 learns the shape estimation model 121. The shape estimation model 121 according to the first embodiment will be described below with reference to Fig. 3 and Fig. 4. Fig. 3 is a diagram showing an example of the shape estimation model 121.

[0039] As shown in FIG. 3, the shape estimation model 121 is a trained model that outputs a model representing the open shape of the mitral valve (the mitral valve in the mid-diastolic phase) (hereinafter also referred to as the open shape model) and a model representing the closed shape of the mitral valve (the mitral valve in the early systolic phase) (hereinafter also referred to as the closed shape model) in response to input of simulation parameters.

[0040] Here, the simulation parameters are parameters used to simulate the dynamics of the mitral valve or the dynamics of blood flow near the mitral valve using numerical mechanics techniques such as the finite element method, the finite difference method, the immersion boundary method, and the particle method.

[0041] Specifically, the simulation parameters are parameters related to the calculation of forces such as elastic force, contact force, viscous force, external force, and friction force.

[0042] Some of the simulation parameters used are specific to the subject (hereinafter referred to as intrinsic parameters), such as the shape of the anterior and posterior leaflets, the Young's modulus of the anterior and posterior leaflets, the time change in pressure on the valve, the natural length of the chordae tendineae, the position of the chordae tendineae, the viscosity coefficient, the bulk modulus, the fiber direction of the valve, and residual stress.

[0043] Next, a description will be given of a method for generating the shape estimation model 121. Fig. 4 is an explanatory diagram showing an example of a method for generating the shape estimation model 121. As shown in Fig. 4, the shape estimation model 121 uses, as learning data, an open shape model and a plurality of simulation parameters based on medical images collected from a plurality of subjects, and a closed shape model based on a simulation by calculation using the open shape model and the plurality of simulation parameters (hereinafter also referred to as a computational closed shape model).

[0044] By performing machine learning using the above learning data, it is possible to generate a shape estimation model 121 that has learned the relationship between an open shape model and simulation parameters and a closed shape model. Specifically, a trained model (shape estimation model) that is functionalized to output a closed shape model in response to input of an open shape model and simulation parameters is generated.

[0045] Specifically, the learning function 151 collects CT images of mitral valves in an open shape (mid-diastole) from a plurality of subjects. Furthermore, a first acquisition function 152, which will be described later, acquires a plurality of open shape models based on the plurality of CT images collected by the learning function 151. Furthermore, the learning function 151 prepares a plurality of simulation parameters.

[0046] For example, the learning function 151 prepares a plurality of simulation parameters by setting a possible range for each parameter based on literature, doctor's knowledge, etc., and randomly selecting parameters from within that range.

[0047] Furthermore, for example, the learning function 151 prepares a plurality of simulation parameters by setting the distribution of each parameter based on literature, doctor's knowledge, etc., and randomly selecting parameters from the distribution.

[0048] Furthermore, for example, the learning function 151 prepares multiple simulation parameters by performing a simulation in which the possible ranges of some parameters are set based on literature, doctors' knowledge, etc., and estimating the possible ranges of other parameters.

[0049] Furthermore, the learning function 151 causes the simulation function 157, which will be described later, to predict the closed mitral valve shape by computational simulation using the open shape model acquired by the first acquisition function 152 and the simulation parameters prepared by the learning function 151. This allows a computational closed shape model based on the prediction results to be obtained.

[0050] The learning function 151 inputs the open shape model acquired by the first acquisition function 152, the prepared plurality of simulation parameters, and the calculated closed shape model obtained by the simulation by the simulation function 157 as learning data into the machine learning engine, and causes machine learning to be performed. Then, the learning function 151 generates a trained model that is functionalized to output a closed shape model that represents the shape of the mitral valve in a closed state in response to the input of the open shape model and the plurality of simulation parameters.

[0051] Here, as the machine learning engine, for example, a neural network described in the well-known non-patent document "Pattern recognition and machine learning" by Christopher M. Bishop, (USA), 1st edition, Springer, 2006, pp. 225-290, or the like can be applied.

[0052] In addition to the neural network described above, the machine learning engine may also use various algorithms such as deep learning (for example, deep learning models such as full convolutional network, U-net, transformer, or a combination of these), logistic regression analysis, nonlinear discriminant analysis, support vector machine (SVM), random forest, and naive Bayes.

[0053] Furthermore, for example, the learning function 151 causes the shape estimation model 121 to learn so that a closed shape model equivalent to a calculated closed shape model obtained by the simulation function 157 executing a simulation is output.

[0054] For example, the learning function 151 sets a learning termination condition so that the above can be achieved. As an example, the learning termination condition is that a loss value representing the deviation between the calculated closed shape model (correct value) and the output closed shape model (predicted value) becomes equal to or less than a preset threshold. The loss value is calculated using a loss function.

[0055] The loss function is a function for calculating, for example, the average deviation of node positions across the entire valve, the average deviation of node positions in the valve orifice, an index relating to the penetration of the anterior and posterior leaflets, an index relating to the symmetry of the valve shape, and the like.

[0056] The condition for terminating the learning may be other conditions, such as the number of epochs, which indicates how many times the learning data is to be repeatedly learned, reaching a preset threshold, etc. The condition for terminating the learning may also be a combination of a condition related to the loss value and a condition related to the number of epochs.

[0057] As a result of such machine learning, the learning function 151 generates a shape estimation model 121 that is functionalized to output a "closed shape model" in response to input of multiple simulation parameters such as an "open shape model" and "shapes of the anterior and posterior leaflets" and "Young's moduli of the anterior and posterior leaflets" that are input to the learned algorithm.

[0058] Note that dimensionality reduction may be performed using a deep learning model, proper orthogonal decomposition, etc. as preprocessing of the shape estimation model 121. Furthermore, a layer that can take time series into consideration, such as a recurrent neural network or long short-term memory, may be added to the shape estimation model 121.

[0059] In this embodiment, a simulation for predicting a closed shape (an early systole phase) from an open shape (a mid-diastole phase) will be described, but the content of the simulation is not limited to this.

[0060] For example, the simulation may take a specific phase identified based on an electrocardiogram other than mid-diastole as input and predict a specific phase different from the input phase. For example, the specific phase identified based on an electrocardiogram may be mid-systole, 0% cardiac cycle, etc.

[0061] Furthermore, for example, the simulation may use as input a phase having a specific characteristic identified based on the morphological characteristics of the valve or the characteristics of the fluid near the valve extracted from a CT image, and predict a phase different from the phase related to the input. As an example, the phase having a specific characteristic identified based on the morphological characteristics of the valve or the characteristics of the fluid near the valve extracted from a CT image may be the phase in which the mitral valve regurgitation volume is the largest or the phase in which the mitral valve opening area is the largest.

[0062] The first acquisition function 152 acquires first medical information based on a first shape of the target organ in a predetermined first time phase. For example, the first acquisition function 152 acquires a CT image of the subject's mitral valve in mid-diastole from the medical image diagnostic device 30 or the medical image storage device 50. The mid-diastole phase is an example of the first time phase. The open shape is also an example of the first shape.

[0063] Furthermore, for example, the first acquisition function 152 acquires coordinate information of each pixel corresponding to the mitral valve region in the CT image. For example, the first acquisition function 152 receives a designation input of the mitral valve region from the user, and identifies the mitral valve region in the CT image in accordance with the input.

[0064] The identified mitral valve region is represented, for example, by a three-dimensional mesh model. Here, the three-dimensional mesh model represents the mitral valve region as a computational grid (hereinafter also referred to as a mesh) by setting a plurality of grid points on the identified mitral valve region. In this case, the number and arrangement of the grid points may be determined in advance, or may be determined based on the size, shape, etc. of the mitral valve region.

[0065] The first acquisition function 152 acquires a three-dimensional mesh model representing the mitral valve in mid-diastole as an open shape model. The open shape model in this case is an example of first medical information.

[0066] The method for representing the mitral valve region is not limited to a three-dimensional mesh model. For example, the mitral valve region may be represented by a surface model. Alternatively, the mitral valve region may be represented by masking portions of the CT image data other than the identified mitral valve region.

[0067] The first acquisition function 152 may also identify the mitral valve region based on an anatomical structure extracted from a CT image by an existing region extraction method, such as Otsu's binarization method based on CT values, region growing, snake algorithm, graph cut algorithm, and mean shift algorithm.

[0068] The first acquisition function 152 may also identify the mitral valve region using a shape model generated by an existing machine learning technique (including deep learning).

[0069] In this case, for example, the first acquisition function 152 applies a shape model to the acquired CT image, which is capable of extracting coordinate information of a plurality of pixels corresponding to the mitral valve region in the CT image, and then identifies the mitral valve region based on the extraction result obtained by applying the shape model.

[0070] In this case, the shape model is a trained model that learns the relationship between the two using a dataset in which, for example, a CT image depicting the mitral valve region is used as input training data and coordinate information of multiple pixels corresponding to the mitral valve region in the CT image is used as output training data.

[0071] Furthermore, for example, the first acquisition function 152 may receive a selection input from the user as to which of a plurality of region extraction techniques to use for region extraction, and identify the mitral valve region using the selected region extraction technique.

[0072] The second acquisition function 153 acquires second medical information based on the first shape of the target organ in a second time phase different from the first time phase. For example, the second acquisition function 153 acquires a CT image of the subject's mitral valve in early systole from the medical image diagnostic device 30 or the medical image storage device 50. The early systole phase is an example of the second time phase. The closed shape is also an example of the second shape.

[0073] Furthermore, for example, the second acquisition function 153 acquires a correct closed shape model, which is a closed shape model based on a CT image of the mitral valve in the early stage of contraction, using a method similar to that of the first acquisition function 152 described above.

[0074] The estimation function 154 estimates the state of the target organ of the subject in the second time phase based on the first medical information. For example, the estimation function 154 inputs the open shape model acquired by the first acquisition function 152 and a plurality of simulation parameters (initial parameters) used in the initial estimation process to the shape estimation model 121. The initial parameters will be described later.

[0075] The estimation function 154 acquires the closed shape model output by the shape estimation model 121 as an estimated closed shape model estimated from the open shape model and initial parameters. The estimated shape model is an example of third medical information.

[0076] Furthermore, for example, when updated simulation parameters (hereinafter also referred to as update parameters) are sent from the optimization function 156 described below, the estimation function 154 inputs the open shape model acquired by the first acquisition function 152 and the multiple update parameters sent from the optimization function 156 to the shape estimation model 121.

[0077] The estimation function 154 obtains the closed shape model output by the shape estimation model 121 as an estimated closed shape model estimated from the open shape model and the update parameters. The estimation function 154 repeats the above process until the optimization function 156 (described later) determines that the termination condition of the optimization algorithm is satisfied.

[0078] The calculation function 155 calculates a loss value from the correct closed shape model acquired by the second acquisition function 153 and the estimated closed shape model acquired by the estimation function 154.

[0079] The loss value is calculated from a loss function. For example, the loss function is a function that calculates the average of the nodal position deviations for the entire valve, the average of the nodal position deviations for the valve orifice, an index for the penetration of the anterior and posterior leaflets, an index for the symmetry of the valve shape, and combinations of these. The calculation function 155 sends the calculation results to the optimization function 156, which will be described later.

[0080] The optimization function 156 executes the personalization process in cooperation with other functional units. The operation of each functional unit in the personalization process according to the first embodiment will be described below with reference to Fig. 5. Fig. 5 is a diagram illustrating an example of the operation of the medical information processing apparatus 100 according to the first embodiment.

[0081] First, the optimization function 156 sends predetermined initial parameters to the estimation function 154 .

[0082] The optimization function 156 may determine the initial parameters based on a CT image or various medical information. For example, the optimization function 156 may determine the initial parameters by determining the Young's modulus of the valve according to the age of the patient, or by determining the number of chordae tendineae to be set based on the number of chordae tendineae that can be confirmed on the CT image.

[0083] The optimization function 156 may also determine the initial parameters by setting a possible range for each parameter based on literature, doctor's knowledge, etc., and randomly selecting from that range. The optimization function 156 may also determine the initial parameters by setting a distribution for each parameter based on literature, doctor's knowledge, etc., and randomly selecting from that distribution.

[0084] The optimization function 156 may also estimate the possible ranges of other parameters by performing a simulation in which the possible ranges of some parameters are set based on literature, physician knowledge, etc. Here, the possible ranges of each parameter may be fixed for all patients, or may be changed depending on the attributes of the patient. Examples of patient attributes include gender, height, weight, race, and concurrent diseases.

[0085] The optimization function 156 also sets an optimization algorithm to be used in the personalization process. For example, the optimization function 156 can use known optimization methods such as the Levenburg-Markart method, a Kalman filter, and a Covariance Matrix Adaptation Evolution Strategy (CMA-ES).

[0086] The hyperparameters in each method may be set in advance, changed depending on the patient, or changed depending on the patient's condition. Here, the hyperparameters are parameters that are set in advance before the personalization process.

[0087] For example, when using CMA-ES, the optimization function 156 repeats the steps of "generating multiple individuals according to a multivariate normal distribution → extracting the top N individuals with the smallest loss values ​​from the generated individuals → updating the multivariate normal distribution based on the extracted individuals."

[0088] In this case, the optimization function 156 empirically sets the number of individuals to be generated and the number of individuals to be extracted. Note that if a large difference is observed between the patient group used for learning and the patient background, the optimization function 156 may increase the number of individuals to be generated or the number of individuals to be extracted. In this case, the optimization function 156 may determine the magnitude of the difference based on the patient's weight, race, presence or absence of congenital heart disease, etc.

[0089] Furthermore, the first acquisition function 152 sends the acquired open shape model to the estimation function 154. The estimation function 154 inputs the open shape model sent from the first acquisition function 152 and the initial parameters sent from the optimization function 156 to the shape estimation model 121. Next, the estimation function 154 acquires the estimated closed shape model output by the shape estimation model 121.

[0090] The estimation function 154 sends the acquired estimated closed shape model to the calculation function 155. Furthermore, the second acquisition function 153 sends the acquired correct closed shape model to the calculation function 155. Next, the calculation function 155 calculates a loss value from the correct closed shape model sent from the second acquisition function 153 and the estimated closed shape model sent from the estimation function 154. The calculation function 155 sends the calculation result (loss value) to the optimization function 156.

[0091] Next, the optimization function 156 updates some of the simulation parameters using the set optimization algorithm based on the calculation results sent from the calculation function 155. For example, the optimization function 156 updates some of the simulation parameters so that the output closed shape model approaches the correct closed shape model.

[0092] Here, the optimization function 156 sets simulation parameters to be updated in advance. For example, the optimization function 156 sets simulation parameters that vary significantly depending on the subject, other than the shape of the mitral valve, and simulation parameters that have a large influence on the index to be calculated, as simulation parameters to be updated. For example, the optimization function 156 calculates the magnitude of the influence on the index to be calculated by performing a sensitivity calculation.

[0093] Specifically, the optimization function 156 obtains the average value of a specific simulation parameter from literature, etc. The optimization function 156 also prepares five levels in increments of ±10% of the average value and checks how much the index to be calculated changes. The optimization function 156 sets parameters to be updated depending on the magnitude of the change.

[0094] The optimization function 156 sends the updated simulation parameters (updated parameters) to the estimation function 154. Then, the estimation function 154 repeats the process of inputting the open shape model and the updated parameters to the shape estimation model 121 to obtain an estimated closed shape model until the optimization function 156 determines that the termination condition of the optimization algorithm is satisfied.

[0095] Here, the termination condition of the optimization algorithm is, for example, when the loss value calculated by the calculation function 155 falls below a preset threshold, or when the number of iterations of the estimation process by the estimation function 154 exceeds a preset threshold.

[0096] Continuing the explanation, returning to Figure 1, the simulation function 157 executes a simulation regarding the target organ.

[0097] For example, the simulation function 157 determines a treatment technique to be simulated. The treatment technique may be manually set by the user or may be set in advance. For example, a technique of grasping the center of the anterior leaflet and the center of the posterior leaflet of the mitral valve with MitraClip® may be set in advance as the treatment technique.

[0098] Furthermore, for example, although MitraClip® devices come in various sizes, it may be preset to use the smallest NT. It may also be preset to grasp the MitraClip® at the location of the greatest amount of regurgitation identified from the CT image.

[0099] Furthermore, for example, the simulation function 157 may change the size of the MitraClip® depending on the condition of the subject. As an example, if the amount of regurgitation exceeds a threshold, the simulation function 157 may automatically change the size of the MitraClip® from a preset NT to an XTW.

[0100] Alternatively, for example, the user may determine some of the settings related to the MitraClip (registered trademark), and other settings may be automatically set by the simulation function 157. As an example, the size of the MitraClip (registered trademark) may be manually set by the user, and the position of the MitraClip (registered trademark) may be automatically set by the simulation function 157.

[0101] The simulation function 157 uses the calculation model and simulation parameters including the specific parameters personalized by the optimization function 156 to perform a calculation simulation and predict the state after treatment.

[0102] Here, the calculation model is, for example, a model that represents the anterior and posterior leaflets of the mitral valve. As an example, the anterior and posterior leaflets in the calculation model are represented by tetrahedral elements, hexahedral elements, shell elements, etc.

[0103] When expressing the anterior and posterior leaflets of the mitral valve, the thickness of the anterior and posterior leaflets of the mitral valve may be assumed to be constant. Alternatively, the thickness may be set separately for the anterior and posterior leaflets. Alternatively, the thickness may be set to reflect information about the thickness of the valve obtained from CT images.

[0104] Furthermore, for example, the computational model may represent not only the anterior and posterior leaflets of the mitral valve, but also the chordae tendineae, papillary muscles, blood near the mitral valve, etc. As an example, the chordae tendineae in the computational model are represented by tetrahedral elements, hexahedral elements, string elements, etc.

[0105] When representing the chordae tendineae, a predetermined number of chordae may be placed at predetermined positions on the model, or chordae may be placed on the model according to the positional information of the chordae obtained from the CT image.

[0106] Chordae can also be placed on the model based on the shape of the valve. For example, if a CT image indicates that the valve is prolapsed, the chordae near the prolapsed area can be cut, or if there is an area where part of the valve is being pulled, chordae can be placed in that area.

[0107] The above calculation model can also be used in place of the open shape model acquired by the first acquisition function 152 during the learning process of the shape estimation model 121 by the learning function 151 described above.

[0108] For example, the simulation function 157 predicts, through a computational simulation, the behavior of the anterior and posterior leaflets of the mitral valve being grasped by the MitraClip (registered trademark).

[0109] Furthermore, for example, when performing a simulation, the simulation function 157 models the MitraClip (registered trademark) as a rigid body and then predicts the shape change of the MitraClip (registered trademark) when it is grasped, or simulates the grasping behavior of the MitraClip (registered trademark) using a spring.

[0110] The simulation function 157 also controls the display 140 to display the predicted results of the state of the mitral valve after treatment.

[0111] Next, a description will be given of the processing executed by the medical information processing apparatus 100 according to the first embodiment. Fig. 6 is a flowchart showing an example of the processing executed by the medical information processing apparatus 100 according to the first embodiment.

[0112] First, the first acquisition function 152 acquires an open shape model of the subject (step S101). For example, the first acquisition function 241 acquires a CT image of the subject's mitral valve in mid-diastole. The first acquisition function 152 extracts a mitral valve region from the CT image to acquire an open shape model expressed as a 3D mesh model. The first acquisition function 152 sends the acquired open shape model to the estimation function 154.

[0113] Next, the second acquisition function 153 acquires a correct closed shape model of the subject (step S102). For example, the second acquisition function 153 acquires a CT image of the subject's mitral valve in early contraction. The second acquisition function 153 extracts the mitral valve region from the CT image to acquire a correct closed shape model expressed as a 3D mesh model. The second acquisition function 153 sends the acquired correct closed shape model to the calculation function 155.

[0114] Next, the optimization function 156 determines initial parameters (step S103). For example, the optimization function 156 acquires information about the attributes of the subject from an external device such as an electronic medical record system, and determines initial parameters according to the attributes. The optimization function 156 sends the determined initial parameters to the estimation function 154.

[0115] Next, the estimation function 154 estimates the closed shape of the mitral valve from the open shape of the mitral valve (step S104).

[0116] For example, the estimation function 154 inputs the open shape model acquired in step S101 and the initial parameters determined in step S103 to the shape estimation model 121. The estimation function 154 acquires the closed shape model output from the shape estimation model 121 as an estimated closed shape model. The estimation function 154 sends the acquired estimated closed shape model to the calculation function 155.

[0117] Next, the calculation function 155 calculates a loss value (step S105). For example, the calculation function 155 calculates, as a loss value, an average of deviations in node positions of the entire valve between the correct closed shape model acquired in step S102 and the estimated closed shape model acquired in step S104. The calculation function 155 sends the calculation result to the optimization function 156.

[0118] Next, the optimization function 156 determines whether the termination condition of the optimization algorithm is satisfied (step S106). For example, if the loss value calculated in step S105 is below a threshold, the optimization function 156 determines that the termination condition of the optimization algorithm is satisfied. On the other hand, if the loss value is equal to or greater than the threshold, the optimization function 156 determines that the termination condition of the optimization algorithm is not satisfied.

[0119] If the termination condition of the optimization algorithm is not satisfied (step S106: No), the optimization function 156 updates the simulation parameters (step S108). For example, the optimization function 156 updates a plurality of simulation parameters that have been set in advance as update targets so that the closed shape model output from the shape estimation model 121 approaches the correct closed shape model.

[0120] Thereafter, the optimization function 156 sends the updated update parameters to the estimation function 154, and the process returns to step S106. In this case, the estimation function 154 inputs the open shape model acquired in step S101 and the update parameters updated in step S108 to the shape estimation model 121.

[0121] On the other hand, if the termination condition of the optimization algorithm is satisfied in step S106 (step S106: Yes), the optimization function 156 determines the simulation parameters to be used in the simulation for predicting the post-treatment state (step S107).

[0122] For example, when the termination condition of the optimization algorithm is satisfied, the optimization function 156 determines the simulation parameters most recently input to the shape estimation model 121 as the simulation parameters to be used in the simulation for predicting the post-treatment state.

[0123] Next, the simulation function 157 executes a simulation of the state after treatment (step S109).

[0124] For example, the simulation function 157 receives input from the user of the type of MitraClip (registered trademark) as a treatment method and the placement location of the MitraClip (registered trademark). The simulation function 157 performs a simulation using the parameter set determined in step S107 to predict the shape and valve orifice area of ​​the mitral valve when treatment is performed using the input treatment method.

[0125] Next, the simulation function 157 controls the display 140 to display the simulation results (step S110), and then ends this process. For example, the simulation function 157 causes the display 140 to display the shape and valve orifice area of ​​the mitral valve after treatment predicted in step S108.

[0126] The medical information processing apparatus 100 according to the first embodiment described above inputs an open shape model and simulation parameters representing a mitral valve in an open state based on a CT image into the shape estimation model 121, which is a trained model that has learned, using machine learning technology, the relationship between the open shape model and simulation parameters and a closed shape model representing a mitral valve in a closed state, thereby obtaining an estimated closed shape model that estimates the shape of the mitral valve in a closed state, and updates the simulation parameters based on an index representing the difference between the estimated closed shape model and a correct closed shape model representing a mitral valve in a closed state based on a CT image. Furthermore, when the index representing the difference between the estimated closed shape model and the correct closed shape model falls below a threshold, the medical information processing apparatus 100 according to the first embodiment determines the simulation parameters most recently input to the shape estimation model 121 as the simulation parameters optimized for the subject.

[0127] The medical information processing apparatus 100 according to the first embodiment estimates the shape of a closed mitral valve from the shape of an open mitral valve using the shape estimation model 121, which is a trained model trained using machine learning technology. This allows for faster estimation than when a similar estimation is performed using a computational simulation. In the personalization process that optimizes simulation parameters for a subject, the process of estimating the shape of a closed mitral valve from the shape of an open mitral valve is repeatedly performed. Therefore, the medical information processing apparatus 100 according to the first embodiment allows for faster personalization than when a similar estimation is performed using a computational simulation. Furthermore, by speeding up the personalization process, it becomes more likely that the personalization process can be performed even in situations where, for example, a computational simulation would not be possible due to time constraints. Naturally, personalization improves the accuracy of post-treatment simulations. In other words, the medical information processing apparatus 100 according to the first embodiment can simultaneously achieve faster processing speeds and improved accuracy, thereby improving the convenience of simulation techniques for predicting post-treatment conditions.

[0128] (Second embodiment) In the first embodiment described above, a form in which personalization processing is performed using the shape estimation model 121, which is a trained model trained by machine learning technology, has been described. In the second embodiment, a form in which calculation-based simulation is also used in addition to the shape estimation model 121 will be described.

[0129] In the following, differences from the above-described embodiment will be mainly described, and detailed descriptions of commonalities with the contents already described will be omitted. Furthermore, each embodiment described below may be implemented individually or in appropriate combination.

[0130] First, the configuration of the medical information processing apparatus 100a according to the second embodiment will be described below. Fig. 7 is a diagram showing an example of the configuration of the medical information processing apparatus 100a according to the second embodiment.

[0131] As shown in Figure 7, the medical information processing device 100a according to the second embodiment has a configuration substantially similar to that of the medical information processing device 100 according to the first embodiment in Figure 1, but differs from the medical information processing device 100 according to the first embodiment in that it includes a processing circuit 150a.

[0132] Furthermore, the processing circuit 150a differs from the processing circuit 150 according to the first embodiment in that it includes a calculation function 155a, an optimization function 156a, a simulation function 157a, and an adjustment function 158.

[0133] When a calculated closed shape model is sent from a simulation function 157a (described later), the calculation function 155a calculates a loss value from the correct closed shape model acquired by the second acquisition function 153 and the calculated closed shape model sent from the simulation function 157a. The calculation function 155a sends the calculation result to an adjustment function 158 (described later).

[0134] The optimization function 156a performs the same processing as the above-described optimization function 156. However, when the optimization function 156a uses a simulation based on calculations for the personalization processing, it sets a more relaxed termination condition for the optimization algorithm compared to when the simulation based on calculations is not used.

[0135] For example, when a loss value is used as a termination condition for the optimization algorithm, the optimization function 156a sets a larger value as the threshold for the loss value than when a simulation based on calculation is not used. Also, when the number of iterations is used as a termination condition for the optimization algorithm, the optimization function 156a sets a smaller value as the threshold for the number of iterations than when a simulation based on calculation is not used.

[0136] Furthermore, when it is determined that the termination condition of the optimization algorithm is satisfied, the optimization function 156a determines as the optimization parameters the update parameters most recently input to the shape estimation model 121. The optimization function 156a sends the determined optimization parameters to the adjustment function 158.

[0137] When adjustment parameters described below are sent from the adjustment function 158, the simulation function 157a predicts the shape of the mitral valve in a closed state from the open shape model acquired by the first acquisition function 152 through a simulation based on calculations using the adjustment parameters sent from the adjustment function 158. Furthermore, the simulation function 157a sends the prediction result to the calculation function 155a as a calculated closed shape.

[0138] The adjustment function 158 adjusts the optimization parameters sent from the optimization function 156 a. For example, the adjustment function 158 adjusts the optimization parameters using a calculation-based simulation with an algorithm similar to the optimization algorithm (hereinafter, the algorithm used by the adjustment function 158 is also referred to as an adjustment algorithm).

[0139] Here, the adjustment function 158 sets an end condition for the enhanced adjustment algorithm compared to the end condition for the optimization algorithm when no computational simulation is used.

[0140] As an example, when a loss value is used as the termination condition of the adjustment algorithm, the adjustment function 158 sets a threshold value for the loss value that is greater than the termination condition of the optimization algorithm when no calculation-based simulation is used.

[0141] The operation of each functional unit in the personalization process according to the second embodiment will be described below with reference to Fig. 8. Fig. 8 is a diagram illustrating an example of the operation of the medical information processing apparatus 100a according to the second embodiment. The explanation of Fig. 8 is based on the premise that the same process as that shown in Fig. 5 above has been executed and the termination condition of the optimization algorithm has been satisfied.

[0142] The optimization function 156a determines, as the optimization parameters, the update parameters most recently input to the shape estimation model 121. The optimization function 156a sends the determined optimization parameters to the adjustment function 158.

[0143] Next, the adjustment function 158 sends the optimization parameters sent from the optimization function 156a to the simulation function 157a. Here, in the first processing, the optimization parameters are sent to the simulation function 157a as they are, but for convenience of explanation, the simulation parameters sent from the adjustment function 158 to the simulation function 157a will be called adjustment parameters even if they have not been adjusted.

[0144] Furthermore, the first acquisition function 152 sends the acquired open shape model to the simulation function 157a. This open shape model is the same as the open shape model sent by the first acquisition function 152 to the estimation function 154 in the parameter optimization process.

[0145] The simulation function 157 a predicts the shape of the mitral valve in a closed state from the open shape model sent from the first acquisition function 152 by simulation through calculation using the adjustment parameters sent from the adjustment function 158 .

[0146] Next, the simulation function 157a sends the prediction result as a calculated closed shape model to the calculation function 155a. Also, the second acquisition function 153 sends the acquired correct closed shape model to the calculation function 155a.

[0147] Next, the calculation function 155a calculates a loss value from the correct closed shape model sent from the second acquisition function 153 and the calculated closed shape model sent from the simulation function 157a. The calculation function 155a sends the calculation result to the adjustment function 158.

[0148] Next, the adjustment function 158 updates some of the simulation parameters using an adjustment algorithm similar to the optimization algorithm based on the calculation results sent from the calculation function 155a. For example, the adjustment function 158 adjusts some of the simulation parameters so that the computational closed shape model derived by the calculation of the simulation function 157a approaches the ground truth closed shape model.

[0149] The adjustment function 158 sends the adjusted simulation parameters (adjustment parameters) to the simulation function 157a. Then, the simulation function 157a repeats the process of predicting the shape of the mitral valve in a closed state by simulation based on calculations using the adjustment parameters until the adjustment function 158 determines that the termination condition of the adjustment algorithm is satisfied.

[0150] Next, a process executed by the medical information processing apparatus 100a according to the second embodiment will be described. Fig. 9 is a flowchart showing an example of the process executed by the medical information processing apparatus 100a according to the second embodiment. The processes of steps S201 to S206 and step S208 in Fig. 9 are similar to steps S101 to S106 and step S208 in Fig. 6, and therefore will not be described.

[0151] If the termination condition of the optimization algorithm is satisfied in step S206 (step S206: Yes), the optimization function 156a determines the optimization parameters (step S207).

[0152] For example, when the termination condition of the optimization algorithm is satisfied, the optimization function 156a determines the simulation parameters most recently input to the shape estimation model 121 as the optimization parameters. The optimization function 156a sends the determined optimization parameters to the adjustment function 158. The adjustment function 158 sends the optimization parameters sent from the optimization function 156a to the simulation function 157a as adjustment parameters.

[0153] Next, the simulation function 157a predicts the shape of the mitral valve in the closed state from the open shape model through a simulation using calculations (step S209).

[0154] For example, the simulation function 157a predicts the shape of the mitral valve in a closed state from the open shape model acquired in step S206 by simulation through calculation using the adjustment parameters sent from the adjustment function 158. The simulation function 157a sends the prediction result to the calculation function 155a as a calculated closed shape model.

[0155] Next, the calculation function 155a calculates a loss value (step S210). For example, the calculation function 155a calculates, as a loss value, an average of deviations in node positions of the entire valve between the correct closed shape model acquired in step S202 and the calculated closed shape model sent from the simulation function 157a after step S209. The calculation function 155a sends the calculation result to the adjustment function 158.

[0156] Next, the adjustment function 158 determines whether the termination condition of the adjustment algorithm is satisfied (step S211). For example, if the loss value calculated in step S210 is below a threshold, the adjustment function 158 determines that the termination condition of the adjustment algorithm is satisfied. On the other hand, if the loss value is equal to or greater than the threshold, the adjustment function 158 determines that the termination condition of the adjustment algorithm is not satisfied.

[0157] If the termination condition of the adjustment algorithm is not satisfied (step S211: No), the adjustment function 158 updates the adjustment parameters (step S213). For example, the adjustment function 158 updates the adjustment parameters so that the computational closed shape model sent from the simulation function 157a approaches the correct closed shape model.

[0158] Thereafter, the adjustment function 158 sends the updated adjustment parameters to the simulation function 157a, and the process returns to step S209. In this case, the simulation function 157a predicts the shape of the mitral valve in the closed state from the open shape model acquired in step S101 by simulation through calculation using the adjustment parameters updated in step S212.

[0159] If the termination condition of the adjustment algorithm is satisfied in step S211 (step S211: Yes), the adjustment function 158 finalizes the adjustment parameters (step S212).

[0160] For example, when the adjustment algorithm termination condition is satisfied, the adjustment function 158 determines the adjustment parameters most recently sent to the simulation function 157a as the adjustment parameters. The adjustment function 158 sends the determined adjustment parameters to the simulation function 157a and proceeds to the processing of step S214. Steps S214 and S215 are similar to steps S108 and S109 in FIG. 6, and therefore their explanation will be omitted.

[0161] The medical information processing device 100a according to the second embodiment described above optimizes the simulation parameters for the subject by repeatedly estimating the shape of the mitral valve in a closed state from the shape of the mitral valve in an open state using the shape estimation model 121, and then adjusts the simulation parameters by repeatedly predicting the shape of the mitral valve in a closed state from the shape of the mitral valve in an open state using a computational simulation.

[0162] The shape estimation model 121 is a trained model trained using training data generated based on the results of a computational simulation. Therefore, the accuracy of estimating the shape of a mitral valve in a closed state using the shape estimation model 121 is likely to be lower than the accuracy of predicting a mitral valve in a closed state using a computational simulation. The accuracy of a personalization process using the shape estimation model 121 is highly dependent on the estimation accuracy of the shape estimation model 121, and therefore the accuracy of the personalization may be lower than that of a personalization process using a computational simulation. In contrast, the medical information processing device 100a according to the second embodiment performs a parameter optimization process using the shape estimation model 121 as a first stage, and then performs a parameter adjustment process using a computational simulation as a second stage, as described above. Because the parameter optimization process using the shape estimation model 121 is performed as the first stage, the parameter adjustment process using a computational simulation can be started after personalization has progressed to a certain extent. This allows the parameter adjustment process to be completed in a relatively short time. Furthermore, as described above, the personalization process using the shape estimation model 121 is faster than the personalization process using a computational simulation. Therefore, the medical information processing apparatus 100a according to the second embodiment can perform personalization processing faster than personalization processing that uses only simulation by calculation. Furthermore, the medical information processing apparatus 100a according to the second embodiment performs parameter adjustment processing that uses simulation by second-stage calculation, which can also improve the accuracy of personalization.

[0163] The first and second embodiments described above can be appropriately modified and implemented by partially changing the configuration or functions of each device. Therefore, several modifications of the first and second embodiments described above will be described below as other embodiments. The following mainly describes differences from the first and second embodiments described above, and detailed descriptions of commonalities with the contents already described will be omitted. The modifications described below may be implemented individually or in appropriate combinations.

[0164] (Variation 1) In the first and second embodiments described above, the medical image diagnostic apparatus 30 is an X-ray CT apparatus, but is not limited to this. The medical image diagnostic apparatus 30 may be, for example, an MRI (Magnetic Resonance Imaging) apparatus, an angio-CT system, a tomosynthesis apparatus, a SPECT (Single Photon Emission Computed Tomography) apparatus, a PET (Positron Emission Computed Tomography) apparatus, an ultrasound diagnostic apparatus, or the like.

[0165] According to this modification, it is possible to improve the convenience of the simulation technique for predicting the post-treatment condition based on medical images other than CT images.

[0166] (Variation 2) In the second embodiment described above, a form has been described in which personalization processing is performed, including parameter optimization processing using the shape estimation model 121 and parameter adjustment processing using simulation by calculation. In this modification, a form will be described in which distribution estimation processing is performed with the aim of quantifying uncertainty in the personalization processing.

[0167] First, we will explain common distribution estimation methods. Commonly assumed distribution estimation methods are methods that generate a distribution by obtaining multiple outputs from a single input, such as distribution estimation algorithms such as maximum likelihood estimation and Markov Chain Monte Carlo methods (MCMC), or by preparing multiple machine learning models with different weights and biases.

[0168] One advantage of the above method is that it is less likely to fall into a local solution. A local solution is a solution that is only valid within a specific range. It is generally known that falling into a local solution reduces the speed of personalization processing. Therefore, distribution estimation can be expected to improve the speed of personalization processing.

[0169] However, even if multiple output results are obtained using the above-mentioned method, if the output results are overlapping or similar, the resulting distribution may be narrow. In this case, it becomes more likely to fall into a local solution, which increases the possibility that the personalization process will take a long time or will not progress.

[0170] Here, Fig. 13 is a diagram illustrating an example of a distribution estimation method different from that of the embodiment. Fig. 13 shows an example in which a generally assumed distribution estimation method is applied when performing the personalization process (parameter optimization process and parameter adjustment process) of the second embodiment.

[0171] 13, the horizontal axis represents the simulation parameter (the magnitude of the physical property value), and the vertical axis represents the magnitude of the loss value.

[0172] The circles on the graph represent the relationship between the simulation parameters of the training data generated by the simulation and the loss value. The solid line represents the relationship between the simulation parameters of the predicted results of the simulation and the loss value. The dashed-dotted line represents the relationship between the simulation parameters of the estimated results by the shape estimation model 121 and the loss value.

[0173] The model initial sample represents simulation parameters of multiple output data obtained by providing one input to the shape estimation model 121 using a distribution estimation algorithm, etc. The model optimization post-convergence sample represents simulation parameters at the time when it is determined that the termination condition of the optimization algorithm is satisfied.

[0174] The initial simulation sample represents the simulation parameters at the start of the parameter adjustment process. The post-convergence sample for model optimization represents the simulation parameters at the time when it is determined that the termination condition of the optimization algorithm is satisfied. The post-convergence sample for simulation optimization represents the simulation parameters at the time when it is determined that the termination condition of the adjustment algorithm is satisfied.

[0175] 13, if the parameter adjustment process is started with an inappropriately small variance, a large number of epochs may be required before it is determined that the termination condition of the adjustment algorithm is satisfied (a true solution is reached). In this case, performing personalization using the shape estimation model 121 and a simulation based on calculations may take longer than performing personalization using only a simulation based on calculations.

[0176] In order to solve the above-mentioned problems, a medical information processing apparatus 100a according to Modification 2 realizes efficient distribution estimation. First, the configuration of the medical information processing apparatus 100a will be described with reference to Fig. 10. Fig. 10 is a block diagram showing an example of the configuration of the medical information processing apparatus 100a according to Modification 2.

[0177] The medical information processing device 100a according to this modification has substantially the same configuration as the medical information processing device 100a according to the second embodiment shown in Fig. 7. However, it differs from the medical information processing device 100a according to the second embodiment in that the memory 120 stores a plurality of different types of shape estimation models 121 in order to perform efficient distribution estimation, and in that the processing circuitry 150a has a first determination function 159.

[0178] The first determination function 159 calculates an index that indicates the magnitude of the difference between the output results of each of the multiple machine learning models, and determines whether the index exceeds a threshold value.

[0179] Here, an index representing the magnitude of the difference in the output results is, for example, the variance of the results obtained by inputting multiple pre-prepared simulation parameter sets into a machine learning model, or the variance of the weight and bias values ​​of each of multiple machine learning models.

[0180] For example, the first determination function 159 sets a threshold for the above index and determines whether the threshold is exceeded. The first determination function 159 determines that a machine learning model that exceeds the threshold is a machine learning model with a large difference in output results.

[0181] The first determination function 159 sends the determination result to the learning function 151. Then, the learning function 151 adopts the multiple machine learning models determined by the first determination function 159 to have a large difference in output results as the shape estimation model 121 to be used in the personalization process.

[0182] As an example, the learning function 151 starts learning for multiple machine learning models simultaneously. Then, when the loss values ​​of all the machine learning models fall below the threshold, the first determination function 159 calculates the above index and determines whether it exceeds the threshold.

[0183] The learning function 151 may stop learning of each machine learning model when the loss value falls below the threshold. In this case, the first determination function 159 may calculate the above index after learning of all machine learning models has stopped, and determine whether the index exceeds the threshold.

[0184] For a machine learning model whose index is equal to or less than the threshold, the learning function 151 may increase the number of epochs by a certain number and repeat the learning, or may drop out part of the model and repeat the learning, or may add random noise to the weights and biases and repeat the learning. After that, when the loss falls below the threshold, the first determination function 159 may calculate the index and repeat the determination of whether it exceeds the threshold.

[0185] Furthermore, the learning function 151 may select only machine learning models that exceed the threshold without performing processes such as re-learning. For example, the learning function 151 may set the number of models to be selected in advance, and if there are multiple models that exceed the threshold, select models in descending order of index value. Furthermore, the learning function 151 may perform the above-mentioned re-learning process only if there is no combination of models that exceeds the threshold.

[0186] The optimization function 156a performs parameter optimization processing using a plurality of shape estimation models 121 that are adopted as machine learning models to be used in the parameter optimization processing from a plurality of machine learning models stored as shape estimation models 121. During the parameter optimization processing, the calculation function 155 calculates a loss value for each of the plurality of shape estimation models 121 and sends the average value of the loss values ​​to the optimization function 156.

[0187] For example, the optimization function 156a sets the average value of the loss value falling below a threshold as the termination condition of the optimization algorithm.

[0188] Here, Fig. 11 is a diagram illustrating an example of a distribution estimation method according to Modification 2. Fig. 11 shows an example in which the distribution estimation method according to Modification 2 is applied when performing the personalization process (parameter optimization process and parameter adjustment process) of the second embodiment.

[0189] As shown in FIG. 11, in this modification, multiple shape estimation models 121 are stored in the memory 120, and therefore multiple simulation parameter sets are obtained when it is determined that the termination condition of the optimization algorithm is satisfied ("Sample after model optimization convergence" in FIG. 11).

[0190] Furthermore, as described above, the multiple shape estimation models 121 are machine learning models with large differences in output results, so there is a low possibility that the output results will be duplicated or that only similar output results will be obtained. In other words, at the start of the parameter adjustment process, a sample with an appropriate variance near the true solution can be obtained ("initial simulation sample" in FIG. 11).

[0191] This makes it less likely to fall into a local solution, making it possible to reach a true solution in a smaller number of epochs, and improving the efficiency of the parameter adjustment process.

[0192] The optimization function 156a may determine the average value of a plurality of simulation parameter sets as the optimization parameter. Alternatively, the optimization function 156a may determine the simulation parameter set that has the smallest loss value among the plurality of parameter sets as the optimization parameter.

[0193] The simulation function 157a may also simulate and predict the post-treatment state using multiple simulation parameter sets obtained when the termination condition of the optimization algorithm is satisfied. In this case, the simulation function 157a may obtain a prediction result with an error bar.

[0194] According to this modification, it is possible to carry out efficient distribution estimation.

[0195] (Variation 3) In the above-described first and second embodiments, a configuration has been described in which the shape estimation model 121 is not updated. In this modification, a configuration will be described in which the shape estimation model 121 is updated when the current shape estimation model 121 cannot ensure estimation accuracy for input simulation parameters.

[0196] First, the configuration of the medical information processing device 100 according to this modification will be described. Fig. 12 is a block diagram showing an example of the configuration of the medical information processing device 100 according to Modification 3. As shown in Fig. 12, the medical information processing device 100 according to this modification has substantially the same configuration as the medical information processing device 100 according to the first embodiment shown in Fig. 1 above, but differs from the medical information processing device 100 according to the first embodiment in that the processing circuitry 150 has a second determination function 160.

[0197] The second determination function 160 determines the performance of the shape estimation model 121 in response to input simulation parameters. The determination method is, for example, a method of evaluating the uncertainty of the shape estimation model 121. In this case, the second determination function 160 performs distribution estimation of the shape estimation model 121. The distribution estimation method is, for example, a method using MCMC or the method described in Modification 2.

[0198] The second determination function 160 sets monitoring parameters based on the distribution estimation results obtained. The monitoring parameters are, for example, the variance of the distribution estimation results, the difference between the maximum and minimum values ​​of the obtained results, etc.

[0199] If the monitoring parameter exceeds the threshold, the second determination function 160 determines that the uncertainty of the shape estimation model 121 is high. For convenience of explanation, in the following description, the parameter set when the monitoring parameter exceeds the threshold will also be referred to as the parameter set at evaluation time. If the second determination function 160 determines that the uncertainty of the shape estimation model 121 is high, the learning function 151 updates the shape estimation model 121.

[0200] For example, the second determination function 160 monitors the monitoring parameters every epoch. Alternatively, the second determination function 160 may monitor the monitoring parameters every predetermined number of epochs. Alternatively, the second determination function 160 may exclude the first few epochs of the parameter optimization process from monitoring.

[0201] The following describes the process of updating the shape estimation model 121 by the learning function 151. For example, the learning function 151 generates re-learning data for re-learning by the method described in the first embodiment above.

[0202] For example, the learning function 151 randomly selects simulation parameters to be used when creating re-learning data from a parameter set having a distribution obtained by distribution estimation. Note that the learning function 151 may randomly select from the distribution by adapting a variance set in advance.

[0203] Also, for example, the learning function 151 sets a predetermined number as the number of re-learned data items. Note that the learning function 151 may change the number of re-learned data items depending on the magnitude of the variance.

[0204] Further, for example, instead of generating re-learning data using a computational simulation, the learning function 151 may generate re-learning data using a trained model (hereinafter also referred to as a re-learning model) of a different type from the shape estimation model 121.

[0205] In this case, the updated shape estimation model 121 and the re-training model must differ in at least one of the following: the training method, training data, model architecture, loss function for calculating the loss value, and number of epochs.

[0206] The learning function 151 uses the generated re-learning data to re-learn the shape estimation model 121 using the method described in the first embodiment above.

[0207] After the learning function 151 has completed updating the shape estimation model 121, the optimization function 156 resumes the personalization process. For example, the optimization function 156 uses the parameter set at evaluation time as the initial parameter set at the time of resumption. Note that the optimization function 156 may also use a parameter set that is randomly set again as the initial parameters at the time of resumption.

[0208] According to this modification, it is expected that the accuracy of the personalization process will be improved.

[0209] (Variation 4) In the above-described embodiment and modified examples, a computational simulation is performed to predict the post-treatment state of the mitral valve after MitraClip treatment. However, computational simulations to predict the post-treatment state are not limited to this. For example, this method can be applied to any treatment for the mitral valve other than MitraClip treatment.

[0210] Furthermore, in the above-described embodiment and modified examples, a simulation was described in which the shape of the mitral valve in the open state was used as input to predict the shape of the mitral valve in the closed state, but this method can also be applied to simulations other than those described above.

[0211] For example, this method can be applied to simulations that predict the blood flow velocity distribution during diastole from the blood flow velocity distribution near the mitral valve during systole, simulations that predict the shape of the lungs during inspiration from the shape of the lungs during expiration, and simulations that predict the pressure distribution during diastole from the intravascular pressure distribution near a cerebral aneurysm during systole.

[0212] According to this modification, it is possible to improve the convenience of the simulation technique for predicting the post-treatment state for various treatments for various organs.

[0213] According to at least the embodiment and modifications described above, it is possible to improve the convenience of the simulation technique for predicting the post-treatment state.

[0214] The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)).

[0215] The processor realizes its functions by reading and executing the programs stored in the memory 120. Note that instead of storing the programs in the memory 120, the programs may be directly embedded in the circuitry of the processor. In this case, the processor realizes its functions by reading and executing the programs embedded in the circuitry.

[0216] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0217] 100 Medical information processing device 150 Processing Circuit 151 Learning Function 152 First Acquisition Function 153 Second Acquisition Function 154 Estimation Function 155, 155a Calculation function 156, 156a Optimization Function 157, 157a Simulation function 158 Adjustment function 159 1st judgment function 160 Second judgment function

Claims

1. a first acquisition unit that acquires first medical information based on a first shape of a target organ that is a treatment target of a subject at a predetermined first time phase of the target organ; a second acquisition unit that acquires second medical information based on a second shape of the target organ in a second time phase different from the first time phase of the target organ; an estimation unit that repeatedly performs calculations based on the first medical information, first simulation parameters representing initial parameters of simulation parameters used in a simulation process for the target organ, and a learned model that has learned, by machine learning technology, a relationship between the first shape and the second shape predicted from the first shape by calculation using the simulation parameters, and repeatedly acquires third medical information based on the second shape estimated from the first medical information; an optimization unit that identifies second simulation parameters obtained by optimizing the first simulation parameters for the subject, based on the third medical information repeatedly acquired by the estimation unit and a first index indicating a difference between the second medical information and the third medical information; A medical information processing device comprising:

2. the optimization unit repeatedly changes the first simulation parameter based on the first index so as to reduce the first index, causes the estimation unit to repeatedly perform calculations using the repeatedly changed first simulation parameter, and identifies the repeatedly changed first simulation parameter at a point in time when the first index becomes less than a threshold as the second simulation parameter. The medical information processing device according to claim 1 .

3. a prediction unit that repeatedly performs calculations based on the first medical information, the identified second simulation parameters, and calculations that predict the second shape from the first shape, and repeatedly acquires fourth medical information based on the second shape predicted from the first medical information; an adjustment unit that specifies third simulation parameters obtained by adjusting the second simulation parameters based on the fourth medical information repeatedly acquired by the prediction unit and a second index indicating a difference between the second medical information and the fourth medical information; and Further comprising: The medical information processing device according to claim 1 .

4. the prediction unit predicts the state of the target organ after treatment by calculation using the second simulation parameters or the third simulation parameters. The medical information processing device according to claim 3 .

5. A calculation unit is further provided that inputs a plurality of simulation parameters to a plurality of the trained models and calculates a third index representing the magnitude of a difference between output results for each of the plurality of trained models, the estimation unit acquires the third medical information based on the trained model in which the third index exceeds a threshold. The medical information processing device according to claim 1 .

6. an evaluation unit that evaluates the uncertainty of the trained model with respect to the input of the simulation parameters; a re-learning unit that re-learns the trained model when the uncertainty is evaluated to be high; Further provided with When the uncertainty is evaluated to be high, the optimization unit stops the process of identifying the second simulation parameters, and resumes the process of identifying the second simulation parameters after re-learning of the trained model is completed. The medical information processing device according to claim 1 .

7. The simulation parameters include at least one of the shapes of the anterior and posterior leaflets, the Young's modulus of the anterior and posterior leaflets, the time change of the pressure applied to the valve, the natural length of the chordae tendineae, the position of the chordae tendineae, the viscosity coefficient, the bulk modulus, the fiber direction of the valve, and the residual stress. The medical information processing device according to any one of claims 1 to 6.

8. the target organ is a mitral valve, the first phase is mid-diastole of the heart; the first shape is that of an open mitral valve; the second phase is an early systole of the heart; the second shape is a closed mitral valve shape; The simulation parameters are parameters used in a simulation process for predicting the state of the mitral valve after Edge to Edge Repair treatment, which increases the coaptation area by grasping the anterior leaflet and the posterior leaflet of the mitral valve. The medical information processing device according to any one of claims 1 to 6.

9. A medical information processing method by a medical information processing device, comprising: a first acquisition step of acquiring first medical information based on a first shape of a target organ to be treated in a subject at a predetermined first time phase of the target organ; a second acquisition step of acquiring second medical information based on a second shape of the target organ in a second time phase different from the first time phase of the target organ; an estimation step of repeatedly performing calculations based on the first medical information, first simulation parameters representing initial parameters of simulation parameters used in a simulation process for the target organ, and a learned model that has learned, by machine learning technology, a relationship between the first shape and the second shape predicted from the first shape by calculation using the simulation parameters, to repeatedly acquire third medical information based on the second shape estimated from the first medical information; an optimization unit that identifies second simulation parameters that optimize the first simulation parameters for the subject based on the third medical information repeatedly acquired in the estimation step and a first index indicating a difference between the second medical information and the third medical information; a prediction step that repeatedly performs calculations based on the first medical information, the identified second simulation parameters, and calculations that predict the second shape from the first shape, and repeatedly acquires fourth medical information based on the second shape predicted from the first medical information; an adjusting step of specifying third simulation parameters obtained by adjusting the second simulation parameters based on the fourth medical information repeatedly acquired in the predicting step and a second index indicating a difference between the second medical information and the fourth medical information; A medical information processing method including:

Images