Medical data processing device, medical data processing method, and medical data processing program

Abstract

To improve model inference accuracy.SOLUTION: A medical data processing device according to an embodiment comprises an acquisition part and an execution part. The acquisition part acquires medical data. The execution part generates output data by using a synthetic function comprising one or more functions for performing arithmetic processing of a fixed coefficient associated with attentional information in the medical data, for the medical data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical data processing device, a medical data processing method, and a medical data processing program. [Background technology]

[0002] There are methods for estimating medical classification based on medical data, such as using a trained model to classify tumor tissues and estimate their grades for MR (Magnetic Resonance) images and MRS (Magnetic Resonance Spectroscopy) spectral data, etc. However, there is a problem with the inference of the trained model used in this method in that the trained model does not use information about regions of interest or important regions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-41978 [Patent Document 2] JP 2019-154854 A [Non-patent literature]

[0004] [Non-Patent Document 1] Hidenori Takeshima, Deep Learning and Its Application to Function Approximation for MR in Medicine: An Overview, Magn Reson Med Sci, doi:10.2463 / mrms.rev.2021-004, Published Online: September 17, 2021 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the inference accuracy of a model. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] A medical data processing device according to an embodiment includes an acquisition unit and an execution unit. The acquisition unit acquires medical data. The execution unit generates output data by using a composite function for the medical data, the composite function including one or more functions for performing a calculation process of a fixed coefficient associated with information of interest in the medical data. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a medical data processing device according to this embodiment. [Diagram 2] FIG. 2 is a flowchart showing an example of the operation of the medical data processing device according to this embodiment. [Diagram 3] FIG. 3 is a diagram showing the input / output relationship of the trained model according to this embodiment. [Figure 4] FIG. 4 is a conceptual diagram showing an MR spectrum, which is an example of medical data. [Diagram 5] FIG. 5 is a diagram showing a first example of a model including a layer of interest according to this embodiment. [Figure 6] FIG. 6 is a diagram showing a second example of a model including a layer of interest according to this embodiment. [Figure 7] FIG. 7 is a diagram showing a third example of a model including a layer of interest according to this embodiment. [Figure 8] FIG. 8 is a diagram showing a fourth example of a model including a layer of interest according to this embodiment. [Figure 9] FIG. 9 is a diagram showing a fifth example of a model including a layer of interest according to this embodiment. [Figure 10]FIG. 10 is a diagram showing a fifth example of a model including a layer of interest according to this embodiment. [Figure 11] FIG. 11 is a diagram showing a sixth example of a model including a layer of interest according to this embodiment. [Figure 12] FIG. 12 is a diagram showing a sixth example of a model including a layer of interest according to this embodiment. [Figure 13] FIG. 13 is a diagram showing a seventh example of a model including a layer of interest according to this embodiment. [Figure 14] FIG. 14 is a diagram showing an eighth example of a model including a layer of interest according to this embodiment. [Figure 15] FIG. 15 is a diagram showing a ninth example of a model including a layer of interest according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of a medical data processing device, a medical data processing method, and a medical data processing program will be described in detail with reference to the drawings. In the following embodiments, parts with the same reference numerals perform the same operations, and duplicated descriptions will be omitted as appropriate. Hereinafter, one embodiment will be described with reference to the drawings.

[0009] The medical data processing device according to this embodiment will be described with reference to the block diagram of FIG. As shown in FIG. 1, the medical data processing device 1 is a computer having, for example, a processing circuit 10, a memory 11, an input interface 12, a communication interface 13, and a display 14. The medical data processing device 1 may be mounted on a console of various medical image diagnostic devices such as an MRI (Magnetic Resonance Imaging) device or an X-ray CT (Computed Tomography) device, or may be mounted on a workstation. Alternatively, it may exist independently of the medical image diagnostic device and the workstation. The medical data processing device 1 may be realized without including at least one other configuration among the memory 11, the input interface 12, the communication interface 13, and the display 14. The medical data processing device 1 may be realized, for example, by the processing circuit 10 alone.

[0010] The processing circuit 10 includes a processor as a hardware resource and functions as the core of the medical data processing device 1.

[0011] The processing circuit 10 acquires various medical data by the acquisition function 101. Examples of various medical data include medical images such as MR data, such as MR images and MR spectra by MRS (Magnetic Resonance Spectroscopy), CT images, X-ray images, ultrasound images, PET (Positron Emission Tomography) images, and SPECT (Single Photon Emission Computed Tomography) images, as well as intermediate data thereof, such as k-space data, projection data, and sinogram data. The medical data may also be data related to other examinations, such as vital data and biopsy data. The medical data may also be time-series data, one-dimensional data such as waveform data and examination data, two-dimensional data such as image data, and three-dimensional data such as volume data or data having more dimensions. The processing circuit 10 may acquire medical data directly from a medical image diagnostic device by the acquisition function 101, or may acquire medical data once stored in the memory 11 from the memory 11.

[0012] The processing circuit 10 sets, by the setting function 102, one or more functions for performing arithmetic processing of fixed coefficients associated with the information of interest in the medical data for the composite function. The information of interest is information indicating items or areas that are noteworthy or important in the medical data. In other words, the information of interest is information that affects the output from the composite function or information that is identified in the training of the composite function as having an effect on the output from the composite function. The fixed coefficients are real values. In other words, the fixed coefficients are data related to the medical data and are values ​​based on data related to medical physics or magnetic resonance physics. The composite function is, for example, a model used in machine learning such as a neural network. Hereinafter, the composite function will be described as being a trained model.

[0013] The processing circuit 10 generates output data by using the set synthesis function through the model execution function 103. The output data is, for example, a medical classification result related to medical data. The medical classification result is, for example, information related to the determination of the presence or absence of a tumor or the grading of a tumor. Note that the output data is not limited to the medical classification result, and may be clinical decision support (CDS) such as segmentation, prognosis estimation, and treatment proposal.

[0014] The display control function 104 causes the processing circuitry 10 to display various information on the display 14. For example, medical data and medical classification results generated by the model execution function 103 are displayed on the display 14.

[0015] The model training function 105 causes the processing circuit 10 to train a machine learning model and calculate fixed coefficients related to the attention information.

[0016] The memory 11 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an integrated circuit storage device that stores various information. The memory 11 may also be a drive device that reads and writes various information from and to a portable storage medium such as a CD-ROM drive, a DVD drive, or a flash memory. For example, the memory 11 stores medical data (spectrum data, image data, etc.), output data (medical classification results, etc.), control programs, etc.

[0017] The input interface 12 includes an input device that accepts various commands from a user. Examples of the input device that can be used include a keyboard, a mouse, various switches, a touch screen, a touch pad, and the like. Note that the input device is not limited to one that has physical operation parts such as a mouse and a keyboard. For example, an example of the input interface 12 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the medical data processing device 1 and outputs the received electrical signal to various circuits. The input interface 12 may also be a voice recognition device that converts a voice signal collected by a microphone into an instruction signal.

[0018] The communication interface 13 is an interface that connects the medical data processing device 1 to a medical image diagnostic device, a workstation, a PACS (Picture Archiving and Communication System), a HIS (Hospital Information System), a RIS (Radiology Information System), etc. via a LAN (Local Area Network) or the like.

[0019] The display 14 displays various information using a display control function 106. As the display 14, for example, a CRT display, a liquid crystal display, an organic EL display, an LED display, a plasma display, or any other display known in the art can be appropriately used.

[0020] The above configuration is an example, and is not limited thereto. For example, the storage area of ​​the data collection conditions in the memory 11 does not need to be implemented in the medical data processing device 1, and may be implemented in a storage device connected to the medical data processing device 1 via a network. Furthermore, each function of the processing circuit 10 may be realized across multiple devices. For example, the acquisition function 101, the setting function 102, the model execution function 103, and the display control function 104 may be incorporated into and executed in the console of the medical image diagnostic device, and the model training function 105 may be executed in a workstation. As long as each function of the medical data processing device 1 can be realized in cooperation with multiple devices, it does not matter which device executes each process.

[0021] An example of the operation of the medical data processing device 1 according to this embodiment will be described in detail below with reference to the flowchart in FIG.

[0022] In step SA1, the processing circuit 10 acquires medical data by the acquisition function 101. In step SA2, the processing circuit 10 sets fixed coefficients associated with the attention information according to the medical data and the task using the setting function 102. The setting of the fixed coefficients may be realized, for example, by incorporating an attention layer, which is a layer structure of a neural network in which fixed coefficients related to the attention information are set as parameters, into the trained model.

[0023] In step SA3, the processing circuit 10 applies the trained model incorporating the attention layer to the medical data by the model execution function 103, and the trained model generates output data. In other words, a calculation process of the fixed coefficient related to the attention layer is performed on the medical data. Here, medical classification is performed on the medical data to obtain a medical classification result. In step SS4, the processing circuitry 10 outputs the medical classification results, for example via the display control function 104, to the display 14 or the like.

[0024] In addition, in the process of step SA2, if a layer of interest has already been incorporated into the trained model, it is not necessary to set a new layer of interest in the trained model.

[0025] Next, the input / output relationship of the trained model according to this embodiment will be described with reference to FIG. As shown in Fig. 3, the trained model 30 is a model in which parameters (e.g., weights and biases) are trained so as to input medical data 31 and output a medical classification result 32. Here, the trained model 30 is assumed to be a deep convolutional neural network having a convolutional layer such as Transformer, ResNet, or DenseNet. Note that the trained model 30 is not limited to a deep convolutional neural network, and any model may be used as long as it is a composite function (model) used in the field of machine learning.

[0026] The trained model 30 includes an input layer, an intermediate layer 30-2, and an output layer 30-3. Here, the input layer corresponds to the attention layer 30-1. The attention layer 30-1 is a layer that performs arithmetic processing of fixed coefficients related to attention information, such as molecular information of interest, an area of ​​interest, etc. The attention layer 30-1 is formed, for example, of a fully connected layer (FC), and the medical data 31 is multiplied by fixed coefficients. In other words, the attention layer 30-1 behaves so as to execute processing dependent on the position information of the medical data 31.

[0027] On the other hand, it is assumed that multiple convolutional layers are arranged in the intermediate layer 30-2. Therefore, in the intermediate layer 30-2, if the receptive field in the convolutional layer is taken into consideration, the deeper the network is, the more independent the processing that is executed is of position information. The output layer 30-3 outputs a medical classification result, for example a probability value for the classification task.

[0028] Next, an MR spectrum, which is an example of medical data according to this embodiment, will be described with reference to the conceptual diagram of FIG. The MR spectrum shown in FIG. 4 is a spectrum based on data obtained by MRS, which is a type of chemical shift measurement. MRS is a data acquisition method for measuring a spectrum of chemical shifts. Note that, in addition to MRS, any measurement method such as CSI (chemical shift imaging), CEST (chemical exchange spectroscopy), ZAPPED (Z-spectrum analysis provides proton environment data), and diffusion weighted MRS may be used as medical data according to this embodiment. Diffusion weighted MRS is MRS by a pulse sequence with a diffusion weighting gradient magnetic field (MPG: Motion Probing Gradient), and the spectrum obtained is obtained by enhancing the diffusion of hydrogen nuclei in molecules by applying MPG.

[0029] Specifically, the method of generating the spectral data is to synthesize k-space data for the number of integration times collected based on the pulse sequence of MRS by point resolved spectroscopy (PRESS), stimulated echo acquisition mode (STEAM), Image-Selected In vivo Spectroscopy (ISIS), Spin Echo Full Intensity Acquired Localized (SPECIAL), semi-localization by adiabatic selective refocusing (semi-LASER), LASER, etc. The synthesized k-space data is subjected to Fourier transformation to convert the MR signal intensity value into digital data that expresses the MR signal intensity value as a frequency function, and post-processing such as phase correction and baseline correction is performed on the converted digital data to generate the MRS spectral data.

[0030] In the MR spectrum shown in FIG. 4, the vertical axis is defined as the MR signal intensity value [AU (Arbitrary Unit)], and the horizontal axis is defined as the difference from the reference frequency, i.e., the chemical shift [ppm (parts per million)]. The reference frequency is set to the frequency of an arbitrarily selected reference material. The reference material is not particularly limited, but is set to, for example, tetramethylsilane (TMS). According to the MR spectrum, the abundance of molecules such as N-acetylaspartic acid (NAA), creatine (Cr), and choline (Cho) can be visualized.

[0031] Next, a first example of a model including a layer of interest according to this embodiment will be described with reference to FIG. When the molecule of interest is unknown, the processing circuit 10 can generate a learned model 30 by training the machine learning model 50 using the model training function 105.

[0032] As training data, a combination of medical data (here, as an example, MRS data 41) as input data and medical classification results 42 collected from past cases as correct answer data may be used. The MRS data 41 as input data is generated, for example, by an MRI device. The medical classification results 42 as correct answer data may be classification results determined by a medical professional using the MRS data 41 and medical information such as biopsy data and MR images related to the same subject as the MRS data 41. The grade may be determined by a computer according to a predetermined algorithm based on the input data or medical information.

[0033] The setting function 102 causes the processing circuit 10 to incorporate an attention layer 51 for the machine learning model to be trained. The attention layer 51 may be set to include the number of channels 52 corresponding to the type of attention information. Specifically, for example, if there are three types of molecules that are attention information, the number of channels 52 in the attention layer 51 may be set to three.

[0034] Thereafter, the processing circuit 10 applies the machine learning model 50 to the MRS data 41, which is the input data, by the model training function 105, performs forward propagation processing, and outputs a medical classification result (hereinafter, referred to as an estimated medical classification result). Next, the processing circuit 10 uses the model training function 105 to calculate the difference (error) between the estimated medical classification result and the medical classification result 42, which is the correct answer data, by an error function, and updates the parameters of the machine learning model 50 by the error backpropagation method and the stochastic gradient descent method so that the error is minimized. The error function may be, for example, MSE (Mean Squared Error), cross entropy, or another function to be used depending on the task of the machine learning model.

[0035] The processing circuit 10 uses the model training function 105 to determine whether the training has been completed. For example, the determination of whether the training has been completed may be made by determining that the training has been completed when the error value calculated by the error function is equal to or less than a threshold value. Alternatively, the training may be determined to have been completed when the reduction in the error value converges within a predetermined value. Furthermore, the training may be determined to have been completed when a predetermined number of epochs of training have been completed. When the training has been completed, the trained model 30 is generated by assigning the determined parameters to the machine learning model.

[0036] This makes it possible to calculate fixed coefficients for each channel of the attention layer 51 corresponding to the attention information. After that, processing may be executed to output a medical classification result for the new MRS data 41. Next, a second example of a model including a layer of interest according to this embodiment will be described with reference to FIG. 6, the trained model 30 is a model in which parameters are trained so that MRS data 41 indicating an MR spectrum is input as medical data and a medical classification result 42 indicating tissue properties of a region of interest is output. The MRS data 41 is assumed to be, for example, a spectrum as shown in FIG. 4, or intermediate data leading up to the generation of an MR spectrum.

[0037] In the layer of interest 30-1 according to the present embodiment, a layer is introduced that multiplies the attention information, here a fixed coefficient related to the single spectrum of the molecule of interest in the MRS. That is, it is assumed that the fixed coefficient is a spectrum corresponding to the base spectrum or the chemical shift of the molecule. The single spectrum may be data artificially generated by simulation data 43-1 or machine learning, or data obtained by imaging a phantom, or rectangular waveform data 43-2 corresponding to the chemical shift value for each peak of those spectra, or data corresponding to the chemical shift value for a specific peak. By inputting the MRS data 41 into the layer of interest 30-1, a region corresponding to the data related to the spectrum of the molecule of interest is processed while being weighted in the subsequent intermediate layer 30-2 (one or more convolution layers). That is, it can be said that the convolution process is performed while considering whether or not the shape of the single spectrum of the molecule of interest is included in the MR spectrum.

[0038] The medical classification result 42 is output from the output layer 30-3 of the trained model 30, and for example, the presence or absence of a tumor in the subject P is output. Specifically, in the example of FIG. 6, "tumor" is output as the medical classification result 42. The output format is not particularly limited, but a multi-class classification may be used in which, in addition to the presence or absence of a tumor, a grade with a high probability that the subject P belongs to among multiple grades is output as a value close to "1" and a grade with a low probability is output as a value close to "0". Furthermore, the presence or absence of cancer metastasis or the probability of being a case may be output as the medical classification result 42.

[0039] In addition, the presence or absence of IDH1 mutation (isocitrate dehydrogenase-1 mutation) may be output as the medical classification result 42. Mutant IDH1 is known to produce 2-HG (2-hydroxyglutarate) (for example, Jelena Lazovic et al., "Detection of 2-hydroxyglutaric acid in vivo by proton magnetic resonance spectroscopy in U87 glioma cells overexpressing isocitrate dehydrogenase-1 mutation" Neuro-Oncology 14(12):1465-1472,2012). In other words, a correlation is observed between the presence of 2-HG and IDH1 mutation.

[0040] 2-HG can be detected by MRS and can form a peak in the MR spectrum. 2-HG can also be depicted in MR morphological images such as FLAIR images and T2 weighted images. By setting fixed coefficients for the spectrum related to 2-HG in the layer of interest 30-1, it is also possible to output a medical classification result 42 regarding the presence or absence of an IDH1 mutation and a grade indicating the degree of the mutation.

[0041] The MRS data input to the trained model 30 is not limited to the waveform data of the MR spectrum shown in FIG. 6. For example, the MRS data may be a combination of numerical data of a chemical shift value and an MR signal intensity value (peak value) for each of a plurality of peaks of the MR spectrum, or a combination of numerical data of a chemical shift value and an MR signal intensity value (peak value) peak width (half width). The MRS data may be a combination of numerical data of a molecular identifier (name or symbol) and an MR signal intensity value for each of a plurality of peaks of the MR spectrum, or a combination of numerical data of a molecular identifier, an MR signal intensity value, and a half width, or an estimated molecular concentration value or its relative value. Any intermediate product obtained in the process of generating the MRS data may be input to the trained model 30. For example, k-space data before addition, k-space data after addition, and k-space data after Fourier transformation may be input to the trained model 30. These MR spectra and intermediate products of the MR spectra are examples of MRS data.

[0042] The fixed coefficients of the layer of interest 30-1 in cases other than the waveform data of the MR spectrum may be set according to the data format input to the layer of interest 30-1. For example, if the MRS data is a combination of a chemical shift value and an MR signal intensity value, values ​​related to the chemical shift value and the MR signal intensity value of the molecule of interest, which is the information of interest, may be set as fixed coefficients in the layer of interest 30-1. In this way, fixed coefficients may be set in the same way even when the MRS data is in another data format.

[0043] A third example of a model including a layer of interest according to this embodiment will be described with reference to FIG. The model shown in FIG. 7 is different from FIG. 6 in the configuration of the attention layer 61. The attention layer 61 shown in FIG. 7 is assumed to be set in a case where there are multiple molecules of interest that are attention information. For example, a layer may be introduced in which a composite spectrum 62-1 of the attention molecule is generated by synthesizing single spectra of multiple molecules, and a fixed coefficient related to the composite spectrum 62-1 is multiplied. Alternatively, a fixed coefficient corresponding to each single spectrum may be set as a channel 62-2. In the attention layer 61, the sum of products of the MRS data 41 and the channel 62-2 is calculated, so that fixed coefficients related to multiple single spectra can be used in the attention layer, as in the composite spectrum 62-1.

[0044] Next, a fourth example of a model including a layer of interest according to this embodiment will be described with reference to FIG. FIG. 8 shows an example in which both the case where the attention information is known and the case where the attention information is unknown are combined. That is, the machine learning model 50 includes an attention layer 30-1 related to the known attention information and an attention layer 51 related to the unknown attention information. In the example of FIG. 8, the machine learning model 50 including the attention layer 30-1 and the attention layer 51 is trained to calculate fixed coefficients corresponding to a single spectrum of a known molecule and fixed coefficients corresponding to channels of the single spectrum of one or more unknown molecules. The machine learning model 50 may be trained by a method similar to that shown in FIG. 5. This makes it possible to generate fixed coefficients corresponding to single spectra for unknown molecules other than known molecules.

[0045] Next, a fifth example of a model including a layer of interest according to this embodiment will be described with reference to FIGS. In the examples of Figures 5 to 8, an example is shown in which the attention layer 30-1 is incorporated at the position of the input layer of the trained model 30 or the machine learning model 50, but this is not limited to the above, and the attention layer 30-1 may be incorporated at any position in the trained model 30 or the machine learning model 50 other than the position of the output layer 30-3.

[0046] Fig. 9 shows an example in which the layer of attention 30-1 is arranged after the input layer 80 in the trained model 30. Fig. 10 shows an example in which the layer of attention 30-1 is arranged at the end of the intermediate layers, i.e., before the output layer 30-3. In this manner, the layer of attention 30-1 may be arranged in any position. Note that the fixed coefficients of the layer of attention 30-1 may be adjusted according to the output size from the previous layer (the input layer 80 or the intermediate layer 30-2) since the size of the data input to the layer of attention 30-1 differs depending on the depth of the network.

[0047] Next, a sixth example of a model including a layer of interest according to this embodiment will be described with reference to FIGS. In FIG. 11, two-dimensional data, that is, medical images, are assumed as input medical data. For example, fixed coefficients corresponding to mask data, weight data, segmentation data, and annotation data, which are data related to medical physics, are set in the attention layer 30-1 related to the medical image. In the example of FIG. 11, the trained model 30 is applied to an MR image 91, and for example, fixed coefficients corresponding to mask data 92 in which a tumor part is a region of interest (ROI: Region of Interest) are set in the attention layer 30-1. Note that, as the MR image, a T1 weighted image, a T2 weighted image, a T2* weighted image, a FLAIR (fluid attenuated inversion recovery) image, an MR angiography, a diffusion weighted image, or an image collected by another imaging method are assumed. As a result, it can be said that inference is performed on the MR image 91 with the region set by the mask data as the center.

[0048] The fixed coefficients of the layer of interest 30-1 related to the tumor region may be set to correspond to a mask region manually generated for the input MR image, or mask data may be generated from annotated segmentation data using U-net or the like, and the corresponding fixed coefficients may be obtained. The fixed coefficients may be expressed, for example, as weights in which the region of the ROI that is the information of interest is 1 and other regions are 0. The fixed coefficients may not be set with a binary weight of 0 or 1, but may be set by assigning values ​​between 0 and 1 as weights to multiple tissues. For example, in the case of a mask region targeting a tumor, the fixed coefficients may be set by assigning weights of "0.1" to the parenchyma, "0.8" to the edema, and "1" to the entire tumor.

[0049] 12 shows an example in which the attention layer 30-1 has fixed coefficients corresponding to mask data on gray matter, in other words, segmentation data, as parameters. This means that when the trained model 30 is applied to the MR image 91, inference is performed centered on the area set by the mask data. As a medical classification result 42, for example, classification as Alzheimer's dementia is performed.

[0050] The fixed coefficients of the layer of interest 30-1 for the gray matter in Figure 12 may be set to correspond to a mask region manually generated for the input MR image, or mask data may be generated from segmentation data for the gray matter using U-net or the like, based on signal intensity and position information corresponding to the gray matter, and the corresponding fixed coefficients may be obtained.

[0051] In addition, the layer of interest 30-1 may be set as mask data for tissues other than the gray matter mask, such as white matter, cerebrospinal fluid (CSF), etc. Furthermore, the layer of interest 30-1 may be set as mask data by combining regions of a plurality of tissues, such as by using regions of gray matter and CSF as mask data, and the corresponding fixed coefficients may be set as the layer of interest 30-1.

[0052] Next, a seventh example of a model including a layer of interest according to this embodiment will be described with reference to FIG. As in the case where the medical data is MRS data, when the mask region that is the information of interest is unknown, the processing circuit 10 can generate a learned model 30 by training the machine learning model 50 using the model training function 105.

[0053] As training data, a combination of MR images 91 as input data and medical classification results 42 collected from past cases as correct answer data may be used. The MR images 91 as input data are generated by, for example, an MRI device. The medical classification results 42 as correct answer data may be a classification result determined by a medical professional using the MR image 91 or medical information such as biopsy data and MR images related to the same subject as the MR image 91. The grade may be determined by a computer based on the input data or medical information in accordance with a predetermined algorithm.

[0054] The setting function 102 causes the processing circuit 10 to incorporate an attention layer 51 for the machine learning model to be trained. The attention layer 51 may be set to include the number of channels 52 corresponding to the type of attention information. Specifically, for example, if there are three types of mask regions that are attention information, the number of channels 52 in the attention layer 51 may be set to three.

[0055] Thereafter, the processing circuit 10 applies the machine learning model 50 to the MR image 91 as input data by the model training function 105, performs forward propagation processing, and outputs an estimated medical classification result, as in the case shown in Fig. 5. Next, the processing circuit 10 uses the model training function 105 to calculate the difference (error) between the estimated medical classification result and the medical classification result 42, which is the correct answer data, by an error function, and updates the parameters of the machine learning model 50 by the error backpropagation method and the stochastic gradient descent method so as to minimize the error. When the training is completed, the trained model 30 is generated.

[0056] This makes it possible to calculate fixed coefficients for each channel of the attention layer 51 corresponding to the attention information. After that, processing may be executed to output a medical classification result for the new MRS data 41.

[0057] Note that this is not limited to the MR image 91 collected by the MRI device, but can also be realized with medical images collected by other medical image diagnostic devices. For example, even when a CT image taken by an X-ray CT device, or a material decomposition image taken by a dual energy CT device or a photon counting CT device is input, the fixed coefficients of the layer of interest 30-1 may be set by, for example, mask data 92 of the region of interest.

[0058] Next, an eighth example of the model including the target layer according to this embodiment will be described with reference to FIG. In FIG. 14, three-dimensional data of ultrasound data, that is, volume data 1401, is assumed as input medical data. The attention layer 30-1 for the volume data 1401 includes fixed coefficients corresponding to, for example, mask data, segmentation data, and annotation data for the volume data as parameters. In the example of FIG. 13, the trained model 30 is applied to the volume data 1401. In the attention layer 30-1 of the trained model 30, for example, mask data 1402 for volume data having a tumor portion as a region of interest (voxel of interest) is set. As a result, inference is performed for the volume data 1401 with the region set by the mask data as the center, similar to the case of two-dimensional images as shown in FIG. 11 and FIG. 12.

[0059] As described above, any of the first to fifth examples of the model shown in Fig. 3 to Fig. 10 can be applied to any medical data. That is, for example, for medical images such as MR images 91 and volume data 1401, a model may be trained by combining the attention layer 30-1 related to known attention information and the attention layer 51 related to unknown attention information, and the position where the attention layer 30-1 is incorporated in the model may be freely changed.

[0060] In the above example, the fixed coefficients are assumed to be mask data and segmentation data related to the molecule of interest and the region of interest, but are not limited thereto and may be values ​​based on parameters related to imaging of the medical image diagnostic device that collected the medical data. For example, if the medical image diagnostic device is an MRI device, the fixed coefficients may be values ​​corresponding to data related to magnetic field inhomogeneity related to magnetic resonance physics, and transmission sensitivity data and reception sensitivity data of the coil in the MRI device.

[0061] Next, a ninth example of the model including the target layer according to this embodiment will be described with reference to FIG. As shown in FIG. 15, the layer configuration of the trained model excluding the output layer 30-3 of the trained model can be used for methods other than classification tasks. For example, a processing layer (e.g., a decoder) that performs upsampling can be arranged as the output layer, or a network such as U-net can be used to perform image reconstruction using a layer configuration including the attention layer 30-1. Alternatively, it may be used for denoising, segmentation, super-resolution processing, etc. for images. That is, by switching the configuration of the output layer 30-3 for the trained model including the attention layer 30-1 according to this embodiment, that is, by switching the so-called head, it becomes possible to apply it to multiple tasks, and processing for the desired task can be performed.

[0062] According to the present embodiment described above, a synthetic function such as a machine learning model or a trained model having one or more functions for performing arithmetic processing of fixed coefficients associated with information of interest on medical data is used to generate inference results such as medical result classification as output data. As a result, for example, in inference using a trained model, a fixed coefficient related to an important region is multiplied to medical data, and information related to the important region is incorporated, and inference focused on the important region is performed. In other words, the accuracy of the model can be improved.

[0063] According to at least one of the embodiments described above, it is possible to improve the accuracy of medical classification using medical data.

[0064] The term "processor" used in the above description means a circuit such as a CPU, a GPU, or an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). The processor realizes a function by reading and executing a program stored in a memory circuit. Instead of storing a program in a memory circuit, the processor may be configured to directly incorporate the program in the circuit. In this case, the processor realizes a function by reading and executing a program incorporated in the circuit. Instead of executing a program, a function corresponding to the program may be realized by a combination of logic circuits. Each processor in this embodiment is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, multiple components may be integrated into a single processor to realize its function.

[0065] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0066] 1 Medical data processing device 10 Processing circuit 11. Memory 12 Input Interface 13 Communication Interface 14 Display 30 Pre-trained models 30-1 Attention demographic 30-2 Middle class 30-3 Output layer 31 Medical Data 32,42 Medical classification results 41 MRS data 43-1 Simulation data 43-2 Waveform data 50 Machine Learning Models 51,61 Popular demographic 51-1 Synthetic spectrum 52,62-2 Channel 62-1 Synthetic Spectrum 80 Input Layer 91 MR Imaging 92,1402 Mask Data 101 Acquisition Function 102 Setting Function 103 Model execution function 104 Display control function 105 Model training function 106 Display control function 1401 Volume Data

Claims

1. An acquisition unit for acquiring medical data; an execution unit that generates output data by using a synthesis function including one or more functions that perform a calculation process of a fixed coefficient associated with information of interest in the medical data; A medical data processing device comprising:

2. The medical data processing device according to claim 1 , wherein the information of interest is information in the medical data that affects an output from the composite function or information identified in training of the composite function as having an effect on the output from the composite function.

3. The medical data processing apparatus according to claim 1 , wherein the synthesis function multiplies the medical data by the fixed coefficient as the arithmetic processing.

4. The medical data processing apparatus according to claim 1 , wherein the fixed coefficient is data relating to the medical data and is a value based on data relating to medical physics or magnetic resonance physics.

5. 2. The medical data processing device according to claim 1, wherein the medical data is at least one of MR (Magnetic Resonance) data, CT (Computed Tomography) data, ultrasound data, PET data, SPECT data, vital data, and biopsy data.

6. The medical data processing device according to claim 1 , wherein the medical data is at least one of time series data, image data, waveform data, examination data, and three- or more dimensional data.

7. The medical data is MRS data, The medical data processing device according to claim 1 , wherein the fixed coefficient is a basis spectrum or a spectrum corresponding to a chemical shift of a molecule.

8. The medical data is MRS data, The medical data processing device according to claim 1 , wherein the synthesis function calculates the fixed coefficients related to molecular information of interest for the MRS data.

9. The medical data is MRS data, The medical data processing apparatus according to claim 1 , wherein the synthesis function is trained with training data to set the fixed coefficients related to unknown molecular information for the MRS data.

10. the medical data is an MR image, The medical data processing apparatus according to claim 1 , wherein the synthesis function calculates the fixed coefficients relating to an image region of interest in the MR image.

11. the medical data is an MR image, The medical data processing apparatus according to claim 1 , wherein the synthesis function is trained with training data to set the fixed coefficients for extracting unknown image regions from the MR images.

12. the synthesis function is a neural network; 2. The medical data processing device according to claim 1, wherein a function for performing arithmetic processing of fixed coefficients is a layer of the neural network that includes the fixed coefficients as parameters.

13. The medical data is MRS data, The medical data processing device according to claim 12 , wherein, when there are multiple pieces of molecular information of interest in the MRS data, the layer sets the fixed coefficients to the composite data of the multiple pieces of molecular information, or to channels corresponding to the multiple pieces of molecular information.

14. the medical data is an MR image, The medical data processing device according to claim 12 , wherein, when there are multiple image regions of interest in the MR image, the layer is set with the fixed coefficients relating to composite data of the multiple image regions, or with the fixed coefficients being set to channels corresponding to the multiple image regions respectively.

15. the synthesis function is a neural network; The medical data processing device according to claim 1 , wherein the output data is applied to a plurality of tasks by switching a configuration of an output layer of the neural network.

16. The medical data processing device according to claim 1 , wherein the output data is data resulting from any one of medical classification processing, image reconstruction processing, denoising processing, segmentation processing, and super-resolution processing of the medical data.

17. 2. The medical data processing device according to claim 1, wherein the fixed coefficients are values ​​corresponding to data on magnetic field inhomogeneity of an MRI (Magnetic Resonance Imaging) device that collected the medical data, and transmission and reception sensitivity data of a coil.

18. Acquire medical data, generating output data by using a synthesis function including one or more functions for performing a calculation process of fixed coefficients associated with information of interest in the medical data; Medical data processing method.

19. On the computer, an acquisition function for acquiring medical data; a generating function for generating output data by using a synthesis function including one or more functions for performing a calculation process of a fixed coefficient associated with information of interest in the medical data; A medical data processing program that makes this possible.

Images