Medical image processing device, medical image processing method, and program

Abstract

To efficiently visualize a region in a medical image whose influence degree on a result of calculation of Radiomics feature amount is large.SOLUTION: A medical image processing device includes an acquisition unit, a first calculation unit, a specification unit, and a display control unit. The acquisition unit acquires a medical image. A first calculation unit calculates an image feature amount indicating a feature of relationships between one pixel on the medical image and one or a plurality of pixels other than the aforesaid pixel. The specification unit specifies a contributing region indicating a region on the medical image from which a feature whose influence degree on a result of calculation of the image feature amount is high is extracted. The display control unit causes the medical image expressing the contributing region to be displayed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to a medical image processing device, a medical image processing method, and a program. [Background technology]

[0002] Radiomics features have been known as features that can be comprehensively extracted from medical images. Recent research has also shown that texture features, which represent texture and are one type of Radiomics feature, are useful for diagnosis and prognosis prediction.

[0003] One example of such technology is to input texture features into a machine learning model that has learned the relationship between texture features and diagnosis names (e.g., disease names, symptoms, etc.) using CNN or the like, and estimate a diagnosis based on the output of the machine learning model. In this case, the basis for estimation is a numerical value representing the texture feature, but it is generally difficult for medical professionals such as doctors to understand the meaning of the numerical value.

[0004] Therefore, by calculating the frequency of occurrence of texture features on medical images and displaying medical images that emphasize features that appear frequently, the basis for the diagnosis estimation results can be visualized on the medical images.

[0005] However, even if a feature appears frequently in a medical image, it may not have a significant effect on the calculation results of the texture feature. In this case, the feature that appears frequently in a medical image cannot be used as a basis for estimating the diagnosis.

[0006] Therefore, there is a need for technology to visualize areas on medical images that have a significant impact on the calculation results of texture features. However, in order to visualize these areas, complicated processing such as analysis of the calculation results of texture features may be required. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Special Publication No. 2022-527240 Summary of the Invention [Problem to be solved by the invention]

[0008] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to efficiently visualize regions on a medical image that have a large influence on the calculation results of Radiomics features. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0009] A medical image processing apparatus according to an embodiment includes an acquisition unit, a first calculation unit, an identification unit, and a display control unit. The acquisition unit acquires a medical image. The first calculation unit calculates an image feature amount indicating characteristics of a relationship between one pixel on the medical image and one or more pixels other than the pixel. The identification unit identifies a contributing area indicating an area on the medical image in which features that have a high degree of influence on the calculation result of the image feature amount are depicted. The display control unit displays a medical image that represents the contributing area. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a medical image processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a shape estimation model according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a method for calculating texture features according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating another example of texture features according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a calculation result of the influence degree according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating another example of the calculation result of the influence degree according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of X-ray CT image data expressing the influence of each element according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of processing executed by the medical image processing apparatus according to the embodiment. [Figure 9] FIG. 9 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the first modification. [Figure 10] FIG. 10 is a diagram illustrating an example of weights of a disease state estimation model according to the first modification. [Figure 11] FIG. 11 is a diagram illustrating an example of the calculation process of the integrated influence degree according to the first modification. [Figure 12] FIG. 12 is a flowchart showing an example of processing executed by the medical image processing apparatus according to the first modification. [Figure 13] FIG. 13 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the first modification. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a medical image processing apparatus, a medical image processing method, and a program will be described in detail with reference to the drawings.

[0012] Fig. 1 is a diagram showing an example of the overall configuration of a medical image processing system S according to the first embodiment. As shown in Fig. 1, the medical image processing system S includes, as an example, a medical image processing device 100 and a medical image diagnostic device 200. The medical image processing system S is installed in, for example, a medical institution such as a hospital.

[0013] The medical image diagnostic device 200 is a device that captures medical images of a subject, and is, for example, an X-ray CT (Computed Tomography) device, an X-ray diagnostic device, an ultrasound diagnostic device, a PET (Positron Emission Tomography) device, an MRI (Magnetic Resonance Imaging) device, a SPECT (Single Photon Emission Computed Tomography) device, etc., but is not limited to these. The medical image diagnostic device 200 is also called a modality.

[0014] 1 illustrates one medical image diagnostic apparatus 200, but it is also possible to provide a plurality of medical image diagnostic apparatuses 200. For ease of explanation, the following description of this embodiment will be given taking as an example a case where the medical image diagnostic apparatus 200 is an X-ray CT apparatus that captures X-ray CT images.

[0015] The medical image processing apparatus 100 is, for example, a server apparatus or a computer such as a PC (Personal Computer). The medical image processing apparatus 100 and the medical image diagnostic apparatus 200 are communicably connected via a network 300 such as an in-hospital LAN (Local Area Network).

[0016] The medical image processing device 100 includes a NW (network) interface 110, a memory 120, an input interface 130, a display 140, and a processing circuit 150.

[0017] The NW interface 110 is connected to the processing circuit 150 and controls the transmission and communication of various data between the medical image processing apparatus 100 and the medical image diagnostic apparatus 200. The NW interface 110 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0018] The memory 120 stores in advance various types of information to be used by the processing circuit 150. The memory 120 also stores various programs.

[0019] The memory 120 is, for example, a storage device that stores various information, such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), an integrated circuit storage device, etc. In addition to an HDD or SSD, the memory 120 may also be a drive device that reads and writes various information from / to portable storage media such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a flash memory, or a semiconductor memory element such as a RAM (Random Access Memory).

[0020] For example, the memory 120 stores a disease state estimation model 121. The disease state estimation model 121 is a trained model generated by a known machine learning (including deep learning) technique using texture features and information representing a disease state (for example, disease name and stage) as training data. The disease state estimation model 121 will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of the disease state estimation model 121.

[0021] 2, the disease state estimation model 121 is a trained model that outputs a disease state estimation result in response to an input of texture features. The disease state estimation result is a sentence that expresses the estimated disease state and the probability of the symptom, such as "There is a 90% chance that this tumor is a grade 4 glioma."

[0022] Note that multiple models may be stored as the disease state estimation model 121 depending on the target organ. Furthermore, the texture feature input to the disease state estimation model 121 may be feature values ​​of multiple types of texture features. Furthermore, multiple estimated symptoms may be output to the disease state estimation model 121. In this case, the disease state estimation model 121 may output the probability of each of the multiple symptoms being that symptom so that the total probability is 100%.

[0023] The input interface 130 is realized by a pen tablet (drawing tablet) that combines a touch pen and a tablet that accepts user operations, a trackball, switch buttons, a mouse, a keyboard, a touchpad that performs input operations by touching the operating surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, and a voice input circuit, etc.

[0024] The input interface 130 may include a plurality of devices that accept operations by a user. The input interface 130 is connected to the processing circuit 150, converts the input operations received from the user into electrical signals, and outputs the electrical signals to the processing circuit 150.

[0025] In this specification, the input interface is not limited to an interface having physical operation parts such as a mouse, keyboard, etc. For example, an example of an input interface also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to processing circuit 150.

[0026] The display 140 displays various types of information under the control of the processing circuitry 150. For example, the display 140 outputs an image interpretation viewer including medical images generated by the processing circuitry 150, a GUI (Graphical User Interface) for accepting various operations from the user, and the like.

[0027] Specifically, the display 140 is a liquid crystal display, a CRT (Cathode Ray Tube) display, etc. The input interface 130 and the display 140 may be integrated together. For example, the input interface 130 and the display 140 may be realized by a touch panel.

[0028] The processing circuitry 150 is a processor that realizes functions corresponding to each program by reading and executing the programs from the memory 120. The processing circuitry 150 of this embodiment includes an acquisition function 151, a first calculation function 152, an estimation function 153, a second calculation function 154, and a display control function 155.

[0029] The acquisition function 151 is an example of an acquisition unit. The first calculation function 152 is an example of a first calculation unit and a second calculation unit. The estimation function 153 is an example of an estimation unit. The second calculation function 154 is an example of a second calculation unit and an identification unit. The display control function 155 is an example of a display control unit.

[0030] Here, for example, each of the processing functions of the processing circuit 150, namely, the acquisition function 151, the first calculation function 152, the estimation function 153, the second calculation function 154, and the display control function 155, is stored in the memory 120 in the form of a program executable by a computer. The processing circuit 150 is a processor.

[0031] For example, the processing circuitry 150 realizes the functions corresponding to each program by reading and executing the programs from the memory 120. In other words, after reading each program, the processing circuitry 150 has the functions shown in the processing circuitry 150 of FIG.

[0032] In FIG. 1, it has been described that the processing functions performed by the acquisition function 151, the first calculation function 152, the estimation function 153, the second calculation function 154, and the display control function 155 are realized by a single processor, but it is also possible to combine multiple independent processors to form the processing circuit 150, and realize the functions by each processor executing a program.

[0033] Furthermore, although FIG. 1 illustrates a single memory 120 storing programs corresponding to each processing function, multiple memories may be distributed and arranged, and the processing circuit 150 may read corresponding programs from individual memories.

[0034] In the above description, an example has been described in which the "processor" reads out from memory a program corresponding to each function and executes it, but the embodiment is not limited to this.

[0035] The term "processor" refers to circuits such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)).

[0036] When the processor is, for example, a CPU, the processor realizes its functions by reading and executing a program stored in a memory.

[0037] On the other hand, if the processor is an ASIC, the function is directly incorporated as a logic circuit in the processor circuitry instead of storing a program in memory 120. Note that each processor in this embodiment is not limited to being configured as a single circuit, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the multiple components in FIG. 1 may be integrated into a single processor to realize its function.

[0038] The acquisition function 151 acquires medical image data of an image of a subject from the medical image diagnostic device 200 via the network 300 and the NW interface 110. The medical image data is, for example, X-ray CT image data. The medical image data may be, for example, two-dimensional image data or three-dimensional image data (volume data).

[0039] The source of the medical image data obtained by the obtaining function 151 is not limited to the medical image diagnostic apparatus 200, but may be another server apparatus or the memory 120 of the medical image processing apparatus 100.

[0040] The first calculation function 152 calculates a texture feature, which is one of the Radiomics features. The texture feature is an example of an image feature. In this embodiment, an example will be described in which a texture feature is used as the image feature, but the image feature is not limited to a texture feature. The image feature may be any feature related to a feature that represents a relationship between a pixel and pixels other than the pixel.

[0041] For example, the first calculation function 152 calculates the texture feature of the X-ray CT image data acquired by the acquisition function 151 according to a calculation method stored in the memory 120 and defined for each type of texture feature.

[0042] As an example, the first calculation function 152 calculates a Joint Energy feature, which is one of texture features calculated by texture analysis using a Gray-Level Co-Occurrence Matrix (GLCM).

[0043] The Joint Energy feature amount is calculated by the following formula (1).

[0044]

number

[0045] N in the above formula (1) g represents the maximum pixel value in a medical image (X-ray CT image data). The Joint Energy feature can be obtained by creating an appearance probability map of combinations of adjacent pixel values, calculating the feature value for each combination of adjacent pixel values ​​in the medical image, and summing these feature values.

[0046] A method for calculating a Joint Energy feature amount will be described below with reference to Fig. 3. Fig. 3 is a diagram for explaining an example of a method for calculating a texture feature amount according to the embodiment.

[0047] 3 shows the relationship between Table V1, which shows the pixel values ​​of each pixel in a medical image, and Map M1, which is a map of the occurrence probability of combinations of adjacent pixel values. Map M1 shows the arrangement of adjacent pixel values ​​in the horizontal (row) direction and the number of occurrences of the combination of adjacent pixel values ​​on the medical image.

[0048] 3, the bolded portions of the map M1 indicate that there are four pixel combinations on the table V1 where a pixel with a pixel value of "1" is adjacent to a pixel with a pixel value of "2" in that order. In this way, the first calculation function 152 creates the map M1 by counting the number of occurrences of each element of the Joint Energy feature for each combination of adjacent pixel values ​​(hereinafter also referred to as the element of the Joint Energy feature).

[0049] Next, the first calculation function 152 calculates element feature values ​​of the Joint Energy feature values, which are feature values ​​for each element of the Joint Energy feature values, according to the created map M1. The first calculation function 152 calculates the Joint Energy feature values ​​by summing the element feature values ​​of the calculated Joint Energy feature values. Specifically, the first calculation function 152 calculates the Joint Energy feature values ​​as shown in the following formula (2).

[0050]

number

[0051] p(0,0) in equation (2) 2represents the element feature of the Joint Energy feature of the element of the Joint Energy feature adjacent in the order of a pixel with a pixel value of “0” and a pixel with a pixel value of “0.” In the example of formula (2), the first calculation function 152 calculates 300, which is the total value of the element feature of the Joint Energy feature, as the Joint Energy feature.

[0052] As another example, the first calculation function 152 calculates an SAE (Small Area Emphasis) feature, which is one of the texture features calculated by texture analysis using a matrix (GLSZM: Gray Level Size Zone Matrix) obtained by counting the number of groups of connected pixels.

[0053] The SAE feature amount is calculated by the following formula (3).

[0054]

number

[0055] N in the above formula (3) g represents the maximum pixel value in the medical image. s represents the maximum number of connected components in a medical image. The SAE feature can be obtained by creating an appearance probability map of the maximum connected component of pixel values, calculating the feature for each maximum connected component of pixel values ​​in the medical image, and summing these feature values.

[0056] A method for calculating an SAE feature amount will be described below with reference to Fig. 4. Fig. 4 is a diagram illustrating another example of a method for calculating a texture feature amount according to the embodiment.

[0057] Figure 4 shows the relationship between Table V2, which lists the pixel values ​​of each pixel in a medical image, and Map M2, which is a map of the occurrence probability of the maximum connected component of the pixel value. Map M2 shows, for each pixel value, the number of occurrences of the connectivity number, which indicates the number of connected pixels of that pixel value.

[0058] 4, the bold part of map M2 indicates that there is one area on table V2 where five pixels with a pixel value of 2 are connected. In this way, the first calculation function 152 creates map M2 by counting the number of occurrences of each SAE element for each connection number for each pixel value (hereinafter also referred to as an element of the SAE feature).

[0059] Next, the first calculation function 152 calculates element feature quantities of the SAE features, which are feature quantities for each element of the SAE features, according to the created map M2. The first calculation function 152 calculates the SAE features by summing the element feature quantities of the calculated SAE features. Specifically, the first calculation function 152 calculates the SAE features as shown in the following equation (4).

[0060]

number

[0061] p(1,1) / 1 in equation (4) 2 represents the element feature of the SAE feature of an element of the SAE feature in which "1" pixels whose pixel value is "1" are connected. In the example of formula (4), the first calculation function 152 calculates 100, which is the total value of the element feature of the SAE feature, as the SAE feature.

[0062] Returning to Fig. 1, the explanation will continue. The estimation function 153 estimates the condition of the subject. For example, the estimation function 153 inputs the texture feature calculated by the first calculation function 152 to a condition estimation model 121 stored in the memory 120. The estimation function 153 estimates the condition of the subject based on the output of the condition estimation model 121.

[0063] As an example, if the output of the disease state estimation model 121 is "There is a 90% probability that this tumor is glioma grade 4," the estimation function 153 estimates "There is a 90% probability that this tumor is glioma grade 4."

[0064] In addition, if the disease state estimation model 121 outputs the probability of each of multiple symptoms being that symptom so that the total probability is 100%, the estimation function 153 may output only symptoms whose probability exceeds a threshold as the estimation result.

[0065] In addition, when multiple models are stored as the disease state estimation model 121 according to the target organ, the estimation function 153 may use a known image recognition technique or the like to detect the organ depicted in the X-ray CT image data, and estimate the disease state using the model corresponding to the organ.

[0066] The second calculation function 154 calculates, for each element, an influence level indicating the degree of influence on the calculation result of the texture feature amount. The influence level in this case is an example of the first influence level.

[0067] For example, the second calculation function 154 calculates the influence degree for each element based on the texture feature amount calculated by the first calculation function 152 and the element feature amount for each element of the texture feature amount. Specifically, the second calculation function 154 calculates the influence degree for each element by dividing each element feature amount by the texture feature amount.

[0068] The second calculation function 154 may specify a contributing region indicating a region on the X-ray CT image data in which a feature that has a high degree of influence on the calculation result of the texture feature amount is depicted.

[0069] For example, the second calculation function 154 may sort the calculated influences in ascending order and identify, as the contributing region, a region on the X-ray CT image data corresponding to an element with the highest calculated influence. Alternatively, for example, the second calculation function 154 may identify, as the contributing region, a region corresponding to a feature whose influence exceeds a threshold.

[0070] Here, Fig. 5 illustrates an example of the calculation result of the influence degree. Fig. 5 shows the Joint Energy feature calculated by Fig. 3 and formula (2) sorted in ascending order by the influence degree of the elements.

[0071] In the example of FIG. 3 and equation (2), p(1,2) 2 This shows that the influence of the above factor is the highest W / 300. Also, p(1,2) 2 This element indicates that the pixel on the X-ray CT image data is a region in which a pixel with a pixel value of "1" is adjacent to a pixel with a pixel value of "2" in the horizontal direction.

[0072] Similarly, in the example of FIG. 3 and equation (2), p(4,4) 2 This shows that the influence of this factor is the lowest Z / 300. Also, p(4,4) 2 This element indicates that the pixel on the X-ray CT image data is a region in which a pixel with a pixel value of "4" is adjacent to another pixel with a pixel value of "4" in the horizontal direction.

[0073] When identifying the contributing region in the example of FIG. 3 and formula (2), the second calculation function 154 calculates the region p(1,2) with the highest influence. 2 The region on the X-ray CT image data corresponding to the element is identified as the contributing region.

[0074] 6 illustrates another example of the calculation result of the influence degree. In FIG. 6, the influence degrees of the element features are sorted in ascending order for the SAE features calculated by FIG. 4 and formula (4).

[0075] In the example of FIG. 4 and equation (4), P(2,1) / 1 2 This shows that the influence of the above factors is the highest V / 100. Also, P(2,1) / 1 2 This element indicates that the region on the X-ray CT image data is a region where one pixel with a pixel value of 2 is connected.

[0076] Similarly, in the example of FIG. 4 and equation (4), P(1,5) / 5 2 This indicates that the influence of the element is the lowest, 0. Also, P(1,5) / 5 2 This element represents an area where five pixels with a pixel value of "1" are connected on the X-ray CT image data.

[0077] When identifying the contributing region in the example of FIG. 4 and Equation (4), the second calculation function 154 determines the P(1,5) / 5 region with the highest influence. 2 The region on the X-ray CT image data corresponding to the element is identified as the contributing region.

[0078] Continuing the explanation, returning to Fig. 1, the display control function 155 causes the display device to display the X-ray CT image data that expresses the degree of influence of each element.

[0079] For example, when a user instructs the display control function 155 to display the estimated basis for the condition, the display control function 155 causes the display 140 to display X-ray CT image data that expresses the influence of each element by changing the display color.

[0080] As an example, the display control function 155 performs a process of gradually changing the display color of each pixel of the X-ray CT image data in accordance with the calculation result of the influence by the second calculation function 154, so that the area with the highest influence is red, the area corresponding to the median influence is yellow, and the area with the lowest influence is blue.

[0081] In the example of FIG. 3 and equation (2), the display control function 155 calculates p(1,2) on the X-ray CT image data. 2 The area corresponding to the element p(4,4) is red. 2 The display 140 displays the X-ray CT image data in which the display color of each pixel is changed in stages so that the areas corresponding to the elements are blue.

[0082] In the example of FIG. 4 and equation (4), the display control function 155 calculates P(2,1) / 1 on the X-ray CT image data. 2 The area corresponding to the element P(1,5) / 5 is red. 2 The display 140 displays the X-ray CT image data in which the display color of each pixel is changed in stages so that the areas corresponding to the elements are blue.

[0083] Here, Fig. 7 is a diagram illustrating an example of displaying the estimation basis. As shown in Fig. 7, a processed image PI1 and a color bar CB are displayed on the display screen of the estimation basis. The processed image PI1 is X-ray CT image data in which the display color of each pixel is changed according to the degree of influence calculated by the second calculation function 154. The color bar CB represents the relationship between the display color and the degree of influence.

[0084] Displaying the color bar CB makes it easier for the user to visually understand the relationship between the display color and the degree of influence.

[0085] The display control function 155 may display on the display 140 the estimation result by the estimation function 153 together with the X-ray CT image data (estimation basis) that expresses the degree of influence of each element.

[0086] Furthermore, when multiple types of texture features are calculated by the first calculation function 152, the display control function 155 may cause the display 140 to display X-ray CT image data that expresses the influence of each element for each type of texture feature. Furthermore, the display control function 155 may cause the display 140 to display X-ray CT image data that expresses the influence of each element for one type of texture feature selected by the user.

[0087] In addition, when the contributing area is identified by the second calculation function 154, the display control function 155 may display the X-ray CT image data on the display 140 with the contributing area on the X-ray CT image data highlighted (for example, displayed in red).

[0088] Next, a description will be given of the processing executed by the medical image-processing apparatus 100 of this embodiment configured as described above. Fig. 8 is a flowchart showing an example of the overall flow of processing executed by the medical image-processing apparatus 100 according to the first embodiment.

[0089] First, the acquisition function 151 acquires X-ray CT image data (step ST101). For example, the acquisition function 151 acquires the X-ray CT image data from the medical image diagnostic apparatus (X-ray CT apparatus) 200 via the NW interface 110.

[0090] Next, the first calculation function 152 calculates the texture feature amount (step ST102). For example, the first calculation function 152 calculates the texture feature amount based on the X-ray CT image data acquired in step ST101 and a calculation method predetermined for each type of texture feature.

[0091] Next, the estimation function 153 estimates the condition of the subject (step ST103). For example, the estimation function 153 inputs the texture feature calculated in step ST102 into a condition estimation model 121 stored in the memory 120, and estimates the condition of the subject corresponding to the X-ray CT image data acquired in step ST101 based on the output from the condition estimation model 121.

[0092] Next, the second calculation function 154 calculates the influence of each element of the texture feature (step ST104). For example, the second calculation function 154 calculates the influence of each element of the texture feature by dividing each element feature calculated in the process of calculating the texture feature in step ST102 by the texture feature.

[0093] Next, the display control function 155 controls to display the estimated result of the disease state and the estimation grounds of the estimated result (step ST105), and ends this process. For example, the display control function 155 causes the display 140 to display the estimated result of the disease state in step ST103 and the X-ray CT image data expressing the influence degree of each element calculated in step ST104.

[0094] In this way, the medical image processing device 100 according to this embodiment acquires X-ray CT image data, calculates texture features, which are one of the Radiomics features of the X-ray CT image data, identifies a contributing area indicating an area on the X-ray CT image data in which features that have a high degree of influence on the calculation results of the texture features are depicted, and displays the X-ray CT image data representing the contributing area.

[0095] Generally, it is difficult for even medical professionals such as doctors to understand the meaning of the values ​​of texture features. In contrast, the medical image processing apparatus 100 according to this embodiment identifies and displays contributing regions on X-ray CT image data, thereby making it possible to visualize regions on X-ray CT image data that are thought to be related to the values ​​of calculated texture features. In other words, the medical image processing apparatus 100 according to this embodiment makes it possible to efficiently visualize regions on a medical image that have a large influence on the calculation results of Radiomics features.

[0096] In addition, the medical image processing device 100 according to this embodiment estimates the condition of the subject based on texture features and a condition estimation model 121, which is a trained model that uses the texture features as input, and displays X-ray CT image data representing the contributing area as the basis for the estimation result.

[0097] When a medical condition is estimated using a trained model that uses texture features as input, the frequency of occurrence of texture features on a medical image is calculated, and information representing areas on the medical image that correspond to features that appear frequently is presented to the user as the basis for estimation. However, even if a feature appears frequently, it may not be closely related to the estimated symptoms.

[0098] In contrast, the medical image processing apparatus 100 according to this embodiment presents to the user, as the basis for estimation, information indicating an area on the X-ray CT image data corresponding to a feature that has a large influence on the calculation result of the texture feature. Since a trained model that uses texture feature values ​​as input performs inference based on the values ​​of the texture feature values, a feature that has a large influence on the calculation result of the texture feature values ​​is likely to have a large influence on the output result of the trained model. Therefore, it is considered that the medical image processing apparatus 100 according to this embodiment can provide the user with more appropriate information as the basis for estimation when estimating a pathology using a trained model that uses texture feature values ​​as input.

[0099] The above-described embodiment can be modified as needed by changing part of the configuration or functions of the medical image processing apparatus 100. Therefore, some modifications of the above-described embodiment will be described below as other embodiments. The following mainly focuses on differences from the above-described embodiment, and detailed descriptions of commonalities with the content already described will be omitted. The modifications described below may be implemented individually or in appropriate combination.

[0100] (Variation 1) In the above-described embodiment, when multiple types of texture features are calculated, X-ray CT image data expressing the influence of each element for each type of texture feature is displayed. In this modified example, a form is described in which the influence of each element of multiple types of feature is expressed using one X-ray CT image data.

[0101] First, the configuration of a medical image processing apparatus 100 according to this modification will be described. Fig. 9 is a block diagram showing an example of the configuration of the medical image processing apparatus 100 according to Modification 1. The configuration of the medical image processing apparatus 100 according to this modification is substantially the same as the configuration of the medical image processing apparatus 100 shown in Fig. 1, but differs from Fig. 1 in that weight information 122 is stored in the memory 120 and that the processing circuitry 150 has a third calculation function 156.

[0102] First, the weight information 122 will be described. The weight information 122 is information that defines a weight indicating how much importance is to be attached to each type of texture feature when estimating a disease state for each type of texture feature in a disease state estimation model 121 that outputs a disease state estimation result in response to input of multiple types of texture feature amounts. Note that when multiple disease state estimation models 121 are stored, the weight information 122 may define a weight for each of the multiple models.

[0103] 10 is a diagram illustrating an example of weights of the disease state estimation model 121 according to Modification 1. The disease state estimation model 121 in the example of FIG. 10 is a trained model that outputs a disease state estimation result in response to input of a texture feature amount of texture feature 1, a texture feature amount of texture feature 2, and a texture feature amount of texture feature 3.

[0104] In the example of Fig. 10, the disease state estimation model 121 outputs the sentence "There is a 90% probability that this tumor is glioma grade 4." In addition, the example of Fig. 10 indicates that the weight of texture feature 1 is w1, the weight of texture feature 2 is w2, and the weight of texture feature 3 is w3.

[0105] In the above case, w1 represents how much importance is placed on the texture feature quantity of texture feature 1 when the disease condition estimation model 121 outputs the disease condition estimation result. Similarly, w2 represents how much importance is placed on the texture feature quantity of texture feature 2 when the disease condition estimation model 121 outputs the disease condition estimation result. Similarly, w3 represents how much importance is placed on the texture feature quantity of texture feature 3 when the disease condition estimation model 121 outputs the disease condition estimation result.

[0106] Specifically, w1, w2, and w3 are positive values ​​less than 1, and the values ​​of w1, w2, and w3 are determined so that w1+w2+w3=1.

[0107] Here, the larger the weight value of a texture feature type, the greater the influence it has on the output result. In other words, if w1>w2>w3 in the example of Fig. 10, the texture feature amount of texture feature 1, texture feature amount of texture feature 2, and texture feature amount of texture feature 3 will have the greatest influence on the output result of the disease condition estimation model 121 in that order.

[0108] Next, the third calculation function 156 of the processing circuit 150 will be described. The third calculation function 156 is an example of a third calculation unit. The third calculation function 156 calculates an integrated influence that integrates the influences of multiple types of texture features. The integrated influence is an example of a second influence. The integrated influence represents the degree of influence on the calculation results of multiple types of texture features for each pixel on a medical image.

[0109] For example, the third calculation function 156 refers to the weight information 122 stored in the memory 120 and identifies the weight of each of the multiple texture feature types for the disease state estimation model 121 used to estimate the disease state. Next, the third calculation function 156 calculates a multiplication value for each texture feature type by multiplying the identified weight of the texture feature type by the influence corresponding to a pixel on the X-ray CT image data.

[0110] Then, the third calculation function 156 calculates the integrated influence of a pixel on the X-ray CT image data by summing the multiplied values ​​of all texture feature types. The third calculation function 156 performs the same process for all pixels on the X-ray CT image data to calculate the integrated influence of each pixel on the X-ray CT image data.

[0111] Here, Fig. 11 is a diagram illustrating an example of the calculation process of the integrated influence degree according to Modification 1. Fig. 11 shows an example of a calculation method of the integrated influence degree when the disease state estimation model 121 of Fig. 10 is used to estimate the disease state.

[0112] 11 is X-ray CT image data in which the display color of each pixel has been changed according to the influence of texture feature 1. Processed image PI3 is X-ray CT image data in which the display color of each pixel has been changed according to the influence of texture feature 2. Processed image PI4 is X-ray CT image data in which the display color of each pixel has been changed according to the influence of texture feature 3.

[0113] 11 represents the influence of texture feature 1 corresponding to a pixel on the X-ray CT image data, calculated by the second calculation function 154. Similarly, influence I2 represents the influence of texture feature 2 corresponding to a pixel on the X-ray CT image data. Similarly, influence I3 represents the influence of texture feature 3 corresponding to a pixel on the X-ray CT image data.

[0114] In the example of Figure 11, the third calculation function 156 calculates the integrated influence I of a pixel on the X-ray CT image data by summing w1I1, which is the weight w1 of texture feature 1 multiplied by the influence I1, w2I2, which is the weight w2 of texture feature 2 multiplied by the influence I2, and w3I3, which is the weight w3 of texture feature 3 multiplied by the influence I3.

[0115] This enables the display control function 155 to display on the display 140 a processed image PI5, which is X-ray CT image data in which the display color of each pixel has been changed according to the integrated influence calculated by the third calculation function 156.

[0116] The third calculation function 156 may specify a common contributing region indicating a region on the X-ray CT image data in which features that have a high degree of influence on all calculation results of the multiple types of texture feature amounts are depicted.

[0117] In this case, the third calculation function 156 may identify a contributing area for each of the multiple types of texture features, and identify an area on the X-ray CT image data where all of the identified multiple contributing areas overlap as a common contributing area.

[0118] The display control function 155 according to this modification causes the display 140 to display the X-ray CT image data, which is calculated by the third calculation function 156 and expresses the integrated influence degree for each pixel on the X-ray CT image data.

[0119] In addition, when the third calculation function 156 identifies a common contributing area, the display control function 155 may display, on the display 140, the X-ray CT image data with the common contributing area on the X-ray CT image data highlighted.

[0120] Next, a description will be given of the processing executed by the medical image-processing apparatus 100 according to this modification. Fig. 12 is a flowchart showing an example of the processing executed by the medical image-processing apparatus 100 according to the first modification.

[0121] The processing of steps ST201 to ST204 in Fig. 12 is substantially the same as the processing of steps ST101 to ST104 in Fig. 8, but in Fig. 12, a plurality of types of texture feature amounts are calculated in step ST202. Furthermore, in step ST203, a plurality of types of texture feature amounts are input to the disease state estimation model 121. Furthermore, in step ST204, the influence degree is calculated for each of the plurality of types of texture feature amounts.

[0122] After step ST204, the third calculation function 156 calculates the integrated influence (step ST205).

[0123] For example, the third calculation function 156 refers to the weight information 122 stored in the memory 120 and identifies the weight of each of the types of texture features for the disease state estimation model 121 used in step ST203.

[0124] The third calculation function 156 calculates a multiplication value by multiplying the identified weight by the influence calculated in step ST203 corresponding to a pixel on the X-ray CT image data for each type of texture feature. The third calculation function 156 calculates the integrated influence of a pixel on the X-ray CT image data by summing the multiplication values. The third calculation function 156 performs the same process for all pixels on the X-ray CT image data to calculate the integrated influence for each pixel on the X-ray CT image data.

[0125] Next, the display control function 155 controls the display of the estimated result of the disease state and the estimation grounds of the estimated result (step ST206), and ends this process. For example, the display control function 155 causes the display 140 to display the estimated result of the disease state in step ST203 and the X-ray CT image data expressing the integrated influence degree for each pixel on the X-ray CT image data calculated in step ST205.

[0126] According to this modification, even when a disease state is estimated using multiple types of texture features, it is possible to visualize areas on a medical image that have a large influence on the estimation of the disease state.

[0127] (Variation 2) In the above embodiment, a form in which the estimated result of the disease state and the X-ray CT image data serving as the basis for the estimation are displayed has been described. In this modified example, a form in which a reference image is also displayed will be described.

[0128] First, the configuration of the medical image processing device 100 according to this modification will be described. Fig. 13 is a block diagram showing an example of the configuration of the medical image processing device 100 according to Modification 1. The configuration of the medical image processing device 100 according to this modification is substantially the same as the configuration of the medical image processing device 100 shown in Fig. 1, but differs from Fig. 1 in that an image DB 123 is stored in the memory 120.

[0129] First, we will explain the image DB (Data Base) 123. The image DB 123 is a database that stores various medical images.

[0130] For example, the image DB 123 stores a symptom (e.g., "glioma grade 4") and a medical image showing a typical example of the symptom in association with each other. Furthermore, for example, the image DB 123 stores a medical image and a value of the texture feature of the medical image in association with each other. Furthermore, for example, the image DB 123 stores a plurality of medical images each representing the influence of each element.

[0131] The display control function 155 according to this modification causes the display 140 to display a reference image together with the medical image that is the basis for estimation.

[0132] For example, when the display control function 155 receives an instruction from a user to display a typical image (hereinafter also referred to as a typical example) of an estimated symptom as a reference image, it refers to the image DB 123, identifies X-ray CT image data corresponding to the symptom estimated by the estimation function 153, and displays the identified X-ray CT image data as a reference image on the display 140 together with the medical image that serves as the basis for the estimation.

[0133] Furthermore, for example, when the display control function 155 receives an instruction from a user to display an image with a similar texture feature value as a reference image, it refers to the image DB 123, identifies the X-ray CT image data in the image DB 123 that corresponds to the texture feature value closest to the texture feature value calculated by the first calculation function 152, and displays the identified X-ray CT image data as a reference image on the display 140 together with the medical image that serves as the basis for the estimation.

[0134] For example, when the display control function 155 receives an instruction from the user to display an image having a similar distribution of contributing regions as a reference image, it searches for X-ray CT image data that expresses the degree of influence of each element in the image DB 123. Then, the display control function 155 displays the X-ray CT image data that has the highest similarity to the X-ray CT image data that is the basis for estimation as a reference image on the display 140 together with the X-ray CT image data that is the basis for estimation.

[0135] In the above, an example was described in which, in accordance with a user's instructions, one of a typical example, an image with similar texture feature values, and an image with similar distribution of contributing areas is displayed on the display 140 as a reference image, but the method of displaying the reference image is not limited to this.

[0136] For example, the display control function 155 may cause all of the typical examples, images with similar texture feature values, and images with similar distributions of contributing regions to be displayed as reference images on the display 140. Furthermore, for example, the display control function 155 may cause one or more images, which are set in advance by the user, from among the typical examples, images with similar texture feature values, and images with similar distributions of contributing regions to be displayed as reference images on the display 140.

[0137] Furthermore, the type of image to be used as the reference image is not limited to a typical example, an image having similar values ​​of texture features, or an image having similar distribution of contributing regions. For example, a normal image according to the age, sex, etc. of the subject may be used as the reference image.

[0138] Furthermore, the display control function 155 may also cause the display 140 to display the estimation result by the estimation function 153 in addition to the X-ray CT image data and the reference image that are the basis for the estimation.

[0139] According to this modification, for example, the user can check the type of reference image appropriate to the situation together with the medical image that is the basis for the estimation, making it easier to determine the validity of the basis for the estimation.

[0140] (Variation 3) In the above embodiment, an example in which the present method is applied to the Joint Energy feature and the SAE feature has been described, but the feature to which the present method can be applied is not limited to the above. For example, the present method can be applied to any feature that satisfies any of the following formulas (5) to (8).

[0141]

number

[0142]

number

[0143]

number

[0144]

number

[0145] (Variation 4) In the above-described embodiment, the medical image processing device 100 acquires medical image data from the medical image diagnostic device 200, but the source from which the medical image data is acquired is not limited to this. The medical image processing device 100 may acquire the medical image data from a medical image storage device that stores medical images captured by the medical image diagnostic device 200.

[0146] The medical image storage device is, for example, a server device of a PACS (Picture Archiving and Communication System), and stores medical image data in a format that complies with DICOM (Digital Imaging and Communications in Medicine).

[0147] (Variation 5) In the above embodiment, a series of processes is performed on a medical image. However, the images that can be processed by this method are not limited to medical images. For example, this method can also be applied to images depicting plants, animals, objects, etc.

[0148] For example, when using an image of a plant, the estimation function 153 may perform a process of estimating the name (type) of the plant depicted in the image. In this case, the estimation function 153 may perform a process of estimating the name of the plant by inputting the calculated texture feature into a trained model that outputs the name of the plant in response to the input of the texture feature trained by a known machine learning technique.

[0149] Furthermore, for example, when an image of an animal is used, the estimation function 153 may perform a process of estimating the name (type) of the animal depicted in the image. In this case, the estimation function 153 may perform a process of estimating the name of the animal by inputting the calculated texture feature into a trained model that outputs the name of the animal in response to the input of the texture feature learned by a known machine learning technique.

[0150] Furthermore, for example, when an image of an item is used, the estimation function 153 may perform a process of estimating the name (type) of the item depicted in the image. In this case, the estimation function 153 may perform a process of estimating the name of the item by inputting the calculated texture feature into a trained model that outputs the name of the item in response to the input of the texture feature trained by a known machine learning technique.

[0151] According to this modification, even when targeting images other than medical images, it is possible to efficiently visualize areas on the image that have a large influence on the calculation results of texture feature amounts.

[0152] The various data handled in this specification are typically digital data.

[0153] According to at least one of the embodiments described above, it is possible to efficiently visualize regions on a medical image that have a large influence on the calculation results of Radiomics features.

[0154] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0155] 100 Medical image processing device 110 Network Interface 120 memory 130 Input Interface 140 Display 150 Processing Circuit 151 Acquisition Function 152 First calculation function 153 Estimation Function 154 Second calculation function 155 Display control function 156 Third calculation function 200 Medical imaging diagnostic equipment 300 Network S Medical Image Processing System

Claims

1. an acquisition unit for acquiring medical images; a first calculation unit that calculates an image feature amount indicating a characteristic of a relationship between one pixel on the medical image and one or more pixels other than the pixel; an identifying unit that identifies a contributing area indicating an area on the medical image in which the feature that has a high degree of influence on the calculation result of the image feature amount is depicted; a display control unit that displays the medical image representing the contributing region; A medical image processing device comprising:

2. a second calculation unit that calculates, for each of the features, an element feature amount that is a feature amount corresponding to each of the features on the medical image, and calculates a first influence level that indicates a degree of influence that the feature has on a calculation result of the image feature amount based on the image feature amount and the element feature amount; The identification unit identifies the contributing region based on the first influence degree. The medical image processing device according to claim 1 .

3. the identifying unit identifies an area on the medical image corresponding to the feature having the highest first influence degree as the contributing area. The medical image processing device according to claim 2 .

4. the identifying unit identifies, as the contributing region, a region on the medical image corresponding to the feature in which the first influence degree exceeds a threshold value; The medical image processing device according to claim 2 .

5. an estimation unit that estimates a pathology of the subject based on the image feature and a trained model that is functionally configured by machine learning to output a pathology estimation result in response to an input of the image feature; the display control unit displays the medical image representing the contributing region as an estimation basis for the estimation result. The medical image processing device according to claim 2 .

6. the first calculation unit calculates a plurality of types of image feature amounts; the second calculation unit calculates the first influence degree for each of the plurality of types of image feature amounts; a third calculation unit that calculates, for each pixel on the medical image, a second influence indicating the degree of influence that the pixel has on each of the multiple types of image features based on the first influence and a weight indicating the degree to which the trained model places importance on a specific type of image feature when outputting the estimation result; The identification unit identifies the contributing region based on the second influence degree. The medical image processing device according to claim 5 .

7. the display control unit identifies a medical image in the image database having a similar distribution of contributing areas on the medical image based on the identified contributing areas and an image database storing a plurality of medical images each representing the contributing areas, and displays the identified medical image as a reference image. The medical image processing apparatus according to any one of claims 1 to 6.

8. an acquisition step of acquiring a medical image; a calculation step of calculating an image feature amount indicating a characteristic of a relationship between one pixel on the medical image and one or more pixels other than the pixel; a specifying step of specifying a contributing area indicating an area on the medical image in which a feature having a high degree of influence on a calculation result of the image feature amount is depicted; a display control step of displaying the medical image representing the contributing region; An image processing method using an image processing device including:

9. an acquisition step of acquiring a medical image; a calculation step of calculating an image feature amount indicating a characteristic of a relationship between one pixel on the medical image and one or more pixels other than the pixel; a specifying step of specifying a contributing area indicating an area on the medical image in which a feature having a high degree of influence on a calculation result of the image feature amount is depicted; a display control step of displaying the medical image representing the contributing region; A program that causes a computer to execute the following.

Images