Medical image processing method, medical image processing device, and medical image processing program

The method enhances MRI image analysis by generating vertical cross-sectional images from horizontal ones using machine learning, improving the understanding of tissue conditions in directions other than the image acquisition direction and enabling three-dimensional tissue representation.

JP7587815B2Active Publication Date: 2024-11-21KYUSHU UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2020181621
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-10-29
Publication Date
2024-11-21
Estimated Expiration
2040-10-29

AI Technical Summary

Technical Problem

MRI images provide detailed examination in the horizontal direction but struggle to grasp the state of tissues in the vertical direction.

Method used

A medical image processing method that generates image information related to a vertical cross section by combining pixel information from horizontal cross-sectional images, using machine learning to specify the region where the target tissue is imaged, allowing for the reconstruction of a vertical cross-sectional image.

Benefits of technology

Enables a more accurate understanding of the target tissue's situation in a direction different from the image acquisition direction, facilitating the creation of three-dimensional data and understanding of periodic tissue changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007587815000001
    Figure 0007587815000001
  • Figure 0007587815000002
    Figure 0007587815000002
  • Figure 0007587815000003
    Figure 0007587815000003
Patent Text Reader

Abstract

To more appropriately grasp a state of an object tissue in a direction different from a direction that is acquired as an image.SOLUTION: A medical image processing method includes: preparing a plurality of pieces of pixel information that constitute a plurality of images of an orthogonal cross section with respect to a first axis obtained by imaging a tissue of an analysis object at mutually different positions along the first axis; generating image information related to an image of a first orthogonal cross section, which is an image of a plane including the first axis by combining the pixel information related to the pixels included in the same plane including the first axis of the plurality of pieces of the prepared pixel information; and specifying a region estimated to have imaged the tissue of the analysis object in the image of the first axis cross section by using machine learning.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a medical image processing method, a medical image processing apparatus, and a medical image processing program. [Background technology]

[0002] A method for identifying a target tissue from an area in which the tissue of a subject and its surroundings are imaged has been studied. For example, Patent Document 1 describes the creation of a model that uses a full-layer convolutional neural network (CNN) to segment at least one part of an anatomical structure from an MRI image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-510463 Summary of the Invention [Problem to be solved by the invention]

[0004] However, although MRI images allow for more detailed examination of horizontal cross sections, it is difficult to grasp, for example, the state of the subject's tissues in the vertical direction.

[0005] The present disclosure has been made in consideration of the above, and aims to provide a technology that makes it possible to more appropriately grasp the condition of a target tissue in a direction different from the direction in which it is acquired as an image. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, a medical image processing method according to one embodiment of the present disclosure includes preparing a plurality of pixel information constituting a plurality of images of a cross section including a first axis obtained by imaging a tissue to be analyzed at different positions along a certain direction, combining pixel information relating to pixels included in the same plane including a second axis that intersects with the first axis from the prepared plurality of pixel information to generate image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, and using machine learning to identify an area in the image of the second axial cross section in which the tissue to be analyzed is estimated to be imaged.

[0007] A medical image processing device according to one embodiment of the present disclosure includes an image preparation unit that prepares a plurality of pixel information constituting a plurality of images of a cross section including a first axis obtained by imaging a tissue to be analyzed at different positions along a certain direction, an image reconstruction unit that generates image information relating to an image of a second axial cross section, which is an image of a plane including a second axis, by combining pixel information relating to pixels included in the same plane including a second axis that intersects with the first axis, from the plurality of prepared pixel information, and a target tissue identification unit that uses machine learning to identify an area in the image of the second axial cross section where the tissue to be analyzed is estimated to have been imaged.

[0008] A medical image processing program according to one embodiment of the present disclosure is a medical image processing program for causing a computer to function as a medical image processing device, and causes a computer to perform the following steps: prepare a plurality of pixel information constituting a plurality of images of a cross section including a first axis obtained by imaging the tissue to be analyzed at different positions along a single direction; combine pixel information relating to pixels included in the same plane including a second axis that intersects the first axis from the prepared plurality of pixel information to generate image information relating to an image of a second axial cross section, which is an image of a plane including the second axis; and identify, using machine learning, an area in the image of the second axial cross section where the tissue to be analyzed is estimated to have been imaged.

[0009] According to the above-mentioned medical image processing method, medical image processing device, and medical image processing program, image information relating to an image of a second axis cross section, which is an image of a plane including the second axis, is generated by combining pixel information relating to pixels included in the same plane including the second axis from a plurality of pixel information constituting a plurality of images of a cross section including the first axis. In addition, a region in the image of the second axis cross section that is estimated to have imaged the tissue to be analyzed is specified using machine learning. In this way, by generating image information relating to the image of the second axis cross section and then specifying a region from the image that is estimated to have imaged the tissue to be analyzed, it becomes possible to more appropriately grasp the state of the target tissue in a direction along the second axis, i.e., in a direction different from the direction in which the image was acquired.

[0010] In identifying an area presumed to have imaged the tissue to be analyzed, the method may further include identifying an area presumed to have imaged the tissue to be analyzed among pixels included in an image of a cross section including the first axis, and creating three-dimensional data using information relating to the area presumed to have imaged the tissue to be analyzed.

[0011] In this case, it is possible to obtain three-dimensional data of the target tissue, so that the three-dimensional shape of the target tissue, including the second axis, can be more appropriately understood.

[0012] The tissue to be analyzed may be a tissue that performs cyclical operations, and in preparing the multiple pieces of pixel information, the method further includes preparing multiple pieces of pixel information constituting multiple images of a cross section including the first axis at each of multiple time points during one cycle, and in generating image information corresponding to the second axial cross section, generating image information relating to the image of the second axial cross section for each of the multiple time points, and in identifying an area in which the tissue to be analyzed is presumed to have been imaged, identifying an area in the image of the second axial cross section in which the tissue to be analyzed is presumed to have been imaged for each of the multiple time points, and calculating information relating to the chronological changes in the tissue to be analyzed during one cycle from the result of identifying the area in which the tissue to be analyzed is presumed to have been imaged.

[0013] In this case, if the subject of analysis is a tissue that changes periodically, it is possible to obtain information related to the changes in the tissue over time, making it possible to properly grasp the periodic changes in the subject tissue in the direction along the second axis. Effect of the Invention

[0014] According to the present disclosure, a technique is provided that makes it possible to more appropriately grasp the condition of a target tissue in a direction different from the direction in which an image is acquired. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram illustrating the configuration of a medical image processing apparatus according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of the medical image processing apparatus. [Diagram 3] FIG. 3 is a flow diagram illustrating the medical image processing method. [Figure 4] FIG. 4 is a diagram for explaining an image of a horizontal cross section. [Diagram 5] FIG. 5 is a diagram showing the result of roughly specifying an area in a horizontal cross-section image where the target tissue is presumed to be imaged. [Figure 6] FIG. 6 is a diagram illustrating a process of reconstructing an image from a horizontal cross-sectional image and identifying a region in the vertical cross-sectional image obtained as a result of the reconstruction, in which a target tissue is estimated to be imaged. [Figure 7] FIG. 7 is a diagram for explaining a case where the identification results in the XZ image and the YZ image are applied to the processing of the XY image. [Figure 8] FIG. 8 is a diagram showing an example of a change in the vertical direction of a tissue. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals and duplicated descriptions will be omitted.

[0017] FIG. 1 is a block diagram for explaining the configuration of a medical image processing device according to an embodiment of the present disclosure. The medical image processing device 1 according to this embodiment is a device that calculates feature amounts related to an organ to be analyzed from image data obtained by capturing an image of the tissue (organ, etc.) of a subject. The target tissue (organ) is not particularly limited, but for example, an organ present in the body can be the target. In addition, an organ that moves (deforms) in a specific cycle can be selected as the target organ. Examples of such organs include, but are not limited to, the heart (atrium or ventricle), lungs, liver, and prostate. For example, the heart repeatedly deforms in a cycle corresponding to the heartbeat. The medical image processing device 1 is also capable of calculating feature amounts focusing on changes along a time series. Therefore, it is also possible to calculate feature amounts focusing on periodic changes in tissues (organs) that repeatedly move (deform) in a specific cycle. Furthermore, the medical image processing device 1 is capable of calculating feature amounts related to deformation in the body axis direction (vertical direction) of the subject, as will be described in detail later. Therefore, it is particularly useful when a tissue that periodically changes (deforms) in the body axis direction, such as the right ventricle, is the target.

[0018] When the target tissue is the heart, an imaging test on a subject, for example, an MRI test, obtains a cine image of the short axis of the heart. The direction parallel to the cross section of this cine image is the first axis. From the cine image of the short axis of the heart, the movement of the heart along the first axis can be grasped. On the other hand, from this cine image, it is difficult to grasp the movement of the heart in a direction perpendicular to the first axis. In the following embodiment, an image of an axis (second axis) different from the first axis, for example, perpendicular to the first axis, is automatically reconstructed by a computer, and information of the target tissue in the second axis direction is added, thereby extracting the target tissue and grasping the information of the target tissue.

[0019] In the following embodiment, it may be assumed that the right ventricle of the heart is selected as the target tissue (organ, etc.). The right ventricle is a type of tissue whose shape changes periodically due to pulsation. In the following embodiment, the feature amount related to the shape change of the right ventricle due to pulsation is acquired using the medical image processing device 1. In the following embodiment, the first axis is an axis extending horizontally (direction perpendicular to the body axis) and the second axis is an axis extending vertically (in the body axis direction). However, this setting is one example and may be changed depending on the characteristics of the target tissue, the method of acquiring the image used by the medical image processing device 1, etc. For example, the first axis and the second axis may not be perpendicular to each other.

[0020] 1, the medical image processing device 1 has an image acquisition unit 11 (image preparation unit), an image processing unit 12, a feature amount calculation unit 13, a storage unit 14, and an output unit 15. The image processing unit 12 also includes an image reconstruction unit 21 and a target tissue identification unit 22.

[0021] The hardware of the medical image processing device 1 is composed of, for example, one or more control computers. For example, the medical image processing device 1 has a circuit 120 shown in FIG. 2. The circuit 120 has one or more processors 121, a memory 122, a storage 123, an input / output port 124, and a timer 125. The storage 123 has a computer-readable storage medium such as a hard disk. The storage medium stores a program (medical image processing program) for causing the medical image processing device 1 to execute procedures related to medical image processing described below. The storage medium may be a removable medium such as a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 122 temporarily stores the program loaded from the storage medium of the storage 123 and the calculation results by the processor 121. The processor 121 configures each of the above-mentioned functional units by executing the above-mentioned programs in cooperation with the memory 122. The input / output port 124 inputs and outputs electrical signals between each of the functional units of the medical image processing device 1 according to instructions from the processor 121. The timer 125 measures the elapsed time by, for example, counting a reference pulse at a constant period.

[0022] Returning to FIG. 1, each functional unit of the medical image processing device 1 will be described. The image acquisition unit 11 of the medical image processing device 1 has a function of acquiring a plurality of image data obtained by imaging a target tissue (organ) of a subject for which a feature amount is to be calculated from an external device or the like. The plurality of image data are a plurality of image data obtained by imaging a target organ. The plurality of image data are, for example, a plurality of image data obtained by imaging a plane perpendicular to the body axis at different positions along the body axis of the subject. As such image data, for example, a magnetic resonance dynamic image (CMR: Cardiovascular MRI) can be used. More specifically, a cine MRI image can be used. In general, a CMR image is obtained by imaging a plurality of times along the body axis (along one direction) to acquire three-dimensional information of the target tissue (organ) and its surroundings. However, the type of image data is not limited to the above. In addition, the type of image data can be changed depending on the target organ. In the medical image processing device 1, a plurality of image data including information related to three axes, namely the body axis and two axes perpendicular to the body axis, are acquired and a subsequent process is performed. Note that the body axis generally extends in a vertical direction, and the plane perpendicular to the body axis corresponds to a horizontal plane. A CMR image corresponds to an image taken of a horizontal cross section of a subject.

[0023] In order to obtain time-series changes in the subject's target organ, the image acquisition unit 11 may acquire image data captured multiple times at a predetermined interval. As an example, when an organ changes periodically, a group of image data acquired at multiple timings capturing the changes during one cycle may be acquired. For example, when the target organ is the heart (atrium or ventricle), expansion and contraction occur during one cycle (one beat). Therefore, in the case of the heart, image data at multiple timings including the expansion and contraction are prepared for calculating the feature amount, and the medical image processing device 1 may acquire these image data.

[0024] The image processing unit 12 has a function of reconstructing an image from a plurality of image data related to a target organ in order to obtain feature quantities related to tissues of other phenomena, and performing processing to identify tissues included in the reconstructed image. The image processing unit 12 has an image reconstruction unit 21 and a target tissue identification unit 22.

[0025] The image reconstruction unit 21 has a function of performing various processes using multiple image data for calculating feature amounts in the target tissue identification unit 22. The image reconstruction unit 21 has a function of reconstructing an image along a plane (cross section including the vertical direction) including the body axis from information (pixel information) of each pixel included in an image of a cross section (horizontal cross section) perpendicular to the body axis.

[0026] Furthermore, the target tissue specifying unit 22 has a function of specifying a target tissue (such as an organ) in the image reconstructed by the image reconstruction unit 21.

[0027] The image reconstruction unit 21 and the target tissue identification unit 22 exert the functions of each unit by using machine learning, statistical processing, etc. using brightness information at each position (pixel) included in the image information. Although details will be described later, image processing is performed by repeatedly performing machine learning, statistical analysis, etc. Machine learning may include supervised learning (linear regression, support vector machine (SVM), logistic regression, etc.), unsupervised learning (principal component analysis, k-means method, etc.), reinforcement learning, deep learning (neural network, etc.), deep reinforcement learning, etc. In addition, principal component analysis, clustering, etc., which are considered to be a type of statistical analysis method, are also included in "machine learning" in this embodiment. In the following description, machine learning, statistical analysis, etc. will be collectively described as "machine learning, etc." When machine learning, etc. is used, an appropriate method may be selected depending on the type and number of data to be used.

[0028] The feature amount calculation unit 13 has a function of calculating a preset feature amount from the image after processing in the image processing unit 12. Examples of the feature amount include various parameters related to the size, shape, and capacity of the target tissue (organ, etc.) at a certain time, and differences in the rate of change of these over time and differences in the timing of change. Machine learning, statistical processing, etc. may also be used to calculate the feature amount. The feature amount calculation unit 13 may have a function of calculating a preset feature amount from the image acquired by the image acquisition unit 11.

[0029] The storage unit 14 has a function of storing image data acquired by the image acquisition unit 11, image data reconstructed by the image processing unit 12, analysis results by the feature amount calculation unit 13, and the like.

[0030] The output unit 15 has a function of outputting the calculation result by the feature amount calculation unit 13. Examples of the output destination include a monitor provided in the medical image processing device 1, an external device, and the like. The output contents and the like are not particularly limited, and for example, the feature amount calculated by the feature amount calculation unit 13 may be output as it is, or some evaluation may be performed based on the calculated feature amount and the result may be output. When performing an evaluation based on the feature amount, logic for performing the evaluation may be stored in the storage unit 14, and the evaluation may be performed using the logic after the feature amount is calculated. Furthermore, the output unit 15 may have a function of outputting image data reconstructed by the image processing unit 12, image data in which tissues are identified by the image processing unit 12, and the like. The output unit 15 may also be configured to generate and output three-dimensional data from the image data in which tissues are identified by the image processing unit 12.

[0031] Next, a procedure for image processing (image reconstruction) and feature amount calculation for a plurality of image data capturing target tissues (organs) by the medical image processing device 1 will be described with reference to Fig. 3 to Fig. 6. Fig. 3 is a flow diagram for explaining the process related to feature amount calculation performed by the medical image processing device 1.

[0032] First, in the medical image processing device 1, the image acquiring unit 11 acquires a plurality of image data obtained by capturing an image of an area including a target tissue (organ) (step S01: image acquiring step). An example of image data acquired by the image acquiring unit 11 is shown in FIG. 4. FIG. 4 shows a schematic diagram of a state in which a plurality of image data obtained in the image acquiring step are configured as an image. The plurality of image data are acquired, for example, at the same time (or within a period that can be regarded as the same time). The plurality of images are images captured along the body axis (Z axis) on an XY plane (horizontal plane) perpendicular to the body axis, and are data captured at different positions along the Z axis. The XY axes can be in any direction, but as an example, the X axis may be the left-right direction of the subject, and the Y axis may be the front-back direction of the subject, as shown in FIG. 4.

[0033] Furthermore, when observing periodic deformation of tissue, etc., it is necessary to acquire multiple image data multiple times along a time series t. As an example, FIG. 4 shows that multiple image data were acquired under the same conditions at three times, t1, t2, and t3. For example, at time t1, images P11, P12, ... P1 of horizontal cross sections along the body axis (Z axis) were acquired. n At time t2, horizontal cross-sectional images P21, P22, ... P2 n At time t3, images P31, P32, ... P3 n Assume that the following images are acquired. Each image is captured along a plane perpendicular to the body axis, and corresponds to image P1 shown in FIG. 5. As an example, each of these horizontal cross-sectional images contains pixel information of 256 pixels in the X and Y directions. Images P11, P12, ... P1 n are images of cross sections parallel to each other and perpendicular to the body axis, and are images of the same position in the horizontal direction (X-axis direction and Y-axis direction).

[0034] For example, l pixels are provided at positions x1 to xl in the X-axis direction, and m pixels are provided at positions y1 to ym in the Y-axis direction. In this case, if image P11 is an image of a horizontal plane at Z=z1, the arrangement of the pixels constituting the image data of image P11 can be expressed as, for example, coordinates (x1 to xl, y1 to ym, z1). Similarly, image P1 n For example, if Z=zn is an image of a horizontal plane, then image P1 n The arrangement of pixels constituting the image data can be expressed as, for example, coordinates (x1 to xl, y1 to ym, zn). As described above, when pixel information for 256 pixels is included in each of the X and Y directions, l=m=256.

[0035] When imaging is performed multiple times in a time series, the interval and number of imaging are appropriately changed depending on the target tissue. For example, the number and interval of imaging may be set so that imaging is performed multiple times within one cardiac cycle.

[0036] Next, in the medical image processing device 1, the image reconstruction unit 21 of the image processing unit 12 specifies an approximate area (rough area) that is estimated to have captured the target tissue from the image (step S02). The reason for specifying the "approximate area" is that the area specified at this stage is used as a reference for image reconstruction in the subsequent processing. Therefore, the "approximate area" is preferably set to include the entire area in which the target tissue is captured. FIG. 5 shows an image of specifying the above-mentioned "approximate area". In FIG. 5, for example, in the image P1, the area A1 corresponds to the "approximate area". It is estimated that the area with high brightness (whitish) in the area A1 is the area in which the target tissue is captured, but the area A1 may be set as the "approximate area" so as to cover the entire area in which the target tissue is captured. As a method for specifying the "approximate area", a method of specifying the "approximate area" in which the target tissue is captured based on the difference in brightness in the image can be mentioned. In addition, machine learning or the like may be used as a method for specifying the approximate area in which the target tissue is captured. As an example, the image reconstruction unit 21 may prepare a plurality of images in which the "approximate area" has been specified in advance as teacher data, and learn a criterion for specifying the approximate area of ​​the target from these images. As an example, a feature amount for distinguishing the pixels capturing the "approximate area" from the other pixels may be set using a plurality of images in which the "approximate area" has been specified, and a feature space for distinguishing the image capturing the "approximate area" from the other pixels may be set using this. In addition, the "approximate area" of the target may be specified from the image of the subject based on learning. When machine learning or the like is used, preprocessing such as adjusting the luminance information of each pixel may be performed.

[0037] Next, in the medical image processing device 1, the image reconstruction unit 21 of the image processing unit 12 reconstructs the image for which the approximate region has been specified (step S03). This image reconstruction generates an image group corresponding to the XZ plane or the YZ plane from an image group captured along the XY plane.

[0038] Here, images P11 to P1n The following will be explained using images P11-P1. n Using the XYZ axes shown in FIG. 4, the area (position of pixels) captured by each image can be expressed as three-dimensional coordinates, for example, as follows (l, m, n are natural numbers): P11;(x1,y1,z1)…(xl,ym,z1) P12;(x1,y1,z2)…(xl,ym,z2) P13;(x1,y1,z3)…(xl,ym,z3) … P1 n ;(x1,y1,zn)…(xl,ym,zn)

[0039] From the information of each pixel contained in these images, for example, an image along the XZ plane is reconstructed. In this case, the image along the XZ plane can be constructed by combining information of pixels with different X and Z coordinates while keeping the Y coordinate constant. When the Y axis varies from 1 to m, for example, m images P1 are reconstructed. y1 ~P1 ym can be generated. P1 y1 ;(x1,y1,z1)…(xl,y1,zn) P1 y2 ;(x1,y2,z1)…(xl,y2,zn) P1 y3 ;(x1,y3,z1)…(xl,y3,zn) … P1 ym ;(x1,ym,z1)…(xl,ym,zn)

[0040] Reconstruction of an image along the YZ plane can be performed in a similar manner. An image along the YZ plane can be constructed by combining information on pixels with different Y and Z coordinates while keeping the X coordinate constant. When the X axis changes from 1 to l, for example, l images P1 are reconstructed. x1 ~P1 xl can be generated. P1 x1 ;(x1,y1,z1)…(x1,ym,zn) P1 x2 ;(x2,y1,z1)…(x2,ym,zn) P1 x3 ;(x3,y1,z1)…(x3,ym,zn) … P1 xl ;(xl,y1,z1)…(xl,ym,zn)

[0041] In this way, by changing the combination of pixel information contained in one image for each pixel contained in n images along the Z axis, an XZ image or YZ image can be reconstructed as a cross-sectional image related to a plane including the vertical direction (Z axis).

[0042] In addition, when reconstructing images in two planes in all patterns while changing all conditions as described above, reconstructed images in m+l patterns can be generated. Furthermore, when a plurality of images (for example, when t times of imaging) are captured in a time series, reconstructed images in (m+l)×t patterns can be generated. At this time, in this step, instead of reconstructing images in all the above patterns, only a part of the images may be reconstructed. For example, image reconstruction may be performed for an XZ plane with a specific Y coordinate value, or image reconstruction may be performed for a YZ plane with a specific X coordinate value. The target for which image reconstruction is performed may be specified by, for example, a user (operator) of the device, or the image reconstruction unit 21 of the image processing unit 12 may determine which plane the image is to be reconstructed on based on a predetermined rule. In addition, when it is assumed that three-dimensional data is to be created, image reconstruction may be performed in a plurality of patterns in order to grasp the shape of the target tissue more accurately.

[0043] Next, in the medical image processing device 1, the target tissue identifying unit 22 of the image processing unit 12 identifies the area in which the target tissue is imaged from the images (XZ image, YZ image) reconstructed by the image reconstruction unit 21 (step S04).

[0044] A method for identifying the region where the target tissue is imaged includes a method for identifying the region where the target tissue is imaged based on the difference in brightness in the image. Machine learning or the like may be used as a method for identifying the region where the target tissue is imaged. As an example, the image reconstruction unit 21 may prepare a plurality of images in which the region where the target tissue is imaged is identified in advance as teacher data, and learn a criterion for identifying the region where the target is imaged from these images. Also, the region where the target is imaged may be identified from the image of the subject based on learning. When machine learning or the like is used, preprocessing such as adjusting the brightness information of each pixel may be performed. Note that the machine learning or the like used in this step S04 may be different from the machine learning or the like used in other steps. Also, the teacher data used in the machine learning or the like may be different depending on, for example, the XZ image, the YZ image, or the like.

[0045] An example of the processing results in steps S03 and S04 is shown in Fig. 6. Image P1 shown in Fig. 6 is the same as image P1 shown in Fig. 5, and it is assumed that the approximate area A1 has been specified. This image P1 corresponds to the above-mentioned XY image. Note that image P1 is an image in which the Y axis extends horizontally and the X axis extends vertically (see the lower left of image P1).

[0046] For example, suppose that an XZ image and a YZ image are reconstructed along a cross section passing through the center of gravity AC of area A1. Since the position of the center of gravity AC can be specified by three-dimensional coordinates, a reconstructed image can be obtained by combining information on pixels capturing an image of a plane having the same X or Y value as the coordinates of the center of gravity AC.

[0047] Image P2 is an XZ image along the dashed line L1 in image P1, and image P3 is a YZ image along the dashed line L2 in image P1. They are reconstructed images in a cross section passing through the center of gravity AC. When images are acquired multiple times along a time series, the center of gravity is not calculated for each image, but an image of a plane passing through the center of gravity AC in the image data of a specific timing among the multiple images is reconstructed for each time. That is, for each of the multiple image data along a time series, the same center of gravity coordinates are used to reconstruct the XZ image and the YZ image. As a result, it is possible to identify the shape change of the target tissue in the same cross section along a time series.

[0048] Image P4 is an image obtained by extracting the area of ​​the target tissue from image P2. Image P5 is an image obtained by extracting the area of ​​the target tissue from image P3. Images P4 and P5 are both displayed with the brightness of the area identified as the area of ​​the target tissue increased, so the identified area appears whiter than images P2 and P3.

[0049] Next, in the medical image processing device 1, the target tissue identification unit 22 of the image processing unit 12 identifies, if necessary, the area in which the target tissue is imaged in the original image used when the image reconstruction unit 21 reconstructs the image, i.e., the XY image along the XY plane (step S05).

[0050] In the steps up to this point, only the "approximate area" where the target tissue is imaged is specified for the XY image (step S02). Therefore, the area where the target tissue is imaged may be specified again for the XY image using machine learning or the like. As a method for specifying the area where the target tissue is imaged, a method for specifying the area where the target tissue is imaged based on the difference in brightness in the image may be used. Also, as with other planar images, machine learning or the like may be used as a method for specifying the area where the target tissue is imaged.

[0051] This step may be omitted depending on the type of feature to be calculated later, etc. In other words, when it is determined that it is not necessary to specify the region in which the target tissue is imaged in the XY image, this process may be omitted.

[0052] Next, for the original image used for reconstruction by the image reconstruction unit 21, that is, the XY image along the XY plane, a region in which the target tissue is imaged is specified as necessary (step S05).

[0053] In the steps up to this point, only the "approximate area" where the target tissue is imaged is specified for the XY image (step S02). Therefore, the area where the target tissue is imaged may be specified again for the XY image using machine learning or the like. As a method for specifying the area where the target tissue is imaged, a method for specifying the area where the target tissue is imaged based on the difference in brightness in the image may be used. Also, as with other planar images, machine learning or the like may be used as a method for specifying the area where the target tissue is imaged.

[0054] By specifying the area where the object is captured in the XY image (step S05) after specifying the area where the object is captured in the XZ image or YZ image (step S04), the accuracy of specifying the area where the object is captured may be improved. This point will be described with reference to FIG.

[0055] In FIG. 7, it is assumed that imaging is performed 20 times (20 frames) along the time series t, and 15 images (15 slices) are acquired along the Z axis (body axis) for each imaging. The number of pixels in the XY plane can be, for example, 256×256 pixels. In FIG. 7, one image is shown as one cell C. Here, instead of displaying 15 cells, only 4 cells are shown. Under such conditions, when identifying the region in which the target tissue is imaged in the XY image, the region in which the target is imaged is usually identified for all images, i.e., 15×20 images.

[0056] However, as a result of identifying the region in which the target tissue is imaged in the XZ image and the YZ image, an estimation result is obtained as to which range in the Z-axis direction the region in which the target tissue is imaged exists. Therefore, it is possible to identify the image in which the region in which the target tissue is imaged exists among the 15×20 images. In FIG. 7, the cell C corresponding to the image in which the region in which the target tissue is imaged exists is hatched. In this way, by identifying the image in which the region in which the target tissue is imaged exists in advance, it is possible to perform a process of identifying the region in which the target tissue is imaged only in the identified XY image. By adopting such a process flow, it is possible to configure a configuration in which the process of identifying the region in which the target tissue is imaged is not performed for an image in which it is estimated that the region in which the target tissue is imaged does not exist. Therefore, it is possible to prevent erroneous detection and improve workability.

[0057] This step may be omitted depending on the type of feature to be calculated later, etc. In other words, when it is determined that it is not necessary to specify the region in which the target tissue is imaged in the XY image, this process may be omitted.

[0058] Next, in the medical image processing apparatus 1, the feature amount calculation unit 13 calculates feature amounts based on the region identified as the region in which the target tissue is imaged (step S06). The feature amounts to be calculated from the captured image (XY image) and the reconstructed images (XZ image, YZ image) can be set according to the type of target tissue or the disease to be analyzed. Therefore, the calculation target and calculation method of the feature amount can be appropriately changed.

[0059] FIG. 8 shows two images (YZ image) corresponding to image P5 and image P5' taken later in the time series under the same conditions. Image P5 corresponds to an image of the right ventricle in diastole, and image P5' corresponds to an image of the systole. Images P5 and P5' are shown adjacent to each other. Comparing images P5 and P5', it is confirmed that the size of the target tissue in the Z direction (the area indicated by the white line in the figure) has changed. In this way, using an image of a plane including the Z axis makes it easier to detect changes in the tissue along the Z axis direction (such as deformation of an organ). In addition, it is also possible to calculate feature amounts based on the images, making quantitative evaluation easier.

[0060] 3, the output unit 15 of the medical image processing device 1 outputs the result related to the feature amount calculated by the above process (step S07). The output result may be the feature amount calculated by the above process, or, in the case where an evaluation is performed based on the feature amount, may be the evaluation result.

[0061] When the results are output from the output unit 15, various evaluations, data processing, etc. may be performed as described above. As an example, when a process for identifying the region in which the target tissue is imaged is performed in each of the XZ image, YZ image, and XY image, a configuration may be used in which three-dimensional data is created based on these identification results. In the above three types of images, images obtained at multiple positions different from each other along each axis are acquired or reconstructed, and a process for identifying the region in which the target tissue is imaged is performed for each of them, so that the region in which the target tissue is imaged can be grasped three-dimensionally. In this case, when the information grasped three-dimensionally is combined and output as three-dimensional data, the user can grasp the shape of the tissue three-dimensionally.

[0062] Furthermore, when the above process is performed using images captured multiple times in a time series, it becomes possible to obtain information related to the change in the feature amount according to the time series in the calculation of the feature amount (step S06). In this case, if the images in the time series are acquired for one or more periods, it is possible to obtain the periodic change in the feature amount. Therefore, the configuration may be such that information related to this periodic change (for example, maximum value, minimum value, etc.) is calculated and output.

[0063] When using images of the ventricles of the heart, the change in the size of the ventricles in the body axis direction that changes over time may be useful for analyzing heart disease or related diseases. For example, when remodeling of the heart (right ventricle) occurs, the change in shape accompanying pulsation may be useful information. Therefore, by acquiring feature amounts related to periodic changes in tissue, it may be possible to more appropriately grasp the change in the shape of the right ventricle.

[0064] [Effect] According to the above-mentioned medical image processing method, medical image processing device, and medical image processing program, image information relating to an image of a vertical cross section (second axis cross section), which is an image of a plane including the vertical direction, is generated by combining pixel information relating to pixels included in the same plane including the vertical direction (second axis) from a plurality of pixel information constituting a plurality of images of horizontal cross sections (cross sections including the first axis). In addition, a region in the image of the vertical cross section that is estimated to have imaged the tissue to be analyzed is specified by using machine learning. In this way, by generating image information relating to the image of the vertical cross section and then specifying the region in the image that is estimated to have imaged the tissue to be analyzed, it becomes possible to more appropriately grasp the condition of the target tissue in the vertical direction, i.e., in a direction different from the direction in which the image was acquired.

[0065] Conventionally, analysis of target tissue has been performed using cine MRI images and the like. However, when grasping deformation of tissue in the vertical direction (extension direction of the second axis) along the body axis, it was generally necessary to estimate based on an image of a horizontal cross section (extension direction of the first axis). It is difficult to obtain a cross section of tissue in the vertical direction, and it has been required to grasp the shape of tissue in the vertical direction more accurately from an image of a horizontal cross section. In contrast, according to the above method, an image of a vertical cross section is generated using information contained in a horizontal cross section image, and a region in which the tissue to be analyzed is estimated to be imaged is specified from the image of the vertical cross section using machine learning. Since the region is specified using machine learning, it can be specified quickly and accurately compared to, for example, a case where a user specifies the region individually. Therefore, it is possible to grasp the shape of tissue in the vertical direction more accurately.

[0066] In addition, by identifying the region where the analysis target tissue is estimated to be imaged, the region where the analysis target tissue is estimated to be imaged in the horizontal cross section image may also be identified. In this case, the result of identifying the region may be used to create three-dimensional data related to the tissue. In this case, since it is possible to obtain three-dimensional data of the target tissue, it is possible to more appropriately grasp the three-dimensional shape of the target tissue including the vertical direction, i.e., a direction different from the direction in which the image was acquired.

[0067] Furthermore, when the tissue to be analyzed is a tissue that performs cyclical behavior, a plurality of pieces of pixel information constituting a plurality of images of horizontal cross sections at a plurality of time points during one cycle may be prepared. Also, image information relating to an image of a vertical cross section may be generated at each of a plurality of time points, and the region may be specified. Furthermore, information relating to the change along the time series of the tissue to be analyzed during one cycle may be calculated from the result of the specification. In this case, when the tissue to be analyzed is a tissue that changes cyclically, it is possible to obtain information relating to the change along the time series of the tissue, and therefore it is possible to appropriately grasp the periodic change of the target tissue in the vertical direction, i.e., in a direction different from the direction obtained as an image.

[0068] The present disclosure is not limited to the above-described embodiment, and the components can be modified without departing from the spirit of the present disclosure. In addition, various disclosures can be formed by appropriately combining the multiple components disclosed in the above-described embodiment.

[0069] For example, in the above embodiment, the image of the subject's target tissue (organ) is a cine MRI image, but the type of image is not limited as long as it is an image of a cross section of the tissue in a specific direction. Furthermore, the images including the first axis of the subject's target tissue (organ) may be images taken at different positions along the first axis, or may be images taken at different positions along a direction (one direction) tilted relative to the first axis. Furthermore, the first axis and the second axis may intersect each other, and are not limited to being "orthogonal" as described in the above embodiment. The configuration described in the above embodiment can also be applied to a case where the first axis and the second axis are not orthogonal.

[0070] In addition, in the above embodiment, a case where machine learning or the like is used in multiple steps has been described, but a method other than machine learning may be used except for the case where the region where the tissue to be analyzed is imaged is specified. Also, the rough region specification (step S02) described in the above embodiment may not be performed. In this way, the procedure described in the above embodiment may be appropriately modified. [Explanation of symbols]

[0071] 1... medical image processing device, 11... image acquisition section, 12... image processing section, 13... feature amount calculation section, 14... storage section, 15... output section, 21... image reconstruction section, 22... target tissue identification section.

Claims

1. Preparing a plurality of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along a single direction; generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified; A medical image processing method comprising: In the step of identifying a region in which the tissue to be analyzed is estimated to have been imaged, the region in which the tissue to be analyzed is estimated to have been imaged is further identified from among pixels included in an image of a cross section including the first axis; The medical image processing method further comprises generating three-dimensional data using information relating to a region presumably where the tissue to be analyzed is imaged.

2. Preparing a plurality of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction; generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified; A medical image processing method comprising: the analyzed tissue is a tissue that performs a periodic operation, In preparing the plurality of pieces of pixel information, a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis are prepared at a plurality of time points in one period, In generating image information corresponding to the second axial slice, image information relating to an image of the second axial slice is generated for each of the plurality of time points; In identifying a region in which the tissue to be analyzed is estimated to have been imaged, an area in which the tissue to be analyzed is estimated to have been imaged is identified in the image of the second axial cross section for each of the plurality of time points; The medical image processing method further includes calculating information relating to changes in the second axis along the time series of the tissue to be analyzed during one cycle from the result of identifying an area estimated to have imaged the tissue to be analyzed.

3. an image preparation unit that prepares a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging the tissue to be analyzed at different positions along one direction; an image reconstruction unit that generates image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; a target tissue specifying unit that specifies a region in the image of the second axial cross section in which the tissue to be analyzed is estimated to be imaged by using machine learning; A medical image processing apparatus comprising: the target tissue specifying unit, in specifying a region in which the analysis target tissue is estimated to have been imaged, further specifies a region in which the analysis target tissue is estimated to have been imaged among pixels included in an image of a cross section including the first axis; The medical image processing apparatus further includes an output unit that creates three-dimensional data using information related to a region estimated to have been imaged of the tissue to be analyzed.

4. An image preparation unit that prepares a plurality of pixel information constituting a plurality of images of a cross section including a first axis obtained by imaging a tissue to be analyzed at different positions along one direction; an image reconstruction unit that generates image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; a target tissue specifying unit that specifies a region in the image of the second axial cross section in which the tissue to be analyzed is estimated to be imaged by using machine learning; A medical image processing apparatus comprising: the analyzed tissue is a tissue that performs a periodic operation, the image preparation unit prepares a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis at a plurality of time points in one cycle, the image reconstruction unit generates image information relating to an image of the second axial slice for each of the plurality of time points in generating image information corresponding to the second axial slice; the target tissue specifying unit, in specifying a region in which the analysis target tissue is estimated to have been imaged, specifies a region in which the analysis target tissue is estimated to have been imaged in the image of the second axial cross section for each of the plurality of time points; The medical image processing device further includes a feature calculation unit that calculates information related to changes in the second axis along the time series of the tissue to be analyzed during one cycle from the result of identifying the area estimated to have been imaged of the tissue to be analyzed.

5. A medical image processing program for causing a computer to function as a medical image processing device, comprising: preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction; generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified; Run the following on your computer: In the step of identifying a region in which the tissue to be analyzed is estimated to have been imaged, a region in which the tissue to be analyzed is estimated to have been imaged is further identified from among pixels included in an image of a cross section including the first axis, The medical image processing program further causes the computer to create three-dimensional data using information related to a region estimated to have been imaged of the tissue to be analyzed.

6. A medical image processing program for causing a computer to function as a medical image processing device, comprising: preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction; generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified; Run the following on your computer: the analyzed tissue is a tissue that performs a periodic operation, In preparing the plurality of pieces of pixel information, a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis at each of a plurality of time points in one period are prepared, In generating image information corresponding to the second axial cross section, image information relating to an image of the second axial cross section is generated for each of the plurality of time points; In identifying a region in which the tissue to be analyzed is estimated to have been imaged, a region in which the tissue to be analyzed is estimated to have been imaged is identified in the image of the second axial cross section for each of the plurality of time points; The medical image processing program further causes the computer to calculate information relating to changes in the second axis along the time series of the tissue to be analyzed during one cycle from the result of identifying the area estimated to have been imaged of the tissue to be analyzed.

Citation Information

Patent Citations

  • Medical image display device, method, and program

    JP2011120825A

  • Magnetic resonance imaging apparatus

    JP2016022111A

  • Automated segmentation using full-layer convolutional networks

    JP2020510463A

  • System and method for automatically detecting a target object from a 3D image

    US20190050981A1