Medical image processing method

Abstract

To improve accuracy in identifying a lesion in image diagnosis capturing a plurality of cross-sectional images in each of a plurality of time phases.SOLUTION: A medical image processing method includes the following steps. A first step acquires a plurality of cross-sectional images obtained by imaging cross sections of a subject at each of a plurality of time phases. A second step creates reference image feature data by adding information representing a first time phase to a feature amount of the reference cross-sectional images captured at a first cross section position at the first time phase. A third step selects from the plurality of cross-sectional images captured in the second time phase, reference cross-sectional images captured near the first cross section position and creates reference image feature data by adding information showing the second time phase to a feature amount of the selected reference cross-sectional images. A fourth step outputs information on a subject's lesion by inputting to a learned model the reference image feature data and the reference image feature data.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a method for processing medical images and to a computer program for processing medical images. [Background technology]

[0002] One medical procedure using medical images is to identify whether a lesion is benign or malignant from multiple images obtained by dynamic examination using a contrast agent. In dynamic examination using a contrast agent, imaging is performed at multiple time phases (i.e., multiple different timings) to monitor the movement of the contrast agent within the tissue. Here, for example, in an MRI (Magnetic Resonance Imaging) examination, multiple cross-sectional images are taken of the subject for each time phase. Therefore, a contrast MRI examination, which acquires MRI images at multiple time phases using a contrast agent, generates a large number of images.

[0003] In MRI examinations, doctors who identify whether a lesion is benign or malignant must read the information necessary for judgment from a large number of images, which requires advanced knowledge and experience. However, there is a shortage of doctors with such advanced knowledge and experience. For this reason, methods have been proposed that support image diagnosis using medical image processing using computers. As an example, a medical image processing device has been proposed that enables a definitive diagnosis of tumors without relying on the experience or subjectivity of the operator, such as a doctor (e.g., Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-146455 Summary of the Invention [Problem to be solved by the invention]

[0005] In dynamic examinations such as contrast-enhanced MRI, multiple cross-sectional images are taken at multiple time phases. A physician selects the appropriate cross-sectional image from each time phase. The selected cross-sectional image is then fed into a classification model to determine whether the lesion is benign or malignant.

[0006] However, conventional identification models do not function correctly unless all cross-sectional images captured at each of multiple pre-specified time phases are input. Therefore, for example, if a time phase is affected by the subject's physical condition and cross-sectional images cannot be captured, the necessary cross-sectional images are not input to the identification model, resulting in inaccurate identification. Furthermore, if the subject moves during imaging, the position at which the lesion appears in the scan direction of the cross-sectional imaging may differ for each time phase. Setting an appropriate region of interest for the cross-sectional images at each time phase requires advanced knowledge and experience, but such doctors are not available at all hospitals or medical institutions.

[0007] An object of one aspect of the present invention is to improve the accuracy of identifying a lesion in image diagnosis in which a plurality of cross-sectional images are taken at each of a plurality of time phases. [Means for solving the problem]

[0008] A medical image processing method according to one aspect of the present invention includes acquiring a plurality of cross-sectional images obtained by photographing a plurality of cross sections of a subject at each of a plurality of time phases, creating reference image feature data by adding information representing the first time phase to at least some of the feature quantities of a reference cross-sectional image photographed at a first cross-sectional position at a first time phase among the plurality of time phases, selecting a reference cross-sectional image photographed near the first cross-sectional position from a plurality of cross-sectional images photographed at a second time phase among the plurality of time phases, creating reference image feature data by adding information representing the second time phase to at least some of the feature quantities of the selected reference cross-sectional image, and inputting the reference image feature data and the reference image feature data into a trained model to output information related to a lesion in the subject. [Effects of the Invention]

[0009] According to the above-described aspect, the accuracy of identifying a lesion is improved in image diagnosis in which a plurality of cross-sectional images are captured in each of a plurality of time phases. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a problem in a contrast-enhanced MRI examination. [Figure 2] FIG. 10 is a diagram showing another example of a problem in a contrast-enhanced MRI examination. [Figure 3] 1 is a diagram illustrating an example of a diagnosis support system according to an embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating an example of a medical image processing method according to an embodiment of the present invention. [Figure 5] FIG. 10 is a diagram illustrating an example of a method for setting a reference region of interest. [Figure 6] FIG. 10 is a diagram illustrating an example of a method for setting a reference region of interest. [Figure 7] FIG. 10 is a diagram illustrating an example of a method for creating image feature data. [Figure 8] 10 is a flowchart illustrating an example of a procedure for creating image feature data. [Figure 9] FIG. 10 is a diagram illustrating an example of processing of a discrimination model. [Figure 10] FIG. 2 illustrates an example of a hardware configuration of an operation terminal. DETAILED DESCRIPTION OF THE INVENTION

[0011] Dynamic contrast imaging is one of the medical procedures using medical images. In dynamic contrast imaging, images are taken at multiple time phases (i.e., at multiple different times) to monitor the movement of the contrast agent within the tissue.

[0012] For example, in a contrast-enhanced MRI examination in which a contrast agent is administered to a subject and MRI is used to diagnose liver cancer, cross-sectional images are taken in the pre-contrast, arterial, portal venous, late, and hepatic enhancement phases. Alternatively, images may be taken in more time phases. The pre-contrast phase refers to the state before the contrast agent is administered to the subject. The arterial, portal venous, late, and hepatic enhancement phases refer to the states approximately 30 seconds, 60 seconds, 120 seconds, and 20 minutes after the contrast agent is administered to the subject, respectively. Here, in an MRI examination, multiple cross-sectional images of the subject are taken for each time phase to form a three-dimensional image. That is, in a contrast-enhanced MRI examination, multiple cross-sectional images of the subject are taken for each time phase to form a three-dimensional image.

[0013] FIG. 1 shows an example of a problem in a contrast-enhanced MRI examination. In this example, MRI is performed in five time phases (pre-contrast, arterial phase, portal venous phase, late phase, and hepatic enhancement phase). Furthermore, multiple cross-sections of the subject are captured in each time phase. In the following description, multiple cross-sectional images obtained in one time phase may be referred to as "volume data." A doctor diagnosing the subject then selects one cross-sectional image suitable for diagnosis from the volume data for each time phase.

[0014] As shown in Figure 1A, cross-sectional images selected from the volume data for each time phase are input to a discrimination model. The discrimination model is an example of a trained model, and is assumed to have already learned the correspondence between the input cross-sectional images and the actual discrimination results (benign or malignant). In other words, the discrimination model is assumed to have been trained by providing "correct answers (or training data)" for input images of past subjects. Then, by inputting cross-sectional images of a new subject into the discrimination model, a discrimination result is output.

[0015] However, in existing image processing methods, only cross-sectional images selected from volume data for each time phase are input to the identification model, so the identification model cannot operate correctly unless cross-sectional images for all time phases are input. For example, if a cross-sectional image for one of the time phases (in the example shown in Figure 1B, a cross-sectional image for the arterial phase) cannot be obtained due to the subject's physical condition, the identification model cannot operate correctly. Alternatively, there is a risk that the model will not be able to correctly identify whether a lesion is benign or malignant.

[0016] Figure 2 shows another example of a problem in contrast-enhanced MRI examinations. In this example, MRI scans are performed in the pre-contrast, arterial, portal venous, late, and hepatic enhancement phases. As shown in Figure 2A, multiple cross-sections of the subject are captured in each phase. In the following description, the scanning direction of MRI scans is sometimes referred to as the "Z-axis." Positions within the captured cross-sections are represented by X and Y coordinates.

[0017] As described above, cross-sectional images selected from the volume data for each time phase are input to the discrimination model. However, if the subject moves during MRI imaging, an appropriate cross-sectional image may not be selected. In the example shown in Figure 2A, the subject moves during the late phase imaging. In this case, as shown in Figure 2B, the cross-sectional image selected from the volume data for the late phase may not properly include the lesion. This significantly reduces the discrimination accuracy of the discrimination model.

[0018] This problem can be solved if the doctor examining the patient checks all cross-sectional images and selects the appropriate one from each time phase. However, contrast-enhanced MRI examinations generate a large number of cross-sectional images, which places a heavy burden on doctors. In addition, there is a shortage of doctors who can properly analyze cross-sectional images.

[0019] Fig. 3 shows an example of a diagnosis support system according to an embodiment of the present invention. The diagnosis support system 1 according to the embodiment of the present invention includes an imaging device 10, an examination image server 20, an operation terminal 30, and a display device 40. Note that the diagnosis support system 1 may further include other devices or functions not shown in Fig. 3.

[0020] In this embodiment, the imaging device 10 is an MRI imaging device, which captures multiple cross sections of a subject. When a contrast MRI examination is performed, the imaging device 10 performs MRI imaging at each of multiple time phases. In this case, multiple cross-sectional images (i.e., volume data) are generated for each time phase. Note that the embodiment of the present invention is not limited to MRI examinations, but is also applicable to diagnostic support for capturing multiple cross sections of a subject.

[0021] The examination image server 20 stores the examination images captured by the imaging device 10. The examination images are stored in association with the imaging date and time, examination ID, subject ID, etc. In contrast MRI examinations, the examination images are stored in association with information indicating the time phase at which the images were captured, in addition to the imaging date and time, examination ID, and subject ID.

[0022] The operation terminal 30 identifies a lesion in a subject based on the examination image stored in the examination image server 20. The operation terminal 30 includes an examination image acquisition unit 31, a storage unit 32, and a lesion identification unit 33. However, the operation terminal 30 may further include other functions not shown in FIG. 3. For example, the operation terminal 30 includes a user interface that accepts instructions from a user (here, a doctor involved in the MRI examination). The operation terminal 30 is realized by a computer including a processor and a memory.

[0023] In response to an instruction from a user, the examination image acquisition unit 31 acquires an examination image of an object to be identified from the examination image server 20. In a contrast MRI examination, the examination image acquisition unit 31 acquires volume data of each time phase of the subject from the examination image server 20. The examination images acquired by the examination image acquisition unit 31 are stored in the storage unit 32.

[0024] Lesion identification unit 33 outputs information related to the lesion of the subject using the test image stored in memory unit 32. As an example, lesion identification unit 33 identifies whether the lesion of the subject is benign or malignant and outputs the identification result. To perform this identification, lesion identification unit 33 includes image feature data creation unit 34 and identification model 35. However, lesion identification unit 33 may further include other functions not shown in FIG. 3.

[0025] The image feature data creation unit 34 uses the test image stored in the storage unit 32 to create image feature data to be input to the identification model 35. The method for creating image feature data will be described later.

[0026] The discrimination model 35 discriminates whether a lesion in a subject is benign or malignant based on the image feature data created by the image feature data creation unit 34. Alternatively, the discrimination model 35 represents the probability that a lesion is benign (or the probability that a lesion is malignant). The discrimination model 35 has learned the correspondence between image feature data previously input and actual discrimination results. Here, the "actual discrimination result" is, for example, information indicating benign or malignant obtained by collecting cells from the lesion and conducting an examination. In other words, the discrimination model 35 is trained by providing "correct answers (or training data)" to image feature data of past subjects.

[0027] The identification result obtained by lesion identification unit 33 is displayed on display device 40. Note that display device 40 may be a part of operation terminal 30, or may be connected to operation terminal 30 via a network.

[0028] Fig. 4 is a flowchart showing an example of a medical image processing method according to an embodiment of the present invention. In this example, it is assumed that image processing related to a contrast-enhanced MRI examination is performed. That is, in the diagnosis support system 1 shown in Fig. 3, MRI imaging is performed by the imaging device 10 in each of a plurality of time phases, and volume data for each time phase is stored in the examination image server 20. Then, when a user (here, a doctor involved in the MRI examination) specifies a subject using the user interface of the operation terminal 30, the examination image acquisition unit 31 acquires the corresponding examination image from the examination image server 20. Thereafter, the lesion identification unit 33 executes the processes of S1 to S6.

[0029] In S1, the image feature data creation unit 34 sets a region of interest (reference region of interest) including a lesion in any one of a plurality of time phases. Here, "any one of the time phases" is preferably a time phase among the plurality of time phases that is considered to be the most easily recognizable for a lesion. For example, in a contrast-enhanced MRI examination for diagnosing liver cancer, lesions are generally most likely to appear clearly in cross-sectional images taken in the hepatic enhancement phase among the pre-contrast, arterial phase, portal venous phase, late phase, and hepatic enhancement phase. In the following description, the time phase selected to set the reference region of interest described above may be referred to as the "reference phase."

[0030] In the process of setting a region of interest including a lesion, a reference cross-sectional image in which the lesion is most clearly visible is selected from the volume data obtained at the reference time phase. Then, a reference region of interest including the lesion is set in the selected reference cross-sectional image. The reference region of interest is not particularly limited, but is preferably rectangular.

[0031] The process of selecting a reference cross-sectional image from the volume data of the reference time phase and setting a reference region of interest may be performed by a person (i.e., a doctor) or a computer (i.e., the operation terminal 30). Alternatively, the process may be performed by a person using a computer. In this case, for example, a person may select one cross-sectional image from the volume data of the reference time phase, and the computer may set a reference region of interest including a lesion in that cross-sectional image.

[0032] 5 shows an example of a method for setting a reference region of interest. In this embodiment, as shown in FIG. 5A, a plurality of cross-sectional images are captured at a reference time phase (e.g., a hepatic imaging phase). The intervals in the Z direction at which the cross-sectional images are captured are not particularly limited, but are, for example, about 1 to 2 millimeters. Furthermore, information indicating the position in the Z direction is added to each cross-sectional image by the imaging device 10.

[0033] The doctor or the operation terminal 30 selects a reference cross-sectional image in which the lesion appears most clearly from among the multiple cross-sectional images captured at the reference time phase. In the example shown in Fig. 5A, the cross-sectional image captured at "Z coordinate = 1210 [mm]" is selected as the reference cross-sectional image. The Z coordinate represents the position in the scanning direction when the imaging device 10 captures an image of the subject.

[0034] Next, the doctor or the operation terminal 30 sets a region surrounding the image portion that is thought to be a lesion as a reference region of interest in the reference cross-sectional image. For example, when the doctor sets a reference region of interest, the doctor operates the keyboard and / or mouse of the operation terminal 30 to specify a desired region while the reference cross-sectional image is displayed on the display device. In the example shown in FIG. 5B, a region with an X coordinate ranging from X1 to X2 and a Y coordinate ranging from Y1 to Y2 is specified as the reference region of interest. Then, the image within the reference region of interest is cut out from the reference cross-sectional image to obtain the reference image shown in FIG. 5C.

[0035] The reference image I[0] is expressed by the following formula: V0 represents a reference cross-sectional image selected from the multiple cross-sectional images captured at the reference time phase. X1, X2, Y1, and Y2 represent coordinates specifying the range of the region of interest, as described with reference to FIG. 5B. T0 identifies the reference time phase, which represents the hepatic enhancement phase in this embodiment. Z[0] represents the position of the selected cross-sectional image in the scanning direction. I[0] =V0[X1:X2, Y1:Y2, T0, Z[0] ]

[0036] Returning to the description of the flowchart shown in Fig. 4, in S2, the image feature data creation unit 34 sets a corresponding region of interest (reference region of interest) in each of the other phases. Note that when the hepatic enhancement phase is selected as the reference phase in S1, the "other phases" are the pre-contrast, arterial phase, portal venous phase, and late phase.

[0037] The image feature data creation unit 34 selects M cross-sectional images located near the Z coordinate of the reference cross-sectional image from among the multiple cross-sectional images in each of the other time phases. M is an integer equal to or greater than 2. The reason for selecting multiple cross-sectional images in each of the other time phases is to eliminate or mitigate the influence of the subject's movement during imaging. Therefore, the value of M (i.e., the number of cross-sectional images to be selected) may be determined based on, for example, the range in which the subject can move during imaging and the interval in the Z direction at which the cross-sectional images are captured. Alternatively, the value of M may be determined based on simulation.

[0038] Next, the image feature data creation unit 34 sets a region of interest in each cross-sectional image selected from each time phase. In the following description, these regions of interest may be referred to as "reference regions of interest." The range (i.e., position and shape) of the reference region of interest set in each cross-sectional image is the same as that of the reference image set in the reference cross-sectional image. For example, when the reference region of interest is set as shown in FIG. 5B, a region with an X coordinate ranging from X1 to X2 and a Y coordinate ranging from Y1 to Y2 is specified as each reference region of interest.

[0039] The process of selecting M cross-sectional images from the volume data of each of the other time phases and setting the reference region of interest is executed by a computer (i.e., the operation terminal 30). At this time, the Z coordinate of the reference cross-sectional image and the X-coordinate range and Y-coordinate range of the reference region of interest are provided to the image feature data creation unit 34 as parameters for selecting M cross-sectional images and setting the reference region of interest.

[0040] FIG. 6 shows an example of a method for setting a reference region of interest. In this example, a cross-sectional image captured at "Z coordinate = 1210 [mm]" is selected as the reference cross-sectional image during the hepatic imaging phase. Three cross-sectional images are then selected from among the multiple cross-sectional images captured during the time phase Tp. The time phase Tp is a time phase other than the hepatic imaging phase, and represents the pre-contrast, arterial, portal venous, or late phase.

[0041] In this case, the image feature data creation unit 34 selects three cross-sectional images from the volume data Vp of the time phase Tp, the cross-sectional images having Z coordinate values closest to 1210. In the example shown in Fig. 6, the cross-sectional image V[p,1], the second closest cross-sectional image V[p,2], and the third closest cross-sectional image V[p,3] are selected. The Z coordinates of the cross-sectional images V[p,1], V[p,2], and V[p,3] are represented by Z[p,1], Z[p,2], and Z[p,3], respectively.

[0042] Next, the image feature data creation unit 34 sets a reference region of interest in each cross-sectional image V[p,1] to V[p,3]. Here, the position and shape of the reference region of interest are the same as those of the reference region of interest. The image feature data creation unit 34 is provided with the X coordinate range [X1, X2] and the Y coordinate range [Y1, Y2] of the reference region of interest as parameters for setting the reference region of interest. Based on these parameters, the image feature data creation unit 34 sets a reference region of interest in each cross-sectional image V[p,1] to V[p,3]. Then, a reference image is obtained by cutting out the image within the reference region of interest from each cross-sectional image. This process is performed for each of the pre-contrast, arterial phase, portal vein phase, and late phase.

[0043] The reference image I[p,m] generated for the time phase Tp is expressed by the following formula. V[p,m] represents a cross-sectional image selected from the multiple cross-sectional images taken at the time phase Tp. m identifies the M cross-sectional images selected from each time phase. In the case where three cross-sectional images are selected from each time phase, the value of m is 1, 2, or 3. X1, X2, Y1, and Y2 represent coordinates specifying the range of the region of interest. Tp is a time phase other than the reference phase, and represents the pre-contrast, arterial phase, portal venous phase, or late phase. Z[p,m] represents the position in the scanning direction of the cross-sectional image m selected from the volume data of the time phase Tp. I[p,m] =V[p,m] [X1:X2, Y1:Y2]

[0044] In this way, in S1, a reference image is generated based on the cross-sectional images captured in the reference time phase. In addition, in S2, M cross-sectional images are selected from the cross-sectional images captured in each of the other time phases, and reference images are generated based on each selected cross-sectional image. Therefore, in a case where the other time phases are pre-contrast, arterial phase, portal vein phase, and late phase, and the value of M is 3, 12 reference images are generated.

[0045] Returning to the description of the flowchart shown in Fig. 4, in S3, the image feature data creation unit 34 calculates the image feature amount of the image of each region of interest. That is, the image feature data creation unit 34 calculates the feature amount of the base image and the feature amount of each reference image.

[0046] The method for calculating the image feature amount is not particularly limited, and any method can be used as long as the calculation result represents the image feature. For example, the image feature data creation unit 34 may perform image processing on the image of each region of interest using a convolutional neural network (CNN) including a convolutional layer. In this case, the image feature data creation unit 34 calculates the image feature amount of the region of interest by locally processing pixel data within the region of interest using a filter (or kernel) of a predetermined size.

[0047] In S4, the image feature data creation unit 34 creates image feature data by adding information representing the time phase and the position in the Z direction to the image feature of each region of interest calculated in S3. The information representing the time phase indicates the time phase at which the cross-sectional image including the image of the region of interest for which the image feature was calculated was captured. In this embodiment, the information identifies the pre-contrast, arterial, portal venous, late, or hepatic enhancement phase. Alternatively, the information representing the time phase may indicate the phase at which the cross-sectional image including the image of the region of interest for which the image feature was calculated was captured among multiple phases constituting a contrast-enhanced MRI examination. In this case, the information representing the time phase indicates the first to fifth phases. The information representing the position in the Z direction indicates the position of the cross section at which the cross-sectional image including the image of the region of interest for which the image feature was calculated was captured. In this embodiment, the information representing the position in the scanning direction (or the distance from the reference position) when the imaging device 10 images the subject.

[0048] FIG. 7 shows an example of a method for creating image feature data. In this example, MRI scans are performed at five time phases (pre-contrast, arterial, portal venous, late, and hepatic enhancement phases), with the hepatic enhancement phase designated as the reference time phase T0. Therefore, a reference image I[0] is generated from a reference cross-sectional image selected from the volume data of the reference time phase T0. Furthermore, the pre-contrast, arterial, portal venous, and late phases are designated as time phases T1, T2, T3, and T4, respectively. Furthermore, three cross-sectional images are selected for each of the time phases T1 to T4. Therefore, the following reference images are generated from the three cross-sectional images selected from the volume data of each of the time phases T1 to T4. Phase T1 (before contrast): Reference images I[1,1] ~ I[1,3] Time phase T2 (arterial phase): Reference images I[2,1] ~ I[2,3] Phase T3 (portal vein delivery): Reference images I[3,1] ~ I[3,3] Time phase T4 (late phase): Reference image I[4,1] ~I[4,3]

[0049] The image feature data creation unit 34 calculates the image feature F of each image of the region of interest (i.e., the base image and multiple reference images). For example, the image feature F[0] is calculated for the base image I[0], and the image feature F[1,1] is calculated for the reference image I[1,1]. The same applies to the other reference images. Note that the image feature F is calculated by any method, as described above.

[0050] The image feature data creation unit 34 creates image feature data G for each image of the region of interest by adding information representing the time phase and the position in the Z direction to the image feature F. For example, for the base image I[0], image feature data G[0] is created by adding information representing the time phase (T0) and the position in the Z direction (Z[0]) to the image feature F[0]. For the reference image I[1,1], image feature data G[1,1] is created by adding information representing the time phase (T1) and the position in the Z direction (Z[1,1]) to the image feature F[1,1]. The same process is performed for the other reference images. As a result, the base image feature data G[0] and 12 reference image feature data G[1,1] to [4,3] are created.

[0051] Although not particularly limited, the image feature data G preferably has a predetermined data length. In this case, it is preferable that the image feature amount F, the information representing the time phase, and the information representing the position in the Z direction each have a predetermined data length.

[0052] 8 is a flowchart showing an example of a procedure for creating image feature data. The procedure shown in FIG. 8 corresponds to S1 to S4 shown in FIG.

[0053] In S11, the image feature data creation unit 34 reads a reference cross-sectional image. The reference cross-sectional image refers to a reference cross-sectional image in which a lesion appears most clearly among multiple cross-sectional images captured at a reference time phase. The reference time phase refers to a time phase in which a lesion is likely to appear most clearly on a cross-sectional image. In S12, the image feature data creation unit 34 detects a lesion in the reference cross-sectional image. Note that the processes of S11 to S12 may be performed by a person (i.e., a doctor involved in the MRI examination), for example. In this case, the image feature data creation unit 34 performs the processes of S11 to S12 in accordance with instructions from the doctor. Alternatively, the image feature data creation unit 34 may automatically perform the processes of S11 to S12.

[0054] In S13, the image feature data creation unit 34 sets a reference region of interest including the lesion in the reference cross-sectional image. The reference region of interest is preferably rectangular, for example. The reference region of interest may also be specified by a doctor. In this case, the image feature data creation unit 34 sets the reference region of interest in accordance with instructions from the doctor. Alternatively, the image feature data creation unit 34 may automatically set the reference region of interest.

[0055] In S14, the image feature data creation unit 34 cuts out an image within the reference region of interest from the reference cross-sectional image. In the following description, the image within the reference region of interest cut out from the reference cross-sectional image may be referred to as a "reference image."

[0056] The processes of S15 to S17 are executed for each phase other than the reference phase. In the following description, the phase for which the processes of S15 to S17 are executed may be referred to as the "processing target phase."

[0057] In S15, the image feature data creation unit 34 reads M cross-sectional images taken near the position where the reference cross-sectional image was taken, among the multiple cross-sectional images taken in the processing target time phase. In S16, the image feature data creation unit 34 sets a reference region of interest in each of the M cross-sectional images read in S15. The position and shape of each reference region of interest are the same as the position and shape of the reference region of interest. In S17, the image feature data creation unit 34 cuts out an image within the reference region of interest from each of the M cross-sectional images read in S15. In the following description, the image within the reference region of interest cut out from the cross-sectional image read in S15 may be referred to as a "reference image." That is, in S15 to S17, M reference images are obtained for each time phase other than the reference time phase.

[0058] In S18, the image feature data creation unit 34 calculates image feature amounts for each of the reference image and each of the reference images. In S19, the image feature data creation unit 34 adds phase information to the image feature amounts calculated in S18. Specifically, phase information indicating the reference phase is added to the image feature amounts of the reference image. Furthermore, phase information indicating the phase at which a cross-sectional image including the reference image was captured is added to the image feature amounts of each of the reference images. In S20, the image feature data creation unit 34 adds position information to the image feature amounts calculated in S18. The position information indicates the position in the Z direction at which a cross-sectional image including an image represented by the image feature amount was captured. In this way, image feature data is created by adding phase information and position information to the image feature amounts for each of the reference image and each of the reference images.

[0059] Returning to the explanation of the flowchart shown in FIG. 4, in S5, the image feature data creation unit 34 inputs the set of image feature data created in S4 to the discrimination model 35. The set of image feature data is composed of reference image feature data created based on a reference image acquired at the reference time phase and reference image feature data created based on each reference image acquired at each of the other time phases. The discrimination model 35 has already learned the correspondence between image feature data previously input and actual discrimination results. Here, the "actual discrimination result" is, for example, information indicating benign or malignant obtained by collecting cells from a lesion of a subject and conducting an examination. In other words, the discrimination model 35 is trained by providing "correct answers (i.e., training data)" to image feature data of past subjects.

[0060] The discriminant model 35 processes the input image feature data to discriminate whether a lesion is benign or malignant, and may generate information representing the probability that the lesion is benign (or the probability that the lesion is malignant).

[0061] In S6, lesion identification unit 33 outputs the identification result obtained by identification model 35. The identification result is displayed on display device 40. The identification result may indicate whether the lesion is benign or malignant, or may indicate the probability that the lesion is benign or the probability that the lesion is malignant.

[0062] 9 shows an example of processing by the discriminant model 35. The discriminant model 35 is a model that processes time-series data, and is realized by, for example, a neural network including a transformer model, although it is not particularly limited thereto.

[0063] A set of image feature data created by the image feature data creation unit 34 is input to the identification model 35. In the example shown in Fig. 9, the base image feature data G[0] and the reference image feature data G[1,1] to G[4,3] are input to the identification model 35. Note that G[p,m] represents reference image feature data created based on the m-th cross-sectional image among M cross-sectional images when M cross-sectional images are selected from a plurality of cross-sectional images captured at the time phase Tp.

[0064] As described above, each image feature data is composed of an image feature amount, phase information indicating a phase, and position information indicating a position in the Z direction. The discrimination model 35 then multiplies each image feature data G by a corresponding weight W. Each weight W[0] and W[1,1] to W[4,3] is determined by learning.

[0065] The discriminative model 35 processes the image feature data G multiplied by the corresponding weight W in a neural network. The neural network includes, for example, an input layer, one or more hidden layers, and an output layer. Each layer includes a plurality of nodes. One or more image feature data are input to each node in the input layer. At this time, each input data may be multiplied by a coefficient (or weight) at each node in the input layer. Data generated at one or more nodes in the input layer is input to each node in the hidden layer. At this time, each input data may be multiplied by a coefficient at each node in the hidden layer. Data generated at one or more nodes in the hidden layer is input to each node in the output layer. At this time, each input data may be multiplied by a coefficient at each node in the output layer.

[0066] In this example, the output layer includes a benign node corresponding to benign and a malignant node corresponding to malignant. The output of the benign node, for example, takes a value between 0 and 1 and represents the probability that the subject's lesion is benign. The output of the malignant node, for example, takes a value between 0 and 1 and represents the probability that the subject's lesion is malignant.

[0067] The discrimination model 35 is trained using image feature data based on MRI images of past subjects and corresponding training data. The image feature data based on MRI images of past subjects is created according to the procedure in the flowchart shown in Figure 8, just like the image feature data based on MRI images of new subjects. The training data is, for example, information indicating whether a lesion is benign or malignant, obtained by collecting cells from the subject's lesion and conducting an examination.

[0068] The learning method is not particularly limited, but may be, for example, backpropagation. In this case, the coefficients (or weights) used in each node of the neural network are updated so that the output of the discriminant model 35 for input data (i.e., image feature data based on past MRI images of the subject) approaches the training data. In addition, the weight W multiplied by each image feature data in FIG. 9 may also be updated so that the output of the discriminant model 35 approaches the training data.

[0069] Furthermore, the discrimination model 35 is preferably configured to achieve the following functions: (1) Ability to identify lesions even when some image feature data is missing due to inadequate imaging of the subject in any phase. (2) Ability to identify lesions even if the patient moves during imaging

[0070] As described above, the identification model 35 processes image feature data corresponding to multiple time phases designated according to the type of examination. For example, in a contrast-enhanced MRI examination, image feature data corresponding to five time phases (pre-contrast, arterial phase, portal venous phase, late phase, and hepatic enhancement phase) is processed. However, depending on the condition of the subject, appropriate imaging may not be performed in all time phases. In other words, image feature data corresponding to any of the time phases may not be input to the identification model 35.

[0071] In an embodiment of the present invention, each image feature data includes information representing a time phase. In the example shown in Figure 7, for example, the reference image feature data G[0] includes "T0" representing the hepatic enhancement phase as time phase information. Furthermore, the reference image feature data G[1,1] to G[1,3] include "T1" representing the pre-enhancement phase as time phase information.

[0072] When image feature data corresponding to one of the time phases is not input, the identification model 35 identifies the lesion based on image feature data corresponding to the other time phases. For example, in a case where imaging is performed in five time phases (pre-contrast, arterial phase, portal venous phase, late phase, and hepatic enhancement phase) in a contrast-enhanced MRI examination, by referring to similar past cases, it is possible to estimate how the image of the lesion changes over time when a contrast agent is administered. Therefore, for example, if an image of the late phase is missing, the image of the late phase can be estimated based on the changes in the images obtained in the other four time phases. Alternatively, if an image of the late phase is missing, it is also possible to generate an image of the late phase by interpolation processing from an image of the portal venous phase, which is the phase immediately before the late phase, and an image of the hepatic enhancement phase, which is the phase immediately after the late phase.

[0073] In this context, it is preferable that the discrimination model 35 be configured to perform the above-described estimation or interpolation using time phase information added to each image feature data. However, in this embodiment, image feature quantities representing image features, rather than the images themselves, are input to the discrimination model 35. Therefore, the discrimination model 35 may be configured to use the time phase information added to each image feature quantity to implement a procedure for estimating image feature quantities for time phases in which image feature data is missing, or a procedure for regenerating image feature quantities for time phases in which image feature data is missing through interpolation processing. With this configuration, even when image feature data corresponding to any time phase is missing, it is possible to appropriately identify lesions in the subject, just as in the case where image feature data corresponding to all time phases is input.

[0074] Furthermore, the discrimination model 35 may be trained without inputting image feature data corresponding to any of the time phases. For example, the discrimination model 35 is trained by providing image feature data and training data for other time phases (pre-contrast, portal venous, late, and hepatic enhancement phases) without inputting image feature data corresponding to the arterial phase. In this case, even if image feature data corresponding to the arterial phase is missing in a contrast-enhanced MRI examination of a new subject, lesions can be identified with a certain degree of accuracy. The same applies to the other time phases. That is, the discrimination model 35 may be trained without inputting image feature data corresponding to the pre-contrast, portal venous, or late phases.

[0075] On the other hand, as described above, the lesion identification unit 33 identifies lesions in the subject using the reference image feature data created for the reference time phase and the reference image feature data created for other time phases. Here, as described with reference to FIG. 6, the reference image feature data is created for each of M reference cross-sectional images selected from multiple cross-sectional images. As described with reference to FIG. 6, the selected M reference cross-sectional images are captured near the positions where the reference cross-sectional images in which the lesion clearly appears were captured. Therefore, even if the subject moves during imaging, it is likely that the lesion will appear in one of the selected M reference cross-sectional images. Therefore, even if the subject moves during imaging, the identification model 35 can accurately identify the lesion in the subject.

[0076] As the M reference cross-sectional images, cross-sectional images captured near the position in the Z direction of the reference cross-sectional image are used. In the example shown in Fig. 6, M is 3, and the position in the Z direction of the reference cross-sectional image (i.e., the reference position) is 1210. Then, as the M cross-sectional images, a cross-sectional image V1 captured at the Z coordinate closest to the reference position, a cross-sectional image V2 captured at the Z coordinate second closest to the reference position, and a cross-sectional image V3 captured at the Z coordinate third closest to the reference position are selected.

[0077] Here, if the subject does not move during imaging, the lesion is most likely to appear in the cross-sectional image captured at the Z coordinate closest to the reference position. Similarly, the lesion is second most likely to appear in the cross-sectional image captured at the Z coordinate second closest to the reference position, and the lesion is third most likely to appear in the cross-sectional image captured at the Z coordinate third closest to the reference position. Therefore, it is preferable that the identification model 35 is configured so that the image feature data corresponding to the cross-sectional image closer to the reference position is weighted more heavily, and the image feature data corresponding to the cross-sectional image farther from the reference position is weighted less heavily. This weight is determined based on the position information (i.e., information representing the position in the Z direction) included in each image feature data. Furthermore, this weight may be multiplied by the image feature data as "W" shown in FIG. 9.

[0078] With this configuration, if a reference cross-sectional image in which a lesion clearly appears in the reference time phase is selected, a greater weight is assigned to a reference cross-sectional image in which a lesion is more likely to appear in other time phases.The discrimination model 35 then discriminates lesions using image feature data corresponding to the reference cross-sectional image and image feature data corresponding to the reference cross-sectional image weighted according to the probability of the lesion appearing.This increases the probability of accurately discriminating lesions even if the subject moves during imaging.

[0079] <Hardware configuration> 10 shows an example of the hardware configuration of the operation terminal 30. The operation terminal 30 is realized by a computer 100 including, for example, a processor 101, a memory 102, a storage device 103, an input / output device 104, a recording medium reader 105, and a communication interface 106.

[0080] The processor 101 controls the operation of the operation terminal 30 by executing a medical image processing program stored in the storage device 103. The medical image processing program includes program code that describes the procedures of the flowcharts shown in FIGS. 4 and 8. Therefore, when the processor 101 executes this program, the functions of the examination image acquisition unit 31 and the lesion identification unit 33 (image feature data creation unit 34 and identification model 35) shown in FIG. 3 are provided. The memory 102 is used as a working area for the processor 101. The storage device 103 stores the medical image processing program and other programs. The storage device 103 is used to implement the storage device 32.

[0081] The input / output device 104 includes input devices such as a keyboard, a mouse, a touch panel, and a microphone. The input / output device 104 also includes output devices such as a display device and a speaker. The display device 40 shown in FIG. 3 may be an example of the input / output device 104. The recording medium reader 105 can acquire data and information recorded on the recording medium 110. The recording medium 110 is a removable recording medium that can be attached to or detached from the computer 100. The recording medium 110 may be realized by, for example, a semiconductor memory, a medium that records signals optically, or a medium that records signals magnetically. The medical image processing program may be provided to the computer 100 from the recording medium 110. The communication interface 106 provides a function for connecting to a network. When the medical image processing program is stored in the program server 120, the computer 100 may acquire the medical image processing program from the program server 120. [Explanation of symbols]

[0082] 1. Diagnostic support system 10 Imaging equipment 20 Inspection image server 30 Operation terminal 31 Inspection image acquisition unit 32 Storage section 33 Lesion Identification Unit 34 Image feature data creation section 35 Identification Model 40 Display device 100 computers 101 processors

Claims

1. acquiring a plurality of cross-sectional images obtained by photographing a plurality of cross sections of the subject at each of a plurality of time phases; creating reference image feature data by adding information representing the first time phase to feature amounts of at least a part of a reference cross-sectional image captured at a first cross-sectional position in a first time phase among the plurality of time phases; selecting a reference cross-sectional image captured in the vicinity of the first cross-sectional position from a plurality of cross-sectional images captured in a second time phase among the plurality of time phases; creating reference image feature data by adding information representing the second time phase to at least a portion of feature amounts of the selected reference cross-sectional image; The standard image feature data and the reference image feature data are input into a trained model, and information relating to the lesion of the subject is output. A medical image processing method comprising:

2. In the step of selecting the reference cross-sectional images, a plurality of reference cross-sectional images captured in the vicinity of the first cross-sectional position are selected from a plurality of cross-sectional images captured in the second time phase, in the step of creating the reference image feature data, a plurality of reference image feature data are created by adding information representing the second time phase to at least a portion of feature amounts of each of the selected plurality of reference cross-sectional images; In the step of outputting information related to a lesion of the subject, The reference image feature data and the plurality of reference image feature data are input into a trained model; The trained model outputs information related to a lesion of the subject based on the standard image feature data and the plurality of reference image feature data.

2. The medical image processing method according to claim 1.

3. The trained model is trained so that the classification results for the standard image feature data and the reference image feature data created for past subjects approach the actual classification results for the lesions of the past subjects.

2. The medical image processing method according to claim 1.

4. The information relating to the lesion of the subject indicates whether the lesion of the subject is benign or malignant.

2. The medical image processing method according to claim 1.

5. the reference image feature data includes position information representing the first cross-sectional position; The reference image feature data includes position information indicating a cross-sectional position at which the reference cross-sectional image was captured.

2. The medical image processing method according to claim 1.

6. the first time phase is a time phase in which a lesion of the subject appears most clearly in the captured cross-sectional image among the plurality of time phases; The reference image feature data is a reference region of interest including a lesion of the subject is set in the reference cross section image; the reference region of interest is created by adding information representing the first time phase to a feature amount of an image within the reference region of interest, The reference image feature data is a reference region of interest having the same position and shape as the reference region of interest is set in the reference cross-sectional image; The second time phase is generated by adding information representing the second time phase to the feature amount of the image in the reference region of interest.

2. The medical image processing method according to claim 1.

7. acquiring a plurality of cross-sectional images obtained by photographing a plurality of cross sections of the subject at each of a plurality of time phases; creating reference image feature data by adding information representing the first time phase to feature amounts of at least a part of a reference cross-sectional image captured at a first cross-sectional position in a first time phase among the plurality of time phases; selecting a reference cross-sectional image captured in the vicinity of the first cross-sectional position from a plurality of cross-sectional images captured in a second time phase among the plurality of time phases; creating reference image feature data by adding information representing the second time phase to at least a portion of feature amounts of the selected reference cross-sectional image; The standard image feature data and the reference image feature data are input into a trained model, and information relating to the lesion of the subject is output. A medical image processing program that causes a computer to perform processing.

Images