Information processing apparatus, information processing method, and program

Abstract

To improve the extraction accuracy in extracting a region of a predetermined organ from a medical image, and the calculation accuracy of various measurements based on the region.SOLUTION: An information processing apparatus includes an acquisition unit, a calculation unit, and a correction unit. The acquisition unit acquires the same structure from a plurality of medical images captured at different times or by different devices. The calculation unit calculates feature quantities related to a form of the structure for each of the medical images. The correction unit corrects the structure such that a difference in the feature quantities is reduced.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] When formulating a treatment plan using medical images, a region of a predetermined organ is extracted from the medical images, and various measurement values are calculated from the region. For example, a region of an organ can be extracted from medical images by machine learning or the like, and various measurement values can be calculated from the extracted region.

[0003] However, in conventional organ segmentation processing using machine learning or the like, even when an organ has anatomical features that are temporally invariant in terms of shape, the features regarding those invariances are not fully considered, so there are cases where results conflicting with the anatomical features are calculated.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] One of the problems that the embodiments disclosed in this specification and the drawings seek to solve is to improve the extraction accuracy when extracting the region of a predetermined organ from a medical image and the calculation accuracy of various measurement values based on the region. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. It is also possible to position the problems corresponding to the respective effects of the respective configurations shown in the embodiments described later as other problems.

Means for Solving the Problems

[0006] The information processing apparatus according to the embodiment includes an acquisition unit, a calculation unit, and a correction unit. The acquisition unit acquires the same structure from a plurality of medical images captured at different times or by different apparatuses. The calculation unit calculates a feature amount related to the form of the structure for each of the plurality of medical images. The correction unit corrects the structure so that the difference in the feature amounts becomes small.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0008] Hereinafter, embodiments of an information processing apparatus, an information processing method, and a program will be described in detail with reference to the drawings.

[0009] (First Embodiment) FIG. 1 shows a configuration example of an information processing apparatus 140 according to the embodiment. In the case of FIG. 1, the case where the information processing apparatus 140 according to the embodiment is a part of the X-ray CT apparatus 101 is described. Note that the information processing apparatus 140 is not limited to being a part of the X-ray CT apparatus 101, and may be a part of a medical image diagnostic apparatus other than the X-ray CT apparatus, such as an ultrasonic diagnostic apparatus or a magnetic resonance imaging apparatus. As another example, the information processing apparatus 140 may be configured independently of these medical image diagnostic apparatuses.

[0010] As shown in FIG. 1, the X-ray CT apparatus 101 according to the embodiment includes a gantry apparatus 110, a couch apparatus 130, and an information processing apparatus 140. Note that FIG. 1 depicts the gantry apparatus 110 from multiple directions for explanatory purposes and shows the case where the X-ray CT apparatus 101 has one gantry apparatus 110.

[0011] The gantry apparatus 110 includes an X-ray tube 111, an X-ray detector 112, a rotating frame 113, an X-ray high voltage apparatus 114, a control apparatus 115, a wedge 116, a collimator 117, and a DAS (Data Acquisition System) 118.

[0012] The X-ray tube 111 is a vacuum tube having a cathode (filament) that generates thermoelectrons and an anode (target) that generates X-rays upon receiving the collision of thermoelectrons. The X-ray tube 111 generates X-rays to irradiate the subject P by irradiating thermoelectrons from the cathode toward the anode by applying a high voltage from the X-ray high voltage device 114. For example, the X-ray tube 111 includes a rotating anode type X-ray tube that generates X-rays by irradiating the rotating anode with thermoelectrons.

[0013] Note that the X-ray tube 111 and the control device 115 are an example of an X-ray irradiation unit. The X-ray irradiation unit performs a low-flux scan on a phantom composed of a known substance and a transmission length. Specifically, the X-ray irradiation unit performs a low-flux scan by performing an air scan and a scan on a phantom composed of a plurality of different substances at an initial current intensity and each tube voltage setting of the X-ray tube.

[0014] The rotating frame 113 is an annular frame that oppositely supports the X-ray tube 111 and the X-ray detector 112 and rotates the X-ray tube 111 and the X-ray detector 112 by the control device 115. For example, the rotating frame 113 is a casting made of aluminum. Note that the rotating frame 113 can further support the X-ray high voltage device 114, the wedge 116, the collimator 117, the DAS 118, etc. in addition to the X-ray tube 111 and the X-ray detector 112. Further, the rotating frame 113 can further support various configurations not shown in FIG. 1.

[0015] The wedge 116 is a filter for adjusting the X-ray dose irradiated from the X-ray tube 111. Specifically, the wedge 116 is a filter that transmits and attenuates the X-rays irradiated from the X-ray tube 111 so that the distribution of the X-rays irradiated from the X-ray tube 111 to the subject P becomes a predetermined distribution. For example, the wedge 116 is a wedge filter or a bow-tie filter, and is a filter obtained by processing aluminum or the like to have a predetermined target angle and a predetermined thickness.

[0016] The collimator 117 is a lead plate or the like for narrowing down the irradiation range of the X-rays that have passed through the wedge 116, and forms a slit by combining a plurality of lead plates or the like. Note that the collimator 117 may also be referred to as an X-ray aperture. Also, in FIG. 1, the case where the wedge 116 is arranged between the X-ray tube 111 and the collimator 117 is shown, but the collimator 117 may be arranged between the X-ray tube 111 and the wedge 116. In this case, the wedge 116 transmits and attenuates the X-rays irradiated from the X-ray tube 111 and whose irradiation range is limited by the collimator 117.

[0017] The X-ray high-voltage device 114 has electric circuits such as a transformer and a rectifier, and includes a high-voltage generating device that generates a high voltage to be applied to the X-ray tube 111, and an X-ray control device that controls the output voltage according to the X-rays generated by the X-ray tube 111. The high-voltage generating device may be of a transformer type or an inverter type. Note that the X-ray high-voltage device 114 may be provided on the rotating frame 113 or may be provided on a fixed frame (not shown).

[0018] The control device 115 has a processing circuit having a CPU (Central Processing Unit) or the like, and a drive mechanism such as a motor and an actuator. The control device 115 receives an input signal from the input interface 143 and controls the operations of the gantry device 110 and the bed device 130. For example, the control device 115 controls the rotation of the rotating frame 113, the tilt of the gantry device 110, the operations of the bed device 130 and the top plate 133, and the like. Note that the control device 115 may be provided on the gantry device 110 or may be provided on the information processing device 140.

[0019] The X-ray detector 112 is, for example, a photon-counting type detector or an energy-integrating type detector. When the X-ray detector 112 is a photon-detecting type detector, each time an X-ray photon, which is a photon derived from the X-rays irradiated from the X-ray tube 111 and transmitted through the subject P, enters, the X-ray detector 112 outputs a signal capable of measuring the energy value of the X-ray photon. The X-ray detector 112 has a plurality of X-ray detection elements that output an electric signal (analog signal) of one pulse each time an X-ray photon enters.

[0020] The X-ray detection element is, for example, a semiconductor element (semiconductor detection element) such as CdTe (cadmium telluride) or CdZnTe (cadmium zinc telluride), with an anode electrode and a cathode electrode arranged thereon.

[0021] The X-ray detector 112 has a plurality of X-ray detection elements and an ASIC (Application Specific Integrated Circuit), which is a readout circuit connected to the X-ray detection elements and counts the X-ray photons detected by the X-ray detection elements. The ASIC counts the number of X-ray photons incident on the detection elements by discriminating the individual charges output by the X-ray detection elements. Further, the ASIC measures the energy of the counted X-ray photons by performing arithmetic processing based on the magnitude of the individual charges. Furthermore, the ASIC outputs the counting result of the X-ray photons as digital data to the DAS 118.

[0022] The DAS 118 generates detection data based on the result of the counting process input from the X-ray detector 112. The detection data is, for example, a sinogram. The sinogram is data obtained by arranging the results of the counting process incident on each X-ray detection element at each position of the X-ray tube 111. The sinogram is data obtained by arranging the results of the counting process in a two-dimensional orthogonal coordinate system with the view direction and the channel direction as axes. The DAS 118 generates a sinogram, for example, in units of columns in the slice direction in the X-ray detector 112. The DAS 118 transfers the generated detection data to the information processing device 140. The DAS 118 is realized, for example, by a processor.

[0023] The data generated by DAS118 is transmitted from a transmitter having a light-emitting diode (LED) provided in the rotating frame 113 to a receiver having a photodiode provided in a non-rotating portion (e.g., a fixed frame or the like; illustration in FIG. 1 is omitted) of the gantry device 110 by optical communication, and then transferred to the information processing device 140. Here, the non-rotating portion is, for example, a fixed frame that rotatably supports the rotating frame 113. Note that the method of transmitting data from the rotating frame 113 to the non-rotating portion of the gantry device 110 is not limited to optical communication, and any non-contact data transmission method or a contact-type data transmission method may be adopted.

[0024] The bed device 130 is a device for placing and moving a subject P to be photographed, and includes a base 131, a bed driving device 132, a top plate 133, and a support frame 134. The base 131 is a housing that supports the support frame 134 so as to be movable in the vertical direction. The bed driving device 132 is a driving mechanism that moves the top plate 133 on which the subject P is placed in the longitudinal direction of the top plate 133, and includes a motor, an actuator, and the like. The top plate 133 provided on the upper surface of the support frame 134 is a plate on which the subject P is placed. Note that the bed driving device 132 may move the support frame 134 in the longitudinal direction of the top plate 133 in addition to the top plate 133.

[0025] The information processing device 140 includes a memory 141, a display 142, an input interface 143, and a processing circuit 144. Note that the information processing device 140 is described as a separate body from the gantry device 110, but a part of each component of the information processing device 140 may be included in the gantry device 110.

[0026] The memory 141 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, etc. The memory 141 stores, for example, projection data and CT image data. Also, for example, the memory 141 stores a program for the circuits included in the X-ray CT apparatus 101 to realize various functions. The memory 141 may be realized by a server group (cloud) connected to the X-ray CT apparatus 101 via a network.

[0027] The display 142 displays various kinds of information. For example, the display 142 displays various images generated by the processing circuit 144, or displays a GUI (Graphical User Interface) for receiving various operations from the operator. For example, the display 142 is a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 142 may be of a desktop type, or may be configured as a tablet terminal or the like that can communicate wirelessly with the information processing apparatus 140 main body. Also, the display 142 is an example of a display unit.

[0028] The input interface 143 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 144. Also, for example, the input interface 143 receives input operations from the operator such as scan conditions, reconstruction conditions when reconstructing CT image data, and image processing conditions when generating a post-processed image from CT image data.

[0029] For example, the input interface 143 is realized by a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs an input operation by touching an operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input circuit using an optical sensor, an audio input circuit, or the like. Note that the input interface 143 may be provided in the gantry device 110. Further, the input interface 143 may be configured by a tablet terminal or the like capable of wireless communication with the information processing device 140 main body. Further, the input interface 143 is not limited to those having physical operation components such as a mouse and a keyboard. For example, an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the information processing device 140 and outputs this electric signal to the processing circuit 144 is also included in the example of the input interface 143.

[0030] The processing circuit 144 controls the operation of the entire X-ray CT apparatus 101. For example, the processing circuit 144 executes a control function 144a, a preprocessing function 144b, an acquisition function 144c, a setting function 144d, a calculation function 144e, and a correction function 144f. Here, for example, each processing function executed by the control function 144a, the preprocessing function 144b, the acquisition function 144c, the setting function 144d, the calculation function 144e, and the correction function 144f, which are components of the processing circuit 144 shown in FIG. 1, is recorded in the memory 141 in the form of a program executable by a computer. The processing circuit 144 is, for example, a processor, and reads out each program from the memory 141 and executes it to realize the function corresponding to each read program. In other words, the processing circuit 144 in the state of having read out each program has each function shown in the processing circuit 144 of FIG. 1.

[0031] The control function 144a, the preprocessing function 144b, the acquisition function 144c, the setting function 144d, the calculation function 144e, and the correction function 144f are each an example of a control unit, a preprocessing unit, an acquisition unit, a setting unit, a calculation unit, and a correction unit. Further, the control unit is an example of a display control means. Further, the memory 141 is an example of a storage unit.

[0032] In FIG. 1, the case where each processing function of the control function 144a, the preprocessing function 144b, the acquisition function 144c, the setting function 144d, the calculation function 144e, and the correction function 144f is realized by a single processing circuit 144 is shown. However, the embodiment is not limited to this. For example, the processing circuit 144 may be configured by combining a plurality of independent processors, and each processor may realize each processing function by executing each program. Further, each processing function included in the processing circuit 144 may be appropriately distributed or integrated into a single or a plurality of processing circuits and realized.

[0033] The control function 144a controls various processes based on an input operation received from an operator via the input interface 143. Specifically, the control function 144a controls the CT scan performed by the gantry device 110. For example, the control function 144a controls the operations of the X-ray high voltage device 114, the X-ray detector 112, the control device 115, the DAS 118, and the bed driving device 132 to control the collection process of the count results in the gantry device 110. As an example, the control function 144a controls the collection process of projection data in the positioning scan for collecting a positioning image (scanogram) and the imaging (main scan) for collecting an image used for diagnosis, respectively.

[0034] Further, as display control means, the control function 144a causes the display 142 to display an image or the like based on various image data stored in the memory 141.

[0035] The preprocessing function 144b generates projection data by performing preprocessing such as logarithmic conversion processing, offset correction processing, sensitivity correction processing between channels, beam hardening correction, scattered ray correction, and dark count correction on the detection data output from the DAS 118.

[0036] The processing circuit 144 acquires various data from the X-ray detector 112 by the acquisition function 144c. The details of each function of the setting function 144d, the calculation function 144e, and the correction function 144f will be described later.

[0037] The information processing apparatus according to the embodiment includes an acquisition unit, a calculation unit, and a correction unit. The acquisition unit acquires the same structure from a plurality of medical images captured at different times or by different devices. The calculation unit calculates, for each of the plurality of medical images, a feature amount related to the form of the structure. The correction unit corrects the structure so that the difference in the feature amount becomes small.

[0038] Also, the information processing method according to the embodiment acquires the same structure from a plurality of medical images captured at different times or by different devices, calculates, for each of the plurality of medical images, a feature amount related to the form of the structure, and corrects the structure so that the difference in the feature amount becomes small.

[0039] Also, the program according to the embodiment causes a computer to execute a process of acquiring the same structure from a plurality of medical images captured at different times or by different devices, calculating, for each of the plurality of medical images, a feature amount related to the form of the structure, and correcting the structure so that the difference in the feature amount becomes small.

[0040] That is, the information processing apparatus according to the embodiment performs correction processing so that, for example, the difference in the feature amounts at anatomically corresponding positions at different times becomes small. Note that the feature amount is, for example, the length of an organ. Thereby, it is possible to perform correction processing of time-series images while keeping the anatomical feature amounts in an appropriate form.

[0041] Subsequently, the processing performed by the information processing apparatus 140 according to the embodiment will be described with reference to FIGS. 2 to 8 as appropriate. In the following embodiments, the case of describing the processing of the mitral valve has been described, but the embodiment is not limited thereto, and processing may be performed on other types of organs.

[0042] First, in step S110, the processing circuit 144 acquires, by means of the acquisition function 144c, an X-ray CT image of a subject as a medical image from the X-ray CT apparatus 101 or the in-hospital image database connected to the information processing apparatus 140. As an example, the processing circuit 144 acquires, by means of the acquisition function 144c, a plurality of medical images captured at different times or by different apparatuses.

[0043] As an example, the processing circuit 144 acquires, by means of the acquisition function 144c, a plurality of CT images captured at different cardiac phases T = 1, T = 2, and T = 3 as medical images. As an example, the processing circuit 144 acquires, by means of the acquisition function 144c, a CT image 20 captured at cardiac phase T = 1 as shown in FIG. 4 described later.

[0044] Also, the processing circuit 144 acquires, by means of the acquisition function 144c, a CT image 20 captured at cardiac phase T = 1 as shown in FIG. 4 described later as one of the plurality of medical images.

[0045] Also, the processing circuit 144 acquires, by means of the acquisition function 144c, a CT image 30 captured at cardiac phase T = 2 as shown in FIG. 5 described later as one of the plurality of medical images. Also, the processing circuit 144 acquires, by means of the acquisition function 144c, a CT image 40 captured at cardiac phase T = 3 as shown in FIG. 6 described later as one of the plurality of medical images. That is, the processing circuit 144 acquires a 4DCT image by means of the acquisition function 144c.

[0046] Note that the degree of difference in the imaging times of these plurality of CT images acquired by the processing circuit 144 by means of the acquisition function 144c is arbitrary. For example, they may differ only for a short period of less than 1 second like the images of each cardiac phase in a 4DCT image, or conversely, there may be a difference of about 1 year in the imaging times. Also, the processing circuit 144 may acquire, as a plurality of medical images, images captured using, for example, an ultrasonic diagnostic apparatus and images captured using a magnetic resonance imaging apparatus, etc., by performing imaging with a plurality of modalities.

[0047] As for the start condition of step S110, as an example, the processing circuit 144 may acquire a CT image of the subject as the medical image by the acquisition function 144c triggered by receiving an instruction from the user. As another example, when the processing circuit 144 or another processing circuit monitors a storage device for medical images such as a PACS and detects that a new image has been stored in the storage device for medical images, the processing circuit 144 may start the processing of step S110.

[0048] As another example, the processing circuit 144 may determine whether the new image meets predetermined conditions, and if the determined conditions are met, the processing circuit 144 may start the processing of step S110. Examples of the determined conditions include conditions related to the imaging protocol, for example, the condition that the new image was taken according to an imaging protocol targeting the heart, and conditions related to the reconstruction method, for example, the condition that the new image is an image reconstructed by magnification.

[0049] Note that as already described, the medical image acquired by the processing circuit 144 in step S110 is not limited to an X-ray CT image, and may be another type of image in which morphological information of the three-dimensional anatomical structure of the target living tissue is stored. As an example, the medical image acquired by the processing circuit 144 in step S110 may be an ultrasonic image, an MRI image, an X-ray image, a PET image, or a SPECT image.

[0050] Subsequently, in step S120, the processing circuit 144 extracts the target structure by means of the acquisition function 144c. That is, the processing circuit 144, by means of the acquisition function 144c, respectively acquires a predetermined structure from a plurality of medical images captured at different times or by different devices, which are acquired in step S110. For example, the processing circuit 144, by means of the acquisition function 144c, acquires the shape information of an organ included in the medical image as a predetermined structure. That is, the processing circuit 144, by means of the acquisition function 144c, extracts, as a predetermined structure, the region indicating the target organ from the CT image acquired in step S110. For example, the processing circuit 144, by means of the acquisition function 144c, extracts the region of the mitral valve and acquires the coordinate information of the pixels in the region of the mitral valve. In step S120, the processing circuit 144 may also extract a predetermined structure by receiving, by means of the acquisition function 144c, a manual designation of the position of the target structure using the user interface, or may automatically extract a predetermined structure using a known region extraction technique. Examples of such region extraction techniques include, for example, Otsu's binarization method based on CT values, region growing method, snake method, graph cut method, mean shift method, and the like.

[0051] Note that the example of the extraction of the predetermined structure in step S120 is not limited to the above-described method. The processing circuit 144 may also extract and acquire the predetermined structure from a shape model constructed by learning, by means of the acquisition function 144c, learning data prepared in advance using a machine learning technique including deep learning.

[0052] Also, as another example, the processing circuit 144 extracts, by the acquisition function 144c, a related region that is larger than the region of the target organ but smaller than the region of the entire image acquired in step S110, and the above-described method may be used for the related region. Thereby, the computational cost required in step S120 can be reduced. As an example, when acquiring the mitral valve as a predetermined structure, the processing circuit 144 extracts, by the acquisition function 144c, the heart region as the related region, and acquires the predetermined structure using a known region extraction technique or the like for the related region. Also, as another example, the processing circuit 144 may extract, by the acquisition function 144c, for example, the region of the union of the left atrium region and the right atrium region as the related region, and acquire the predetermined structure using a known region extraction technique or the like for the related region. In setting the related region, the processing circuit 144 may receive an input from the user using the user interface by the acquisition function 144c to set the related region.

[0053] Also, the region indicating the target organ may be separately specified for each region having different features and characteristics within the region. As an example, in the case of extracting the mitral valve, since the mitral valve is composed of two valve leaflets, the anterior leaflet and the posterior leaflet, the processing circuit 144 may, by the acquisition function 144c, use the region of the anterior leaflet and the region of the posterior leaflet as the plurality of regions, and acquire the predetermined structure using a known region extraction technique or the like for each region.

[0054] FIG. 3 shows an example of a predetermined structure of the mitral valve region extracted by the processing circuit 144 by the acquisition function 144c at a certain time in step S120. Here, arrow 60 indicates the anterior leaflet portion, and arrow 61 indicates the posterior leaflet portion. Also, arrow 62 represents the annulus portion, which is the outermost portion in the mitral valve region, and arrow 63 represents the tip portion, which is the innermost portion in the mitral valve region. Also, arrow 64 represents the anterior commissure, and arrow 65 represents the posterior commissure.

[0055] In the example of FIG. 3, the processing circuit 144 will be described as extracting the mitral valve region as a mesh represented by a lattice point group of 19 columns and 9 rows for the anterior cusp region and a lattice point group of 25 columns and 9 rows for the posterior cusp region. If the position of each lattice point is represented by an identifier (x, y) where the row number is x and the column number is y as shown in FIG. 3, (8, 0) indicates the anterior commissure and (8, 18) indicates the posterior commissure. Also, the outermost part located at the anterior cusp and the posterior cusp, that is, the position where x = 0 is called the annulus part, and the innermost part, that is, the position where x = 8 is called the valve tip part.

[0056] Note that FIG. 3 is only an example of a predetermined structure extracted by the acquisition function 144c of the processing circuit 144 in step S120, and the number of lattice points, arrangement, array form, etc. may be different from the example shown in FIG. 3. Also, the processing circuit 144 may extract the mitral valve region not as a mesh represented by the above-described lattice point group, but as a three-dimensional image represented by a set of corresponding pixels.

[0057] Also, FIGS. 4 to 6 show an example of a predetermined structure extracted by the acquisition function 144c of the processing circuit 144 in step S120 at each cardiac phase. FIG. 4 shows the structure of the mitral valve extracted by the acquisition function 144c of the processing circuit 144 at the cardiac phase T = 1 and represented in a mesh-form data structure. Also, FIGS. 5 and 6 show the structure of the mitral valve extracted by the acquisition function 144c of the processing circuit 144 at the cardiac phases T = 2 and T = 3, respectively, and represented in a mesh-form data structure.

[0058] Subsequently, in step S130, the processing circuit 144 sets correction locations based on the predetermined structure extracted in step S120 by the setting function 144d. As an example, the processing circuit 144 extracts the same structure at anatomically corresponding positions from the predetermined structure extracted in step S120 by the setting function 144d, and then focuses on morphological information that remains unchanged even when the time changes in the target organ, for example, to set the correction locations. For example, in FIG. 4, the length of the broken line 21, that is, the distance from a predetermined position on the valve annulus of the valve tip to the position on the valve tip end, is the length of the organ when the organ is a heart valve, so it is considered to be unchanged even when the time changes. That is, the length of the broken line 21 at T = 1 shown in FIG. 4, the length of the broken line 31 at T = 2 shown in FIG. 5, and the length of the broken line 41 at T = 3 shown in FIG. 6 are considered to be approximately equal to each other because the broken lines 21, 31, and 41 are considered to be at anatomically corresponding positions of the organs of the same subject. Therefore, the processing circuit 144 sets the broken lines 21, 31, and 41 as correction locations by the setting function 144d.

[0059] Similarly, the processing circuit 144 sets the broken line 22 shown in FIG. 4, the broken line 32 shown in FIG. 5, and the broken line 42 shown in FIG. 6, which are considered to be the same structure at anatomically corresponding positions, as correction locations by the setting function 144d. Also, the processing circuit 144 sets the broken line 23 shown in FIG. 4, the broken line 33 shown in FIG. 5, and the broken line 43 shown in FIG. 6, which are considered to be anatomically corresponding positions, as correction locations by the setting function 144d. Here, the processing circuit 144 can identify corresponding positions among a plurality of images by processing such as template matching by the setting function 144d. The lattice points with the same identifier (x, y) in the target structure represented by the mesh are determined as anatomically corresponding positions, and the correction locations are set.

[0060] In addition, in the above example, the case where the shape of the correction portion is a broken line has been described. However, the embodiment is not limited to this, and the shape of the correction portion may be a line segment instead of a broken line. In this case, it is sufficient to specify only the endpoints to identify the shape of the correction portion. The embodiment is not limited to the case where the shape of the correction portion is one-dimensional, and the shape of the correction portion may be a two-dimensional closed curve or a three-dimensional region.

[0061] In this case, the processing circuit 144, by means of the setting function 144d, sets, as a set of closed curves or the like serving as the correction portion, each closed curve or the like at different times at anatomically corresponding positions of the organs of the same subject, which are considered to be invariant even when the time changes in the target organ.

[0062] Note that the setting of the correction portion in step S130 may be automatically set by the processing circuit 144 by means of the setting function 144d according to a predetermined criterion, or the processing circuit 144 may set the correction portion by means of the setting function 144d by receiving an input of the correction portion from the user through the user interface.

[0063] Instead of receiving an input of the correction portion from the user, the processing circuit 144 may receive an input of the range of the correction portion from the user by means of the setting function 144d. For example, in FIG. 4, when the processing circuit 144 receives an input of the range 24 of the correction portion from the user through the user interface by means of the setting function 144d, the processing circuit 144, by means of the setting function 144d, sets, as a set of lines serving as the correction portion, a set of line segments 21, 31, and 41 that are line segments included in the range 24 of the correction portion, a set of line segments 22, 32, and 42, and a set of line segments 23, 33, and 43.

[0064] If the range 24 of the correction location received from the user through the user interface by the setting function 144d of the processing circuit 144 is inappropriate, the processing circuit 144 may not be able to set the set of lines to be the correction location within the range 24 of the correction location by the setting function 144d. In such a case, the processing circuit 144 may display this fact to the user and request the user to re-enter the range 24 of the correction location, or alternatively, the processing circuit 144 may, by the setting function 144d, set the set of lines that is closest to the given conditions as the set of lines to be the correction location. In this way, the processing circuit 144 sets the correction location from the same structure by the setting function 144d.

[0065] Subsequently, in step S140, the processing circuit 144 sets an evaluation value to be used as a reference when correcting the correction location set in step S130 by the setting function 144d. As an example, the processing circuit 144 determines, by the setting function 144d, what feature amount to be based on when correcting the correction location set in step S130. Here, the processing circuit 144 performs correction using, as a feature amount, the morphological information that remains unchanged even when the time changes in the organ by the setting function 144d. As an example, when the type of the correction location set in step S130 is a line, the feature amount to be based on when performing correction is the length of the line. For example, the processing circuit 144 uses, by the setting function 144d, the length of the organ as a feature amount when correcting the correction location set in step S130, and performs correction based on the feature amount by the correction function 144f. For example, the processing circuit 144 performs correction using, as a feature amount, the distance from the position on the valve ring of the valve tip to the position on the valve tip end by the setting function 144d.

[0066] Also, as another example, the feature amount to be based on when performing correction may be an angle. As an example, the processing circuit 144 performs correction using, as a feature amount, the angle formed with a reference plane or reference line obtained from the structure of the valve tip end, for example, when correcting the correction location set in step S130 by the setting function 144d.

[0067] Also, when the type of the correction location set in step S130 is a closed curve, the original feature amounts used for correction are, for example, perimeter, area, circularity, etc. For example, when correcting the correction location set in step S130, the processing circuit 144 performs correction using, for example, the area or perimeter of the closed curve as a feature amount, by means of the setting function 144d.

[0068] Also, when the type of the correction location set in step S130 is a three-dimensional region, volume, surface area, sphericity, etc. become the feature amounts. In this case, when correcting the correction location set in step S130, the processing circuit 144 performs correction using, for example, volume, surface area, sphericity, etc. as feature amounts, by means of the setting function 144d.

[0069] The setting of the feature amount serving as the correction standard set in step S130 may be performed by the processing circuit 144 manually receiving an input from the user through the user interface by means of the setting function 144d. As another example, conditions may be set in advance so that the feature amount serving as the correction standard is automatically set. Also, the feature amount serving as the correction standard may be automatically determined according to the type of the correction location set in step S130.

[0070] Note that in the embodiment, the case where the process of step S140 is performed after the process of step S130 has been described, but the embodiment is not limited to this, and the process of step S130 may be performed after the process of step S140 has been performed. In this case, options that are not compatible with the correction standard selected by the user in step S140 may be excluded in the process of step S130.

[0071] Subsequently, in step S150, the processing circuit 144 corrects the correction location set in step S130 based on the correction standard set in step S140 by means of the correction function 144f.

[0072] Regarding the process of step S150, with reference to FIGS. 7 and 8, a case will be described where the type of correction location set in step S130 is a line segment, and the feature quantity serving as the correction reference set in step S140 is the length of an organ at anatomically corresponding positions. Here, in FIG. 7, line segments 21, 22, and 23 respectively correspond to line segments 21, 22, and 23 in FIGS. 4, 5, and 6. Line segment 21, line segment 22, and line segment 23 are data at T = 1, T = 2, and T = 3 respectively, and they are data corresponding to anatomically the same position.

[0073] Also, lattice points 10a to 10h represent the lattice points of line segment 21, lattice points 11a to 11h represent the lattice points of line segment 22, and lattice points 12a to 12h represent the lattice points of line segment 23. That is, lattice points 10a to 10h, 11a to 11h, and 12a to 12h respectively correspond to points in the meshes on line segment 21 in FIG. 4, line segment 31 in FIG. 5, and line segment 41 in FIG. 6.

[0074] Here, the processing circuit 144 calculates, by the calculation function 144e, the feature quantities regarding the form of the structure acquired from the medical image in step S110 for each of a plurality of times. As an example, the processing circuit 144 calculates, by the calculation function 144e, the length of the mitral valve as a feature quantity for each of a plurality of times. For example, as shown in FIG. 7, the processing circuit 144 calculates, by the calculation function 144e, the length of line segment 21 which is the length of the mitral valve at T = 1, the length of line segment 31 which is the length of the mitral valve at T = 2, and the length of line segment 41 which is the length of the mitral valve at T = 3 as feature quantities.

[0075] Also, as another example, the processing circuit 144 may calculate, by the calculation function 144e, the lengths of the broken lines constituting line segment 21 etc. for each of a plurality of times, and calculate the average value, median value, minimum value, or maximum value of the lengths of those broken lines as feature quantities.

[0076] Subsequently, the processing circuit 144 corrects the structure acquired from the medical image in step S110 so that the difference in the calculated feature amounts becomes small by the correction function 144f. As an example, the processing circuit 144 averages the feature amounts calculated for each of a plurality of times by the calculation function 144e, and corrects the structure acquired from the medical image in step S110 so that the feature amount at each time becomes close to the average value of the calculated feature amounts.

[0077] An example of such processing is shown in FIG. 8. FIG. 8 is a diagram showing an example of the correction processing performed by the processing circuit 144 by the correction function 144f. Here, the lattice points 53a, 53b, 53c, etc. indicate the lattice points before the correction processing, and the lattice points 51a, 51b, 51c, etc. indicate the lattice points after the correction processing. That is, the processing circuit 144 corrects the positions of the lattice points, for example, as shown by the lattice points 51a to 51c by the correction function 144f so that the feature amount at T = 1 becomes equal to the average value of the feature amounts of the same structure at T = 1, T = 2, and T = 3.

[0078] Also, as an optional component, the processing circuit 144 according to the embodiment may further perform additional processing to perform correction. For example, the processing circuit 144 may further re-arrange the lattice points so that the distances between the corrected lattice points are equally spaced for each of the lattice points 51a, 51b, 51c, etc. in FIG. 8 to update the positions of the plurality of lattice points. By re-arranging the lattice points so that the distances between the lattice points are equally spaced, the processing may be facilitated, for example, when performing some post-processing such as calculating a measured value.

[0079] Moreover, the embodiment is not limited to this, and the feature amount serving as the correction standard may be, for example, an angle. As an example, when the feature amount serving as the correction standard set in step S140 is an angle, the processing circuit 144, by the correction function 144f, makes the angle between the corrected lattice points at each time and a predetermined reference plane of the line segment 21 approach the average value at each time of the angle between the line segment 21 and the predetermined reference plane, and corrects the lattice points at each time. Here, the predetermined reference plane is, for example, the least-squares plane of the closed curve formed by the valve tip portion.

[0080] Also, the correction location set in step S130 may be a closed curve, and the feature amount serving as the correction standard set in step S140 may be the perimeter or area. As an example, the processing circuit 144, by the correction function 144f, corrects the lattice points at each time so that the closed curve calculated from the corrected lattice points at each time approaches the average value of the closed curves at each time.

[0081] Subsequently, in step S160, the processing circuit 144 determines, by a determination function (not shown), whether the correction of all the correction locations set by the setting function 144d in step S160 has been completed. When the correction process has been performed for all the correction locations set in step S140 (step S160 Yes), the correction process ends. On the other hand, when there is a location where the correction process has not been performed (step S160 No), the process returns to step S150, and the processing circuit 144 executes the correction process of step S150 for the set of correction locations for which the correction has not been completed by the correction function 144f.

[0082] Note that the processing circuit 144 usually performs the same type of correction process for all the correction locations set in step S130 by the correction function 144f, but the embodiment is not limited to this. The processing circuit 144 may change the processing of the correction method according to the correction location set in step S130 by the correction function 144f. As an example, the processing circuit 144 may change the method of the correction process executed in step S160 according to the position of the correction location by the correction function 144f.

[0083] Also, the embodiment is not limited to performing correction processing for all correction target locations. Among the correction locations set in step S130, correction processing may be performed only for some of the correction locations. As an example, the processing circuit 144 may further perform a process of determining whether to perform correction processing for each set of correction locations in the process of step S160. Also, as an example, for the set of correction locations set in step S130, the processing circuit 144 first calculates the correction reference in step S140 by the calculation function 144e. If the difference in the correction reference for the set of correction locations is equal to or less than the threshold value, it may not be necessary to perform correction processing for the correction location in step S140. Also, the threshold value may be set in advance, or the setting may be performed by receiving user input from the user interface. Also, as another example, in step S130 or step S140, the processing circuit 144 calculates the difference in the correction reference by the correction function 144f, and sets the sets with the difference in the correction reference equal to or less than the threshold value as not being correction locations in step S150.

[0084] As described above, in the first embodiment, the processing circuit 144 calculates, for each of the plurality of medical images, the feature amount related to the form of the same structure obtained from the plurality of medical images captured at different times or by different devices, and corrects the structure so that the difference in the feature amount becomes small. The processing circuit 144 performs correction processing so that corresponding feature amounts, such as the length of an organ, match at anatomically corresponding positions at different times, for example. Thereby, it is possible to perform correction processing of time-series images while keeping the anatomical feature amounts in a proper form.

[0085] (First Modification Example of the First Embodiment)

[0086] In the first embodiment, the processing circuit 144 sets a line or a plane that divides the structure obtained from the medical image into a plurality of regions by the setting function 144d, calculates a feature amount related to the form of the structure for each of the plurality of regions by the calculation function 144e, and corrects the structure so that the difference in the feature amount becomes small by the correction function 144f. However, the embodiment is not limited to this. The processing circuit 144 may calculate an evaluation value based on the form and shape of the structure by the calculation function 144e, and select a method for correcting the structure based on the calculated evaluation value by the correction function 144f.

[0087] The form and shape mentioned here are, for example, curvature. That is, the processing circuit 144 may calculate the curvature as an evaluation value for each of the plurality of divided regions by the correction function 144f, and select a method for correcting the structure based on the calculated evaluation value. As an example, when the curvature is large, there is a possibility that the reliability of the process in which the processing circuit 144 acquires the structure by the acquisition function 144c in step S120 is low. In this case, in step S150, for example, the processing circuit 144 corrects the structure with a higher curvature according to the structure with a lower curvature among the sets set for the correction locations in step S130 by the correction function 144f. That is, when the curvature of the first correction location is larger than the curvature of the second correction location among the sets set for the correction locations in step S140, the processing circuit 144 selects a method of correcting the first correction location according to the second correction location by the correction function 144f, and when the curvature of the first correction location is smaller than the curvature of the second correction location, selects a method of correcting the second correction location according to the first correction location. In other words, the processing circuit 144 selects a method for correcting the structure according to the reliability of the process of acquiring the structure by the correction function 144f.

[0088] Also, as another example, the processing circuit 144 may ignore the data at the time when the curvature of the correction location is high and perform correction based on the data at the time when the curvature of the correction location is small by the correction function 144f.

[0089] Also, as another example, when comparing with another set set at the correction location in step S130, if the difference in shape between the correction locations of the same set is large, the reliability of the process of obtaining the structure in step S120 in that set may be low. Therefore, the processing circuit 144 may correct the correction locations of the set with a large difference in shape between the correction locations according to the correction locations of the set with a small difference in shape between the correction locations by the correction function 144f.

[0090] Also, as another example, the processing circuit 144, by the calculation function 144e, for each of a plurality of times, when performing the process of step S120, for each pixel, calculates the likelihood of the structure obtained from the medical image for that pixel, that is, the probability that the pixel is the structure as an evaluation value, and may select a method of correcting the structure based on the calculated evaluation value by the correction function 144f. That is, the evaluation value calculated by the processing circuit 144 by the calculation function 144e is the likelihood of the structure. Here, the processing circuit 144 calculates the likelihood using, for example, U-Net by the calculation function 144e. It is considered that a correction location including more pixels with a high likelihood among the sets set at the correction locations has a higher reliability than the other correction location including more pixels with a low likelihood. Therefore, the processing circuit 144 corrects the other correction location according to the correction location including more pixels with a high likelihood by the correction function 144f.

[0091] As described above, in the first modification of the first embodiment, the processing circuit 144 calculates an evaluation value or the like, and further changes the method of the correction process based on the calculated evaluation value or the like. Thereby, the accuracy of the correction process is further improved.

[0092] (Second Modification of the First Embodiment) In the second modification of the first embodiment, a user interface for displaying the corrected image to the user will be described. FIG. 9 shows an example of such a user interface. When the correction of all correction points is completed in step S160, the processing circuit 144 causes the control function 144a to display an image 82 of the area including the corrected correction points on the display screen 80 of the display 142. Here, when the processing circuit 144 receives an instruction from the user to display in parallel the image before correction and the image after correction through the button 83 by the control function 144a, the processing circuit 144 causes the control function 144a to display the image 82 after correction on the display 142 together with the image 81 before correction. Further, when the processing circuit 144 receives an instruction from the user to superimpose and display the image before correction and the image after correction through the button 84 by the control function 144a, the processing circuit 144 causes the control function 144a to superimpose and display the image before correction and the image to be corrected on the display 142. That is, the processing circuit 144 causes the control function 144a to superimpose and display the medical image before correction and the medical image after correction on the display unit.

[0093] In addition, the processing circuit 144 causes the control function 144a to display on the display 142 a screen 87 for asking the user who refers to the corrected image 82 whether to accept the execution of the correction. When the processing circuit 144 receives a user's selection to accept the execution of the correction through the button 88 by the control function 144a, the processing circuit 144 accepts the image 82 after correction as a medical image for which the correction process has been correctly performed, and stores the image 82 after correction in the memory 141. On the other hand, when the processing circuit 144 receives a user's selection not to accept the execution of the correction through the button 89 by the control function 144a, the processing circuit 144 discards the image 82 after correction, ends the process, or changes the conditions and performs the correction process. In this case, the processing circuit 144 may receive an input of the changed conditions from the user by the control function 144a.

[0094] As described above, in the second modification of the first embodiment, the processing circuit 144 includes a user interface that performs processes such as displaying the corrected image to the user or receiving an input from the user. This improves usability.

[0095] (Third Modification of the First Embodiment) The embodiment is not limited to the above example. As an example, in step S160, the processing circuit 144 may calculate a feature amount or a measurement value regarding the corrected portion where correction has been performed by the calculation function 144e, and display the measurement value before correction and the measurement value after correction to the user. Also, as an example, as shown in FIG. 9, the processing circuit 144 causes the control function 144a to display the measurement value before correction on the display 142 as a message 85 to the user, and also causes the measurement value after correction to be displayed on the display 142 as a message 86 to the user.

[0096] Further, the processing circuit 144 causes the control function 144a to display, on the display 142, a screen 87 for asking the user whether to accept the implementation of the correction for the user who has referred to the message 86 to the user regarding the measurement value after correction. When the processing circuit 144 receives, through the button 88, the user's selection to accept the implementation of the correction by the control function 144a, the processing circuit 144 accepts the corrected image 82 as a medical image for which the correction process has been correctly performed, and stores the corrected image 82 in the memory 141. On the other hand, when the processing circuit 144 receives, through the button 89, the user's selection not to accept the implementation of the correction by the control function 144a, the processing circuit 144 discards the corrected image 82, ends the process, or changes the conditions and performs the correction process. In this case, the processing circuit 144 may receive the input of the changed conditions from the user by the control function 144a.

[0097] As described above, in the third modification of the first embodiment, the processing circuit 144 further displays the measurement value for the corrected portion where correction has been performed to the user. This improves usability.

[0098] (Fourth Modification of the First Embodiment) In the previous embodiments, the case where the target organ is the mitral valve has been described. However, the embodiments are not limited to this. The embodiments may be organs other than the heart, such as the brain, vocal cords, uterus, etc. In these organs, there are temporally invariant feature amounts. Therefore, the processing circuit 144 corrects the structure by the correction function 144f so that the difference in those feature amounts becomes small. As a result, the processing circuit 144 can similarly perform correction processing for organs other than the heart.

[0099] According to at least one of the embodiments described above, the image quality can be improved.

[0100] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations of the embodiments can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and its equivalent scope.

Description of Reference Numerals

[0101] 140 Information processing apparatus 141 Memory 142 Display 143 Input interface 144 Processing circuit 144a Control function 144b Preprocessing function 144c Acquisition function 144d Setting function 144e Calculation function 144f Correction function

Claims

1. an acquisition unit that acquires the same structure from a plurality of medical images captured at different times or by different devices; a calculation unit that calculates a feature amount related to a morphology of the structure for each of the plurality of medical images; a correction unit that corrects the structure so that the difference in the feature amount is reduced; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the acquisition unit acquires, as the structures, shape information at anatomically corresponding positions of organs of a same subject included in the plurality of medical images.

3. The information processing device according to claim 2 , wherein the feature amount is a length of the organ.

4. The information processing apparatus according to claim 2 , wherein the feature amount is morphological information that is invariant over time in the organ.

5. the organ is a heart valve, The information processing device according to claim 2 , wherein the feature amount is a distance from a position on the valve annulus to a position on the valve tip.

6. The correction unit calculates an evaluation value based on a form of the structure, The information processing apparatus according to claim 1 , further comprising: selecting a method for correcting the structure based on the calculated evaluation value.

7. A setting unit that sets a correction point from the same structure, The information processing apparatus according to claim 1 , wherein the calculation unit calculates the feature amount for the correction portion set by the setting unit.

8. The information processing device according to claim 1 , wherein the correction unit corrects the structure using a length, an angle, a circumference or an area as a correction standard.

9. The information processing device according to claim 1 , wherein the correction unit selects a method for correcting the structure depending on reliability of a process for acquiring the structure.

10. The information processing apparatus according to claim 1 , further comprising a display control unit that causes the medical image before the correction and the medical image after the correction to be superimposed on each other on a display unit.

11. The information processing device according to claim 6 , wherein the evaluation value is a likelihood of the structure.

12. The information processing device according to claim 11 , wherein the correction unit corrects the other correction location in accordance with the correction location that includes many pixels with a high likelihood.

13. Acquiring the same structure from a plurality of medical images taken at different times or with different devices; Calculating a feature amount relating to a morphology of the structure for each of the plurality of medical images; correcting the structure so that the difference in the feature amount becomes smaller.

14. Acquiring the same structure from a plurality of medical images taken at different times or with different devices; Calculating a feature amount relating to a morphology of the structure for each of the plurality of medical images; A process of correcting the structure so that the difference in the feature amount is reduced is performed by a computer. The program to be executed.

Images