Image processing device, image processing method, and program

Abstract

To calculate a slipping degree with a high degree of precision even in the case where a region of a body part as an observation object included in an imaging range is different among a plurality of medical images of different time phases.SOLUTION: An image processing device includes: moving image acquisition means for acquiring a first moving image and a second moving image obtained by imaging an object from mutually different positions by an imaging device; generation means for generating a first combined image by combining a first reference image obtained from the first moving image and a second reference image obtained from the second moving image, and generating a second combined image by combining a first comparison image obtained from the first moving image and a second comparison image obtained from the second moving image; and slipping degree calculation means for calculating a slipping degree of the object on the basis of deformation information estimated from the first combined image and the second combined image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The disclosure of this specification relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] In the medical field, a doctor diagnoses using medical images obtained by imaging a site to be observed of a subject with various modalities. In particular, when observing a site where the presence or absence of a disease appears in the movement of an organ, such as the lungs or the heart, a plurality of medical images with different time phases may be used.

[0003] As a technique related to diagnosis performed using a plurality of medical images with different time phases, a technique for calculating a slipperiness from two images obtained by imaging the chest region of the same patient at two time phases with different inhalation amounts is known (Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, there is a problem that it is difficult to calculate the slipperiness when the region of the site to be observed included in the imaging range is different between a plurality of medical images with different time phases due to the movement of the site to be observed.

[0006] One object of the disclosure of this specification is to provide an image processing apparatus capable of accurately calculating the slipperiness even when the regions of the site to be observed included in the imaging range are different between a plurality of medical images with different time phases.

[0007] Furthermore, not limited to the aforementioned objectives, the effects and benefits derived from each configuration shown in the embodiments for carrying out the invention described later, which cannot be obtained by conventional art, can also be considered as another objective of the disclosure in this specification. [Means for solving the problem]

[0008] The image processing apparatus disclosed herein is characterized by comprising: a motion image acquisition means for acquiring a first motion image and a second motion image obtained by imaging an object from different positions using an imaging device; a generation means for generating a first combined image by combining a first reference image obtained from the first motion image and a second reference image obtained from the second motion image, and generating a second combined image by combining a first comparison image obtained from the first motion image and a second comparison image obtained from the second motion image; and a slip degree calculation means for calculating the slip degree of the object based on deformation information estimated from the first combined image and the second combined image. [Effects of the Invention]

[0009] According to the disclosures herein, it is possible to accurately calculate the slip degree even when the area of ​​the subject being observed included in the imaging range differs among multiple medical images of different time phases. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing an example of the equipment configuration of the image processing device according to the first embodiment. [Figure 2] A flowchart showing an example of the overall processing procedure in the first embodiment. [Figure 3A] A diagram showing an example of the correspondence between the first video and the second video in the first embodiment. [Figure 3B] A diagram showing an example of the correspondence between the first video and the second video in the first embodiment. [Figure 3C] A diagram showing an example of the correspondence between the first video and the second video in the first embodiment. [Figure 4]A diagram showing an example of the bonding position in the first embodiment. [Figure 5] A diagram showing an example of setting a reference point in the first embodiment. [Figure 6] A diagram showing an example of the equipment configuration of the image processing device according to the second embodiment. [Figure 7] A flowchart showing an example of the overall processing procedure in the second embodiment. [Figure 8] A figure showing an example of phase parameters in the second embodiment. [Modes for carrying out the invention]

[0011] Preferred embodiments of the image processing apparatus disclosed herein will be described in detail below with reference to the accompanying drawings. However, the components described in these embodiments are merely illustrative, and the technical scope of the image processing apparatus disclosed herein is determined by the claims and is not limited by the individual embodiments described below. Furthermore, the disclosure herein is not limited to the embodiments described below, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the disclosure herein, and these are not excluded from the scope of the disclosure herein. That is, configurations combining each of the embodiments and their modified examples described later are all included in the embodiments disclosed herein.

[0012] <First Embodiment> The image processing device according to this embodiment calculates the degree of slippage near the contour of the entire region of an organ that is to be observed, using three-dimensional moving images (first moving image and second moving image) taken multiple times by dividing the region of the organ that is to be observed and observing it. The degree of slippage is movement information that represents how much a certain region in an image slips relative to the surrounding area. In other words, it is the amount of movement relative to the surrounding area of ​​the object. For example, the lungs move in a way that the surface of the lung (also called the visceral pleura) slides relative to its surroundings (also called the parietal pleura) due to respiratory movement. In this case, the image processing device according to this embodiment displays the surface position of the lung and the degree of slippage at that surface position in correspondence. In such a display, if there is adhesion between the surface of the lung and its surroundings (also called the pleural cavity), the degree of slippage at the surface position where adhesion exists will be displayed as smaller than in the part where there is no adhesion.

[0013] The image processing device of this embodiment acquires a first video image and a second video image by an imaging device, capturing an object (subject) from different positions. Specifically, the first video image and the second video image are acquired by dividing the region of the object so that at least a portion of the region of the object overlaps. Then, by analyzing the first video image, a first phase parameter is obtained that represents the phase information of the organ's movement in each time-phase image of the first video image (still images at each time point (each time phase) that constitutes the first video image). Similarly, by analyzing the second video image, a second phase parameter is obtained that represents the phase information of the organ's movement in each time-phase image of the second video image (still images at each time point (each time phase) that constitutes the second video image).

[0014] Next, based on the acquired first phase parameter and second phase parameter, time-phase images with similar phase information of the movement of the organ between the first moving image and the second moving image are associated with each other. Then, by combining the time-phase images of the first moving image and the second moving image associated as the same phase, a combined image in that phase is generated. By generating combined images in a plurality of time phases in this way, an image including the entire region of the observation target in a plurality of time phases can be obtained. And by obtaining deformation information between a plurality of time phases of the combined image, it becomes possible to calculate and display the degree of slippage near the contour of the entire region of the observation target. As a result, the user can observe the combined image including the entire region of the observation target and the degree of slippage.

[0015] Hereinafter, the configuration and processing of the present embodiment will be described with reference to FIGS. 1 to 5. In the present embodiment, a three-dimensional moving image (4D CT image) obtained by imaging the respiratory movement of the lungs with an X-ray CT apparatus will be described as an example. However, the implementation of the image processing apparatus disclosed in this specification is not limited to this, and a moving image obtained by imaging any part that makes spontaneous movements such as the heart may also be used. Further, a moving image obtained by imaging any part of the subject that has performed a periodic movement (for example, flexion and extension movement) may also be used.

[0016] FIG. 1 shows the configuration of the image processing apparatus according to the present embodiment. As shown in the figure, the image processing apparatus 10 in the present embodiment is connected to a data server 11 and a display unit 12.

[0017] The data server 11 holds a first moving image and a second moving image specified by the user as targets for calculating the degree of slippage. The first moving image and the second moving image are moving images (also called three-dimensional moving images or four-dimensional images) composed of three-dimensional tomographic images of a plurality of time phases obtained by previously imaging different imaging ranges of the same subject with the same modality. The modality for imaging the three-dimensional tomographic image may be an MRI apparatus, an X-ray CT apparatus, a three-dimensional ultrasonic imaging apparatus, a photoacoustic tomography apparatus, a PET / SPECT, an OCT apparatus, or the like. The first moving image and the second moving image are input to the image processing apparatus 10 via the data acquisition unit 110.

[0018] The display unit 12 is a monitor that displays the image generated by the image processing apparatus 10.

[0019] The image processing apparatus 10 is composed of the following components. The data acquisition unit 110 acquires the first moving image and the second moving image input to the image processing apparatus 10. The extraction unit 115 extracts the region of the target organ in each time phase of the first moving image and the second moving image. The phase parameter calculation unit 120 acquires a first phase parameter representing the phase information of the movement of the organ in each time phase of the first moving image and a second phase parameter representing the phase information of the movement of the organ in each time phase of the second moving image. The time-phase correspondence information acquisition unit 130 acquires the correspondence information of each time phase of the first moving image and the second moving image based on the first phase parameter and the second phase parameter. That is, for each time-phase image (first time-phase image) of the first moving image, the time-phase image (second time-phase image) of the second moving image is associated. The combination position acquisition unit 140 acquires the combination position between the first time-phase image and the second time-phase image associated as the same phase. The combined image generation unit 150 generates a combined image obtained by combining the first time-phase image and the second time-phase image based on the combination position. The deformation information estimation unit 160 acquires the deformation information between a plurality of time phases for the combined images generated in a plurality of time phases. The slipperiness calculation unit 170 calculates the slipperiness near the contour in the entire region of the organ to be observed based on the deformation information. The display control unit 180 performs display control to display the first moving image, the second moving image, and the slipperiness on the display unit 12.

[0020] Each component of the above image processing apparatus 10 functions according to a computer program. For example, the CPU reads and executes a computer program stored in the ROM or the storage unit with the RAM as a work area, thereby realizing the functions of each component. Note that some or all of the functions of the components of the image processing apparatus 10 may be realized by using dedicated circuits. Also, some of the functions of the components of the CPU may be realized by using a cloud computer.

[0021] For example, a computing device located in a different location from the image processing device 10 may be connected to the image processing device 10 via a network in a communicative manner, and the functions of the components of the image processing device 10 or the control unit may be realized by the image processing device 10 and the computing device sending and receiving data.

[0022] Next, an example of the processing performed by the image processing device 10 in Figure 1 will be explained using Figure 2.

[0023] Figure 2 shows a flowchart of the overall processing procedure performed by the image processing device 10.

[0024] (S200) (Data acquisition) In step S200, the data acquisition unit 110 acquires a first video image and a second video image obtained by imaging the object from different positions using an imaging device input to the image processing device 10. That is, step S200 corresponds to an example of a video image acquisition process in which a first video image and a second video image are obtained by imaging the object from different positions using an imaging device. Specifically, for example, if the object to be observed is the lung, a first video image and a second video image are obtained by dividing the lung in the cranial direction so that at least a part of the lung overlaps. In other words, the first video image includes the apex of the lung, and the second video image includes the base of the lung. Note that the object to be observed is not limited to the lung; for example, it may be the heart, where the presence or absence of disease is reflected in the movement of the organ, similar to the lung, or it may be any other organ.

[0025] The acquired first and second video images are then output to the extraction unit 115, the phase parameter calculation unit 120, and the combination position acquisition unit 140.

[0026] (S205) (Extraction of organ regions) In step S205, the extraction unit 115 extracts the region of the target organ at each time phase of the first and second moving images. The extracted information of the target organ region is then output to the phase parameter calculation unit 120, the joint position acquisition unit 140, and the slip degree calculation unit 170.

[0027] The process of extracting organ regions from images can utilize known image processing techniques. This may involve arbitrary thresholding using pixel value thresholds, or known segmentation techniques such as graph cut processing. Alternatively, the user may manually extract the target organ region using drawing software (not shown), or the user may manually modify the target organ region extracted using known image processing techniques.

[0028] (S210) (Calculation of phase parameters) In step S210, the phase parameter calculation unit 120 calculates a first phase parameter representing the phase information of the periodic motion of the target organ by analyzing the first video image. It also calculates a second phase parameter representing the phase information of the periodic motion of the target organ by analyzing the second video image. The periodic motion of the target organ is, for example, the respiratory motion of the lungs or the beating of the heart.

[0029] In other words, the phase parameter calculation unit 120 corresponds to an example of a phase acquisition means that acquires a first phase parameter indicating the phase information of the object's movement at each time phase of the first video, and a second phase parameter for each time phase of the second video indicating the phase information of the object's movement at each time phase of the second video. Specifically, the phase parameter calculation unit 120 acquires the first phase parameter based on the state of the object captured in the first video, and acquires the second phase parameter based on the state of the object captured in the second video.

[0030] The first and second phase parameters calculated are then output to the time-phase correspondence information acquisition unit 130.

[0031] In this embodiment, the phase parameter calculation unit 120 calculates a value that correlates with the periodic motion of the target organ from each time-phase image (still image of each time phase) included in the first moving image as the first phase parameter. For example, if the target organ is the lung, the volume of the lung changes in conjunction with respiratory movement, so the lung region extracted in step S205 is used to measure the volume of the lung included in each time-phase image, and the size of the lung volume in each time-phase image is used as the first phase parameter. Alternatively, the size of the area of ​​the lung within a predetermined slice plane may be used as the first phase parameter. Furthermore, in CT images, it is known that the pixel value of the air region within the lung region changes according to the volume of the lung. Therefore, the first phase parameter may be calculated based on the distribution information of pixel values ​​within the lung region included in each time-phase image. Note that information from surrounding areas that are linked to respiratory movement may be used, not just information obtainable from the lung. For example, the first phase parameter may be calculated based on the body surface of the chest and abdomen or the movement of other organs. Furthermore, the first phase parameter may be calculated by integrating multiple pieces of information linked to respiratory movement.

[0032] The second phase parameter is obtained by analyzing the second video image using various methods, similar to the first phase parameter. However, the first and second phase parameters do not necessarily have to be calculated using the same method.

[0033] Furthermore, the system may be configured to acquire phase parameters from an external device (not shown). For example, if the target organ is the lungs, the subject's ventilation volume may be measured using a spirometer simultaneously with the acquisition of a moving image, and the measured value may be used as the phase parameter. Alternatively, the movement of the subject's body surface during the acquisition of a moving image may be measured using a laser sensor or pressure sensor, and the measured value may be used as the phase parameter.

[0034] (S220) (Acquisition of temporal phase correspondence information) In step S220, the time phase correspondence information acquisition unit 130 associates the time phase images of the first and second video images that have similar phases, based on the first and second phase parameters. It also acquires the information used in the association process as time phase correspondence information. Here, time phase correspondence information is information that indicates which time phase of the first video image and which time phase of the second video image have the most similar phases.

[0035] In other words, the time-phase correspondence information acquisition unit 130 corresponds to an example of a correspondence means that associates a time-phase image obtained from a first video image with a time-phase image obtained from a second video image based on a first phase parameter and a second phase parameter. Specifically, the time-phase correspondence information acquisition unit 130 associates time-phase images whose first phase parameter and second phase parameter are similar.

[0036] Regarding the correspondence information of phases, we will specifically explain using the example of the lung as the target organ, where the first and second phase parameters are calculated based on the lung volume. Since the lung regions included in the first and second video images are different, even if the lung volumes are similar, the tidal volumes are not necessarily similar. Therefore, for example, the parameters are normalized and calculated so that the value of the first phase parameter at the maximum inspiratory position where the lung volume is maximum in the first video image is 1, and the value of the first phase parameter at the maximum expiratory position where the volume is minimum is 0. Similarly, the second phase parameter is also normalized and calculated so that it is in the range of 0 to 1.

[0037] This allows a predetermined time-phase image having a first phase parameter after normalization in the first video to be associated with a predetermined time-phase image having a second phase parameter similar to the first phase parameter after normalization in the second video. The pair of time-phase images of the first and second videos thus associated is then acquired as time-phase correspondence information.

[0038] Specifically, for example, the time-phase correspondence information acquisition unit 130 associates the time-phase images of the maximum inspiratory position in the first and second video footage where the normalized values ​​of the first and second phase parameters are 1, and acquires the pair of the time-phase images of the maximum inspiratory position in the first video footage and the second video footage as one of the time-phase correspondence information. Furthermore, the time-phase correspondence information acquisition unit 130 associates the time-phase images of the maximum expiratory position in the first and second video footage where the normalized values ​​of the first and second phase parameters are 0, and acquires the pair of the time-phase images of the maximum expiratory position in the first video footage and the second video footage as one of the time-phase correspondence information.

[0039] In other words, the time-phase correspondence information acquisition unit 130 associates the time-phase image with the maximum value of the first phase parameter with the time-phase image with the maximum value of the second phase parameter, and associates the time-phase image with the minimum value of the first phase parameter with the time-phase image with the minimum value of the second phase parameter. Similarly, for phases other than the maximum inspiratory position and the maximum expiratory position, the unit also associates the first time-phase image with the second time-phase image with similar phase parameters.

[0040] Next, Figure 3 shows an example of the results of specifically associating multiple time-phase images using the normalized phase parameters. In Figure 3, with the horizontal axis representing time and the vertical axis representing the phase parameter, the first phase parameter is represented by black dots and the second phase parameter by white dots, arranged from left to right in time-phase order.

[0041] Here, we take the example where the first and second video sequences have 5 time phases, and the phase parameters of the first sequence are {0.0, 0.3, 0.6, 0.8, 1.0} in order from the first time phase, and the phase parameters of the second sequence are {0.0, 0.5, 0.6, 0.8, 1.0}.

[0042] As shown in Figure 3A, when the first phase parameter is used as a reference and the second phase parameter is associated with the most similar time phase, the first and second video images are associated one-to-one starting from the first time phase, resulting in a total of five time phases being associated.

[0043] Furthermore, as shown in Figure 3B, when the second phase parameter is used as the reference, the second phase (0.5) and third phase (0.6) of the second phase parameter both correspond to the third phase (0.6) of the first phase parameter, resulting in a total of five corresponding phases.

[0044] Furthermore, if the phase parameters of multiple time phases on one side are all most similar to those of the same time phase on the other side, only one pair with the most similar phase parameters may be associated. In this case, the other time phases that were not associated may be associated with the second or subsequent time phases with similar phase parameters, or they may not be associated at all and may not be used in subsequent steps.

[0045] In other words, as shown in Figure 3C, if the time phases that were not matched are not used in subsequent steps, then, using the second phase parameter as the reference, the second time phase of the first phase parameter and the second phase parameter will not be matched, and the number of matched time phases will be four.

[0046] As described above, the normalized parameters can be used to associate multiple time-phase images obtained from the first video with multiple time-phase images obtained from the second video.

[0047] The number of phase parameters and corresponding pairs mentioned above are examples only and are not limited to those listed above.

[0048] Alternatively, the time phases may be associated based on the first and second phase parameters without normalization. For example, if the first and second phase parameters are calculated using approximately the same part of the subject between the first and second video images, time phases with similar phase parameters can be associated with time phases with similar ventilation without normalization. Specifically, the device coordinate system obtained from the video image header information is used to obtain the body surface positions of the chest and abdomen at the same device coordinate system position on the first and second video images as the first and second phase parameters. In this case, time phases with similar phase parameters (close body surface positions) between the first and second video images may be associated without normalizing the phase parameters.

[0049] In the following, the maximum inspiratory position will be referred to as the reference phase, and the maximum expiratory position as the comparison phase. The phase images of the first and second video footage, which are associated with the maximum inspiratory position (the reference phase), will be referred to as the first reference image and the second reference image, respectively. Similarly, the phase images of the first and second video footage, which are associated with the maximum expiratory position (the comparison phase), will be referred to as the first comparison image and the second comparison image, respectively. It should be noted that the maximum expiratory position may be used as the reference phase and the maximum inspiratory position as the comparison phase, or other phases obtained from the first and second video footage may be assigned as the reference phase or the comparison phase.

[0050] (S230) (Calculation of joint position information) In step S230, the joining position acquisition unit 140 calculates the joining position information for each time phase associated in step S220, where the first time phase image obtained from the first video image and the second time phase image obtained from the second video image are joined. In other words, the joining position acquisition unit 140 corresponds to an example of a joining position information acquisition means that acquires joining position information between the first reference image obtained from the first video image and the second reference image obtained from the second video image. The calculated joining position information is then output to the joining image generation unit 150.

[0051] The joining position is calculated by determining a predetermined position on the first time-phase image and the corresponding position on the second time-phase image. For example, if the i-th slice position of the first time-phase image is P1_i, the j-th slice position P2_j of the second time-phase image that corresponds to the slice position P1_i of the first time-phase image is searched for. Then, the slice position P2_j of the second time-phase image that corresponds to the slice position P1_i of the first time-phase image is stored as the joining position. Here, it is desirable to select the slice position P1_i from the overlapping region of the first and second video images. The joining position may not be a slice position, but rather the position of each pixel on the second time-phase image that corresponds to each pixel on the slice position P1_i of the first time-phase image.

[0052] The join position can be calculated by searching for pixels that represent the same position between the first and second time-phase images using known image processing techniques. For example, a search is performed for a slice position that increases the image similarity between tomographic images at a given slice position. Commonly used methods such as Sum of Squared Difference (SSD), cross-information, and cross-correlation coefficient can be used to measure image similarity.

[0053] Furthermore, not only the binding position but also the reliability of the binding position can be calculated as binding position information. The movement of the lung in the head-to-tail direction from the reference phase to the comparison phase is approximately unidirectional, and when searching for the slice position P2_j corresponding to the slice position P1_i in each time phase, it is expected that the value of j will change monotonically in one direction or hardly change at all when i is a fixed value. That is, for example, if one time phase shows a binding position that is significantly different from other time phases, the calculation result of the binding position for that phase is likely to be incorrect, and it can be judged that the reliability of the binding position is low. The reliability of the binding position can be calculated based on the value of the second derivative of the binding position calculated in consecutive time phases, for example, by using a value inversely proportional to the value of the second derivative (a value proportional to the level of reliability). That is, the binding position acquisition unit 140 corresponds to an example of a calculation means for calculating the reliability of the binding position of the time phase image obtained from the first video image and the time phase image obtained from the second video image, which are associated by the matching means.

[0054] (S240) (Generation of combined images) In step S240, the combined image generation unit 150 generates a combined image by combining the first time-phase image and the second time-phase image based on the combination positions of each time phase calculated in step S230. The generated combined image is then output to the deformation information estimation unit 160 and the display control unit 180.

[0055] The combined image generation unit 150 generates a combined image by combining the first temporal image and the corresponding second temporal image based on the combination position calculated in step S230, so that the positions on the first temporal image and the corresponding positions on the second temporal image overlap. This will be explained in detail using Figure 4. In Figure 4, image 400 represents the coronal section of the first temporal image, and the dashed line 410 represents the slice position P1_i. Image 420 represents the coronal section of the second temporal image, and the dashed line 430 represents the slice position P2_j corresponding to the slice position P1_i calculated in step S230. Image 440 represents the combined image obtained by translating the second temporal image in the slicing direction and combining it so that the slice position P1_i of the first temporal image and the slice position P2_j of the second temporal image overlap. Region 450 represents a region captured only in the first time-phase image, region 460 represents a region captured in both the first and second time-phase images, and region 470 represents a region captured only in the second time-phase image. The pixel values ​​of the combined images in regions 450 and 470 are the pixel values ​​of either the first or second time-phase image. The pixel values ​​of the combined images in region 460 may be the pixel values ​​of either the first or second time-phase image, or they may be the average value. Furthermore, depending on the location within region 460, a weighted average may be used in which the weight of the pixel values ​​of the first time-phase image is increased for pixels on the apical side of the lung, and the weight of the pixel values ​​of the second time-phase image is increased for pixels on the base side of the lung.

[0056] In this embodiment, the decision of whether or not to generate a combined image may be made based on the confidence level of the join position calculated in step S230. If the confidence level of the join position is calculated, it may be determined whether or not it is above a predetermined threshold, and only if it is above the threshold, it may be determined that the confidence level of the join position is high and a combined image may be generated. In other words, in step S220, a set of time-phase images whose calculated confidence level of the join position is above a predetermined threshold may be selected from the associated set of time-phase images and a combined image may be generated. The selection of the set of time-phase images from which to generate a combined image may be made by the user, or it may be automatically selected based on the confidence level.

[0057] In other words, the combined image generation unit 150 corresponds to an example of a generation means that selects at least two sets of temporal images with a confidence level equal to or greater than a threshold and generates a combined image.

[0058] This reduces the possibility of generating merged images with misaligned joint positions. Furthermore, by using only highly reliable merged images, the slip rate can be calculated with high accuracy in subsequent processing.

[0059] In this embodiment, a combined image was generated by translating the slice position P1_i of the first temporal image and the corresponding slice position P2_j of the second temporal image so that they overlap. However, the method for generating the combined image is not limited to this. For example, if the position of each pixel is calculated as the joining position information instead of the slice position, the second deformed image may be deformed so that each pixel overlaps at the joining position before joining. Also, when generating a combined image by translating the slice position P1_i of the first temporal image and the corresponding slice position P2_j of the second temporal image so that they overlap, the second temporal image may be deformed so that predetermined parts such as body surface positions or bronchial branching positions align before joining.

[0060] This allows for the generation of combined images at multiple time phases. In other words, it is possible to obtain a time-series combined image that includes the entire region of the object of observation. In the following, the combined image obtained by combining the first reference image (which represents the reference phase) and the second reference image will be called the reference combined image (or the first combined image). Similarly, the combined image obtained by combining the first comparison image (which represents the comparison phase) and the second comparison image will be called the comparison combined image (or the second combined image).

[0061] (S250) (Estimation of deformation information) In step S250, the deformation information estimation unit 160 aligns the combined images of multiple time phases and estimates deformation information that represents the time-series deformation of the subject. The estimated deformation information is then output to the slip degree calculation unit 170.

[0062] In this embodiment, deformation information is obtained by known image processing techniques. For example, it is obtained by deforming one of the images so that the image similarity between the deformed images is high. Known deformation models such as Thin Plate Spline (TPS) or Free Form Deformation (FFD) can be used for the image deformation model.

[0063] Instead of relying on image similarity, deformation information may be estimated so that anatomical feature points closely match. For example, feature points may be set for the branching of bronchi and blood vessels within the lung region and for ribs and the sternum outside the lung region, and the amount of movement of these feature points may be estimated as deformation information.

[0064] The deformation information estimates the time-series movement between the reference combined image (a combined image in the reference phase) and the comparison combined image (a combined image in the comparison phase) by performing alignment of the combined images between adjacent time phases. For example, if in step S240 at least one of the reference combined image and the comparison combined image is a combined image in a time phase different from the reference and comparison phases, the time-series movement to be estimated is not limited to the time series between the reference combined image and the comparison combined image.

[0065] This makes alignment easier because even if there is a large deformation between the reference phase and the comparison phase, the deformation becomes smaller between the combined images of adjacent time phases. On the other hand, since the processing time required for alignment increases in proportion to the number of time phases, deformation information may be estimated using only a portion of the time phases that have been thinned out at predetermined intervals. Alternatively, the deformation between the reference combined image and the comparison combined image may be determined directly. The deformation information can be represented, for example, by a displacement vector field from the reference combined image to the comparison combined image.

[0066] Furthermore, in this embodiment, the combined image to be used for estimating deformation information may be selected based on the reliability of the join position calculated in step S230. If the join position is incorrect, anatomical inconsistencies may occur near the join position of the combined image. Therefore, when calculating the reliability of the join position, it is possible to accurately estimate deformation information by determining whether or not it is above a predetermined threshold and selecting only the image to be used for estimating deformation information if it is above the threshold.

[0067] In this case, if at least one combined image is downsampled based on its confidence level, deformation information cannot be estimated unless there are two or more combined images. Therefore, it is desirable that three or more combined images are generated in step S240.

[0068] In other words, the deformation information estimation unit 160 selects two or more combined images from three or more combined images whose confidence level is above a threshold, and estimates deformation information from the selected combined images.

[0069] If the combined images are not necessarily thinned out, two combined images may be generated in step S240.

[0070] This allows deformation information to be estimated using only highly reliable combined images, enabling accurate calculation of the slip degree in subsequent processing.

[0071] (S260) (Calculation of slippage) In step S260, the slip degree calculation unit 170 calculates the slip degree at each position near the contour of the target organ based on the deformation information estimated in step S250. In other words, the slip degree calculation unit 170 is an example of a slip degree calculation means that calculates the slip degree of an object based on deformation information estimated from the first combined image and the second combined image. The calculated slip degree is then output to the display control unit 180.

[0072] In this embodiment, the slip degree is determined by calculating how much a pixel near the contour of the target organ region extracted in step S205 slips relative to a neighboring pixel outside the target organ region. For example, as shown in Figure 5, two pixels, pixel 520 inside the lung region and pixel 530 outside the lung region, are set as reference points, adjacent in the normal direction to the contour line 510 of the lung region in the reference phase combined image 500 (reference combined image). Then, the norm of the difference in deformation information (vector) between the combined image in the reference phase (reference combined image) and the combined image in the comparison phase (comparison combined image) of the two set reference points can be calculated as the slip degree at that position of the target organ. In other words, the slip degree corresponds to an example of the relative amount of movement between a reference point set inside the lung region and a reference point set outside the lung region with respect to the contour line. The norm of the deformation information may also be the norm of the difference in vectors along the surface of the lung contour. Furthermore, the number of reference points to be set is an example and is not limited to the above. In addition, one of the two corresponding reference points may be set on the contour line. Furthermore, the slip degree does not necessarily have to be determined from deformation information between the reference phase and the comparison phase; deformation information calculated from the temporal image of any phase in between can be used. Note that the method for calculating the slip degree is not limited to this, and known image processing techniques may also be used.

[0073] (S270) (Slip degree indication) In step S270, the display control unit 180 visualizes the slip rate calculated in step S260 and displays it on the display unit 12.

[0074] As a method for visualizing the degree of slip, a slip degree map can be generated and displayed on the display unit 12. The slip degree map is an image in which, for example, pixels at the contour positions of the lungs have grayscale or colorscale values ​​corresponding to the degree of slip as pixel values. It may be displayed in parallel with or superimposed on the reference phase combined image (reference combined image). It may also be displayed in parallel with or superimposed on combined images other than the reference phase. In this case, it is desirable to transform the coordinates of the slip degree map so that its position is consistent with the combined images other than the reference phase. It may also be displayed in parallel with or superimposed on combined images of multiple time phases. Furthermore, they may be displayed as a moving image by continuously switching between them. Furthermore, it may be displayed in parallel with or superimposed on the input images, the first moving image and the second moving image. In this case as well, it is desirable to display the slip degree map spatially consistent with the first moving image and the second moving image. Note that the process of displaying the degree of slip is not necessarily required, and the slip degree information calculated in step S260 may be output to the data server 11 and saved.

[0075] The image processing device 10 then performs the processing described above.

[0076] According to the above, by generating a combined image of multiple time phases that include the entire region of the object to be observed, and aligning the combined image between time phases, the slip degree near the contour of the entire region of the object to be observed can be calculated with high accuracy. More specifically, if two reference points set on either side of the contour line are set near the boundary of the moving image, the position of the reference points after the phase change may not fall within the imaging range. For example, when imaging the lungs with a CT scanner, the sensor size of the CT scanner is about the size of the heart, so in order to image the entire lung region, which has a larger volume than the heart, it is necessary to divide the image vertically in the cranial direction and image it vertically. Therefore, it was difficult to calculate the slip degree near the boundary to be combined from the divided images.

[0077] However, as described above, by combining multiple moving images acquired by dividing the organ region, the position of the reference point after phase change will be contained within any combined image. This allows for accurate calculation of the slip degree based on deformation information, even when the two reference points are set near the boundary of the moving images. The user can then observe the combined image containing the entire region of the object of observation and the slip degree. Furthermore, because the slip degree of the entire region of the object of observation (lung) can be calculated accurately, the possibility of the user misidentifying adhesions can be reduced.

[0078] (Extreme Variation 1-1) In this embodiment, in step S230, the slice position P2_j of a second time-phase image corresponding to the slice position P1_i of the first time-phase image was calculated, but it does not necessarily have to be a single position.

[0079] In lung respiratory movements, the direction of movement differs between the lung region and the region outside the lung region. Therefore, when using image similarity to search for the slice position P1_i of the first time-phase image and the corresponding slice position P2_j of the second time-phase image, the join position may be calculated correctly for one of the lung regions (inside or outside the lung region) but not for the other. For this reason, the lung region extracted in step S205 may be used to search for the join positions inside and outside the lung region, respectively. In this case, the slice position P2_k of the second time-phase image with high image similarity between the slice position P1_i of the first time-phase image and the lung region, and the slice position P2_l with high image similarity outside the lung region are used as the join positions. In this case, in step S240, a combined image can be generated by joining at each join position. That is, two combined images are generated for each time phase, joined at the respective join positions inside and outside the lung region. Then, in step S250, the deformation information of each combined image is estimated. The deformation information estimated within and outside the lung region may be integrated into a single deformation information by using the lung region extracted in step S205, combining the deformation information estimated using the combined image of the lung region and the deformation information estimated using the combined image of the lung region and the lung region. Then, in step S260, the degree of slip at each position near the contour of the target organ is calculated based on the deformation information estimated within the lung region and the deformation information estimated outside the lung region. Alternatively, the degree of slip at each position near the contour of the target organ is calculated based on the deformation information obtained by integrating the deformation information estimated within the lung region and the deformation information estimated outside the lung region.

[0080] This allows for the generation of a combined image from the most similar temporal images within and outside the region of the organ being observed, enabling more accurate calculation of the slip degree.

[0081] (Variations 1-2) In step S250 of this embodiment, when estimating deformation information between multiple time-phase combined images, deformation information may be estimated both within and outside the lung region. That is, the slip degree calculation unit 170 may calculate the slip degree of the object based on the deformation information inside the object and the deformation information outside the object estimated from the first combined image and the second combined image. For example, when estimating deformation information of a combined image joined at a joining position within the lung region, preprocessing such as pixel value conversion may be applied to conditions suitable for observation within the lung region (for example, window level (WL) to -600, window width (WW) to 1500) to estimate the deformation information. When estimating deformation information of a combined image joined at a joining position outside the lung region, preprocessing such as pixel value conversion may be applied to conditions suitable for observation outside the lung region (for example, WL to 60, WW to 400) to estimate the deformation information.

[0082] This allows for the estimation of deformation information from conditions suitable for observation both inside and outside the region of the organ being observed, enabling the calculation of slippage with greater accuracy.

[0083] (Variations 1-3) In this embodiment, a combined image is generated in step S240, but deformation information may be estimated without generating a combined image by replacing steps S230 to S250 with the following process. First, the combined position acquisition unit 140 acquires the relative position information of the first reference image included in the first video and the second reference image included in the second video as combined position information. Then, the deformation information estimation unit 160 estimates the first deformation information between the first reference image and the first comparison image included in the first video, and estimates the second deformation information between the second reference image and the second comparison image included in the second video. Furthermore, based on the combined position information, it estimates the third deformation information between the first reference image and the second comparison image. Alternatively, it estimates the deformation information between the first comparison image and the second reference image as the third deformation information. For example, deformation information of a part that is captured in common in the first reference image and the second comparison image can be calculated by integrating the combined position information and the second deformation information. By integrating the first deformation information, second deformation information, and third deformation information calculated through the above process, deformation information similar to the deformation information described in step S250 of the first embodiment is estimated. Specifically, for example, the deformation information of an area captured in common with the first reference image and the first comparison image can be integrated using a common area. In this case, the deformation information of a common area of ​​the second reference image and the second comparison image can be integrated as the second deformation information, and the deformation information of a common area of ​​the first reference image and the second comparison image can be integrated as the third deformation information.

[0084] This allows for the calculation of slip depth without generating a combined image, thereby reducing the computer memory required to store the combined image.

[0085] (Variations 1-4) Furthermore, although multiple time-phase combined images were generated in step S240 in this embodiment, the deformation information may be estimated by generating a combined image of only one time phase by replacing steps S230 to S250 with the following process. The relative position information of the first reference image and the second reference image is obtained as combined position information, and a reference combined image is generated by combining the first reference image and the second reference image based on the combined position information. Then, the fourth deformation information between the reference combined image and the first comparison image is estimated, and the fifth deformation information between the reference combined image and the second comparison image is estimated. By integrating the fourth deformation information and the fifth deformation information calculated by the above process, deformation information similar to the deformation information described in step S250 of the first embodiment is estimated. Specifically, the deformation information of the common area between the reference combined image and the first comparison image can be used as the fourth deformation information, and the deformation information of the common area between the combined image and the second comparison image can be used as the fifth deformation information for integration.

[0086] This reduces the amount of computer memory required to store the combined image, as the generated combined image is only one time phase.

[0087] <Second Embodiment> In the first embodiment, a combined image was generated by combining temporal images that were associated with the same phase in the first and second moving images. On the other hand, the image processing device according to this embodiment generates interpolated images between temporal images as needed and generates a combined image using the interpolated images.

[0088] Specifically, for example, if, in the second video, there is no time-phase image with a second phase parameter similar to the first phase parameter of the first time-phase image of the first video, an interpolated image corresponding to the phase of the first time-phase image is generated from the two time-phase images contained in the second video. Then, a combined image is generated from the first time-phase image and the interpolated image.

[0089] This allows for the generation of an interpolated image with a corresponding phase from a second video image, and the user can observe a combined image formed by combining the phase-corresponding phase images.

[0090] In the following explanation, as in the first embodiment, the organ to be observed is the lung, and the first and second moving images are assumed to be 4D CT images. However, the organ to be observed and the imaging device are not limited to these.

[0091] The configuration and processing of this embodiment will be described below with reference to Figures 6 to 8.

[0092] Figure 6 shows the configuration of the image processing apparatus according to this embodiment. The functions of the time-phase corresponding information acquisition unit 130 and the interpolated image generation unit 610 will be described below. The other components are the same in function as those of the first embodiment, so their description will be omitted.

[0093] The time phase correspondence information acquisition unit 130 acquires the interpolated time phase of the other time phase that corresponds in phase to a predetermined time phase image (first time phase image) of either the first or second moving image. The interpolated time phase represents the time phase between consecutive time phases captured. The interpolated image generation unit 610 generates a time phase image (interpolated image) corresponding to the interpolated time phase from time phase images of other time phases.

[0094] Figure 7 shows a flowchart of the overall processing procedure performed by the image processing device 60. Steps S700 to S710 and S730 to S770 perform the same processing as steps S200 to S210 and S230 to S270 in the first embodiment, so their explanation is omitted. Below, only the differences from the flowchart in Figure 2 will be explained.

[0095] (S720) (Acquisition of temporal phase correspondence information) In step S720, the time-phase correspondence information acquisition unit 130 associates the time-phase images of the first moving image and the second moving image corresponding to the phase based on the first phase parameter and the second phase parameter, in the same manner as the process of step S220 in the first embodiment. Then, information indicating the similarity of the phases is acquired as the time-phase correspondence information. However, in a predetermined (targeted) time-phase image, if there is no time-phase of the other phase parameter that matches one phase parameter, a process of associating with an interpolated time-phase is performed.

[0096] This will be specifically described using FIG. 8. FIG. 8 shows, with the horizontal axis representing time and the vertical axis representing the phase parameter, the first phase parameter as black dots and the second phase parameter as white dots, arranged in the order of time-phase from left to right. Here, when associating the time-phase images with the most similar phase parameters based on the second phase parameter, the time-phase image 800 at the 5th time-phase of the first moving image will not be associated with any time-phase image of the second moving image. At this time, in this embodiment, an interpolated time-phase to be associated with the time-phase image 800 at the 5th time-phase of the first moving image is acquired. For example, the time-phase images at the 4th time-phase of the first moving image and the second moving image are associated with each other, and the time-phase image 810 at the 6th time-phase of the first moving image and the time-phase image 815 at the 5th time-phase of the second moving image are associated with each other. From this, it can be estimated that the time-phase image 800 at the 5th time-phase of the first moving image represents the state of the subject between the time-phase image 805 at the 4th time-phase and the time-phase image 815 at the 5th time-phase of the second moving image. Therefore, the time-phase image 800 at the 5th time-phase of the first moving image is associated with the interpolated time-phase (for example, 4.5 time-phase) between the time-phase image 805 at the 4th time-phase and the time-phase image 815 at the 5th time-phase of the second moving image. The interpolated time-phase can also be calculated as follows.

[0097] For example, taking the case where the time-phase images at the a-th time-phase of the first moving image and the b-th time-phase of the second moving image are associated with each other, and further the time-phase image at the a + c (0 < c) -th time-phase of the first moving image and the time-phase image at the b + d (0 < d) -th time-phase of the second moving image are associated with each other as an example. In this case, the interpolated time-phase b + f (0 < f < d) of the second moving image associated with the time-phase image at the a + e -th time-phase (0 < e < c) of the first moving image can be obtained using Equation (1). b + f = b + d×e / c ··· (1)

[0098] Note that the method for obtaining the interpolation time phase is not limited to the above method. For example, the interpolation time phase may be obtained based on the value of the phase parameter.

[0099] For example, take the case where the value of the first phase parameter of the phase image at the a + e time phase of the first moving image is p_e, and the values of the second phase parameters of the phase images at the b time phase and the b + d time phase of the second moving image are p_b and p_d (p_b < p_e < p_d), respectively. In this case, the interpolation time phase b + f can be obtained using Equation (2) according to the ratio of the phase parameters. b + f = b + d×(p_e - p_b) / (p_d - p_b) ··· (2)

[0100] In this embodiment, a method for obtaining the interpolation time phase of the second moving image corresponding to a predetermined (targeted) time phase image of the first moving image has been described. However, the interpolation time phase of the first moving image corresponding to a predetermined (targeted) time phase image of the second moving image may be obtained by the same method.

[0101] (S725) (Generation of interpolation image) In step S725, when the interpolation time phase is obtained in step S720, the interpolation image generation unit 610 generates an interpolation image at the interpolation time phase. Then, the generated interpolation image is output to the combination position acquisition unit 140 and the combined image generation unit 150 as the time phase image of the second moving image corresponding to the targeted time phase image of the first moving image.

[0102] The interpolation image is generated based on the time-series movement of the subject in the time phase images near the interpolation time phase.

[0103] For example, consider the case of generating an interpolated image of the 4.5th time phase, which is the interpolated time phase of the second video that corresponds to the 5th time phase image of the first video. In this case, the interpolated image is generated based on the time-series movement between the 4th and 5th time phase images, which are the time phase images before and after the 4.5th time phase of the second video. That is, deformation information between the 4th and 5th time phases is estimated, and the interpolated image is generated by, for example, deforming one of the images. At this time, by interpolating the deformation information, an interpolated image representing the state of the subject between the 4th and 5th time phase images is generated. Specifically, when deforming the 5th time phase image, the interpolated image is generated by deforming the 5th time phase image using deformation information obtained by multiplying the amount of deformation from the 5th time phase image to the 4th time phase image by the interpolation rate obtained from the interpolated time phase. In the above case, for example, the interpolation rate used is 5 (time phase) - 4.5 (time phase) = 0.5. This allows for the generation of an interpolated image representing the subject's state between the fourth and fifth time phases.

[0104] Furthermore, the time phases used to generate the interpolated image do not necessarily have to be adjacent time phases before and after the interpolated time phase. For example, the interpolated image can be generated by estimating the deformation information between time phase b before the interpolated time phase b+f and time phase b+d after it, and then using the deformation information obtained by multiplying the deformation amount by the value of (df) / d to deform the time phase image of time phase b+d. Alternatively, the value to be multiplied by the deformation information (interpolation rate) can be determined based on the phase parameters of the two time phases for which the deformation information is estimated.

[0105] Note that the two time phases used when estimating deformation information do not necessarily have to be the time phases before and after the interpolated time phase. For example, consider the case where an interpolated image of the first video is generated that is similar in phase to the time phase image 825 of the 7th time phase of the second video shown in Figure 8. In this case, deformation information between the time phase image 810 of the 6th time phase and the time phase image 820 of the 7th time phase of the first video may be estimated, and the interpolated image may be generated by extrapolating that deformation information.

[0106] Furthermore, in Figure 8, the value of the second phase parameter of the 7th time phase image 825 of the second video is the value between the first phase parameter of the 2nd time phase image and the 3rd time phase image of the first video. Therefore, it can be estimated that the phase is between these two time phases. Accordingly, by estimating the deformation information of the time phase images between the 2nd and 3rd time phases of the first video and deforming either of the phase images, an interpolated image of the interpolated time phase of the first video corresponding to the 7th time phase image 825 of the second video may be generated.

[0107] Furthermore, the method for estimating deformation information in this embodiment can be the same as the method used in step S250 of the first embodiment.

[0108] The image processing device 60 then performs the processing described above.

[0109] This allows for the generation of a composite image by combining time-phase images of similar ventilation volumes in the lung being observed. In other words, even if the respiratory movement paces during the acquisition of the first and second video images are different, time-phase images of any ventilation volume can be combined. The user can then easily observe the movement of the entire lung region by observing the composite image, which combines time-phase images of similar ventilation volumes, in chronological order. Furthermore, since the slip degree can be calculated based on a composite image with more appropriately matched phases, a more accurate slip degree can be calculated.

[0110] <Other Embodiments> Furthermore, the technologies disclosed herein can take the form of systems, apparatus, methods, programs, or recording media (storage media), for example. Specifically, they may be applied to a system consisting of multiple devices (e.g., a host computer, interface devices, imaging devices, web applications, etc.) or to an apparatus consisting of a single device.

[0111] Furthermore, it goes without saying that the objective of the technology disclosed herein is achieved as follows: a recording medium (or storage medium) containing program code (computer program) of software that realizes the functions of the embodiments described above is supplied to a system or device. Such storage medium is, of course, a computer-readable storage medium. The computer (or CPU or MPU) of the system or device then reads and executes the program code stored on the recording medium. In this case, the program code read from the recording medium itself realizes the functions of the embodiments described above, and the recording medium containing that program code constitutes the technology disclosed herein.

[0112] Furthermore, the disclosure herein is not limited to the embodiments described above. Various modifications (including organic combinations of each embodiment) are possible in accordance with the spirit of the disclosure herein, and these are not excluded from the scope of the disclosure herein. That is, configurations combining each of the above-described embodiments and their modified forms are all included in the embodiments disclosed herein.

[0113] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are attached to make the scope of the invention public.

[0114] This application claims priority based on Japanese Patent Application No. 2018-245138, filed on December 27, 2018, and all of its contents are incorporated herein by reference.

Claims

1. An imaging device captures an object from different positions, and a motion image acquisition means acquires a first motion image and a second motion image, each containing different parts of the object. A generation means for obtaining a time-series combined image including the entire region of the object, by combining the first reference image and the second reference image based on the overlapping region of the object contained in the first reference image obtained from the first moving image corresponding to the reference phase and the second reference image obtained from the second moving image corresponding to the reference phase, and by combining the first comparison image and the second comparison image based on the overlapping region of the object contained in the first comparison image obtained from the first moving image corresponding to the comparison phase and the second comparison image obtained from the second moving image corresponding to the comparison phase, thereby obtaining a time-series combined image including the entire region of the object. It has, The aforementioned object is the lung or heart, The image processing apparatus is characterized in that the first and second moving images are moving images obtained by dividing the object in the head-to-tail direction and imaging it.

2. The system further includes a means for obtaining a joint position, which calculates the joint position based on the overlapping region of an object containing a first part and an object containing a second part different from the first part. The image processing apparatus according to claim 1, characterized in that the generation means generates the first combined image and the second combined image based on the combined position.

3. The aforementioned object is an organ that performs periodic movements, A phase acquisition means for acquiring a first phase parameter indicating the phase information of the object's movement in each time phase of the first video, and a second phase parameter indicating the phase information of the object's movement in each time phase of the second video, The system further includes a matching means for associating a plurality of time-phase images, including a first reference image and a first comparison image obtained from a first moving image, with a plurality of time-phase images, including a second reference image and a second comparison image obtained from a second moving image, based on the first phase parameter and the second phase parameter. The image processing apparatus according to claim 1 or 2, characterized in that the first reference image and the second reference image and the first comparison image and the second comparison image are images associated by the association means.

4. The image processing apparatus according to claim 3, characterized in that the phase acquisition means acquires a first phase parameter based on the appearance of the object captured in the first moving image, and acquires a second phase parameter based on the appearance of the object captured in the second moving image.

5. The image processing apparatus according to claim 3 or 4, characterized in that the matching means matches temporal images in which the first phase parameter and the second phase parameter are similar.

6. The image processing apparatus according to any one of claims 3 to 5, characterized in that the matching means matches temporal images within the object in which the first phase parameter and the second phase parameter are similar, and matches temporal images outside the object in which the first phase parameter and the second phase parameter are similar.

7. The image processing apparatus according to any one of claims 3 to 6, further comprising a calculation means for calculating the reliability of the concatenation position of a time-phase image obtained from a first moving image associated with the aforementioned correspondence means and a time-phase image obtained from a second moving image.

8. The image processing apparatus according to claim 7, characterized in that the generation means selects at least two or more sets of temporal images whose reliability is equal to or greater than a threshold and generates a combined image.

9. The image processing apparatus according to any one of claims 1 to 8, characterized in that the generation means generates an interpolated image corresponding to at least one of the first reference image and the first comparison image from at least one temporal image obtained from the first moving image.

10. The image processing apparatus according to claim 9, characterized in that the generation means generates the interpolated image when there is no temporal image corresponding to a second phase parameter indicating phase information of a second motion image corresponding to a first phase parameter indicating phase information of a first motion image.

11. The image processing apparatus according to claim 9 or 10, characterized in that the generation means generates the interpolated image by deforming the first time-phase image based on deformation information estimated from the first time-phase image and the second time-phase image contained in the first moving image.

12. The image processing apparatus according to any one of claims 1 to 8, characterized in that the generation means generates an interpolated image corresponding to at least one of the second reference image and the second comparison image from at least one temporal image obtained from the second moving image.

13. The image processing apparatus according to claim 12, characterized in that the generation means generates the interpolated image when there is no temporal image corresponding to a first phase parameter indicating the phase information of the first motion image corresponding to a second phase parameter indicating the phase information of the second motion image.

14. The image processing apparatus according to claim 12 or 13, characterized in that the generation means generates the interpolated image by deforming the first time-phase image based on deformation information estimated from the first time-phase image and the second time-phase image contained in the second moving image.

15. The image processing apparatus according to claim 14, characterized in that the generation means obtains an interpolation rate for generating the interpolated image based on the time phase of the first time phase image and the time phase of the second time phase image.

16. The image processing apparatus according to claim 15, characterized in that it generates the interpolated image using the interpolation rate and the deformation information.

17. The image processing apparatus according to any one of claims 1 to 16, further comprising a display control unit that displays at least one of the first combined image and the second combined image on a display unit.

18. The imaging apparatus is an X-ray CT apparatus, and the first moving image and the second moving image are four-dimensional CT images, as described in any one of claims 1 to 17.

19. The image processing apparatus according to any one of claims 3 to 7, characterized in that the corresponding means corresponds the time-phase image at the maximum inspiratory position obtained from the first moving image with the time-phase image at the maximum inspiratory position obtained from the second moving image, and corresponds the time-phase image at the maximum expiratory position obtained from the first moving image with the time-phase image at the maximum expiratory position obtained from the second moving image.

20. A video acquisition step involves acquiring a first video image and a second video image, each containing different parts of the object, obtained by imaging the object from different positions using an imaging device. A generation step to obtain a time-series combined image including the entire region of the object, by combining the first reference image and the second reference image based on the overlapping region of the object contained in the first reference image obtained from the first video image corresponding to the reference phase and the second reference image obtained from the second video image corresponding to the reference phase, and by combining the first comparison image and the second comparison image based on the overlapping region of the object contained in the first comparison image obtained from the first video image corresponding to the comparison phase and the second comparison image obtained from the second video image corresponding to the comparison phase, thereby obtaining a time-series combined image including the entire region of the object. It has, The aforementioned object is the lung or heart, The image processing method is characterized in that the first and second moving images are moving images obtained by dividing the object in the head-to-tail direction and imaging it.

21. A program characterized by causing a computer to execute each of the means of the image processing apparatus described in any one of claims 1 to 19.

Images