Image processing device, image processing method, and program
The image processing device accurately calculates slippage in medical images with varying observed regions by combining and analyzing multiple images from different positions, improving diagnostic precision.
Patent Information
- Application Number
- JP2024130996
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2018-12-27
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2039-12-25
AI Technical Summary
Existing methods struggle to accurately calculate the degree of slippage in medical images with different time phases due to variations in the observed region within the imaging range.
An image processing device that acquires and combines multiple medical images from different positions, calculates phase parameters, and estimates deformation information to determine the degree of slippage accurately.
Enables precise calculation of slippage even when the observed regions differ between images, reducing the likelihood of misidentifying adhesions and enhancing diagnostic accuracy.
Smart Images

Figure 2024156932000001_ABST
Abstract
Description
[Technical field]
[0001] The disclosure of this specification relates to an image processing device, an image processing method, and a program. [Background technology]
[0002] In the medical field, doctors make diagnoses using medical images obtained by imaging the target area of a subject using various modalities. In particular, when observing areas such as the lungs and heart, where the presence or absence of disease is indicated by the movement of the organs, doctors may use multiple medical images with different time phases.
[0003] As a technique for diagnosis using multiple medical images with different time phases, a technique is known in which the chest region of the same patient is imaged at two time phases with different intake volumes and the degree of slippage is calculated from two images obtained (Patent Document 1). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2016-67832 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, in Patent Document 1, there was a problem in that it was difficult to calculate the smoothness when the area of the observed area moved and the area of the observed area included in the imaging range differed between multiple medical images with different time phases.
[0006] One of the objectives of the disclosure of this specification is to provide an image processing device that can accurately calculate the degree of smoothness even when the areas of the area to be observed included in the imaging range are different between multiple medical images with different time phases.
[0007] In addition to the above-mentioned objective, the achievement of effects derived from each configuration shown in the description of the embodiment of the invention described below, which cannot be obtained by conventional techniques, can also be considered as one of the other objectives of the disclosure of this specification. [Means for solving the problem]
[0008] The image processing device disclosed in this specification is characterized by having a video acquisition means for acquiring a first video and a second video obtained by imaging an object from different positions using an imaging device, a generation means for generating a first combined image by combining a first reference image obtained from the first video with a second reference image obtained from the second video, and for generating a second combined image by combining a first comparison image obtained from the first video with a second comparison image obtained from the second video, and a smoothness calculation means for calculating the smoothness of the object based on deformation information estimated from the first combined image and the second combined image. Effect of the Invention
[0009] According to the disclosure of this specification, it is possible to calculate the smoothness with high accuracy even when the areas of the observation target included in the imaging range are different between multiple medical images with different time phases. [Brief description of the drawings]
[0010] [Figure 1] FIG. 1 is a diagram showing an example of the device configuration of an image processing apparatus according to a first embodiment. [Diagram 2] FIG. 4 is a flowchart showing an example of an overall processing procedure in the first embodiment. [Figure 3A] FIG. 4 is a diagram showing an example of association between a first video and a second video in the first embodiment. [Figure 3B] FIG. 4 is a diagram showing an example of association between a first video and a second video in the first embodiment. [Figure 3C] FIG. 4 is a diagram showing an example of association between a first video and a second video in the first embodiment. [Figure 4]FIG. 4 is a diagram showing an example of a bonding position in the first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of reference point setting in the first embodiment. [Figure 6] FIG. 11 is a diagram showing an example of the device configuration of an image processing apparatus according to a second embodiment. [Figure 7] FIG. 11 is a flowchart showing an example of an overall processing procedure in the second embodiment. [Figure 8] FIG. 11 is a diagram showing an example of phase parameters in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] A preferred embodiment of the image processing device disclosed in this specification will be described in detail below with reference to the attached drawings. However, the components described in this embodiment are merely examples, and the technical scope of the image processing device disclosed in this specification is determined by the scope of the claims, and is not limited to the individual embodiments described below. Furthermore, the disclosure of this specification is not limited to the following embodiments, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the disclosure of this specification, and are not excluded from the scope of the disclosure of this specification. In other words, configurations that combine each of the embodiments described below and their modified examples are all included in the embodiments disclosed in this specification.
[0012] First Embodiment The image processing device according to this embodiment is a device that calculates the degree of slippage near the contour of the entire region of an organ to be observed by dividing the region of the organ that has a periodic movement and using three-dimensional moving images (first moving image and second moving image) captured multiple times. The degree of slippage is movement information that indicates the degree to which a certain region in an image slides relative to the surrounding area. In other words, it is the amount of movement relative to the surroundings of the object. For example, the lungs move so that the surface of the lungs (also called the visceral pleura) slides relative to its surroundings (also called the parietal pleura) due to respiratory movement. In this case, the image processing device according to this embodiment displays the surface position of the lungs and the degree of slippage at the surface position in correspondence with each other. In such a display, if there is adhesion between the surface of the lungs and its surroundings (also called the pleural cavity), the degree of slippage is displayed smaller at the surface position where adhesion exists than in the part where adhesion does not exist.
[0013] The image processing device of this embodiment acquires a first video sequence and a second video sequence obtained by capturing an observation target (object) from different positions using an imaging device. Specifically, the first video sequence and the second video sequence are acquired by capturing an area of the observation target divided so that at least a part of the area overlaps. Then, the first video sequence is analyzed to acquire a first phase parameter representing phase information of the movement of the organ in each time-phase image of the first video sequence (still images at each time (each time phase) constituting the first video sequence). Similarly, the second video sequence is analyzed to acquire a second phase parameter representing phase information of the movement of the organ in each time-phase image of the second video sequence (still images at each time (each time phase) constituting the second video sequence).
[0014] Next, based on the acquired first phase parameter and second phase parameter, time phase images having similar phase information of the movement of the organ between the first video and the second video are associated with each other. Then, the time phase images of the first video and the second video associated with each other as the same phase are combined to generate a combined image at the phase. By generating combined images at multiple time phases in this manner, an image including the entire region of the observation target at multiple time phases can be obtained. Then, by acquiring deformation information between multiple time phases of the combined image, it becomes possible to calculate and display the degree of smoothness near the contour of the entire region of the observation target. This allows the user to observe the combined image including the entire region of the observation target and the degree of smoothness.
[0015] The configuration and processing of this embodiment will be described below with reference to Fig. 1 to Fig. 5. In this embodiment, a three-dimensional moving image (four-dimensional CT image) of the respiratory movement of the lungs captured by an X-ray CT device will be described as an example. However, the implementation of the image processing device disclosed in this specification is not limited to this, and moving images of any part that moves spontaneously, such as the heart, may be captured. Also, moving images of any part of the subject that performs periodic movement (for example, bending and stretching) may be captured.
[0016] 1 shows the configuration of an image processing device according to this embodiment. As shown in the figure, an image processing device 10 in this embodiment is connected to a data server 11 and a display unit 12.
[0017] The data server 11 holds a first video and a second video designated by a user as targets for calculating the degree of smoothness. The first video and the second video are video (also called a three-dimensional video or a four-dimensional image) consisting of three-dimensional tomographic images of a plurality of time phases obtained by imaging different imaging ranges of the same subject in advance using the same modality. The modality for imaging the three-dimensional tomographic images may be an MRI device, an X-ray CT device, a three-dimensional ultrasonic imaging device, a photoacoustic tomography device, a PET / SPECT device, an OCT device, or the like. The first video and the second video are input to the image processing device 10 via the data acquisition unit 110.
[0018] The display unit 12 is a monitor that displays the image generated by the image processing device 10 .
[0019] The image processing device 10 is composed of the following components. The data acquisition unit 110 acquires the first and second moving images input to the image processing device 10. The extraction unit 115 extracts the region of the target organ in each time phase of the first and second moving images. The phase parameter calculation unit 120 acquires a first phase parameter representing phase information of the movement of the organ in each time phase of the first moving image and a second phase parameter representing phase information of the movement of the organ in each time phase of the second moving image. The phase correspondence information acquisition unit 130 acquires correspondence information of each time phase of the first and second moving images based on the first and second phase parameters. That is, it associates a time phase image (second time phase image) of the second moving image with each time phase image (first time phase image) of the first moving image. The connection position acquisition unit 140 acquires a connection position between the first time phase image and the second time phase image associated as being in the same phase. The combined image generating unit 150 generates a combined image by combining the first time phase image and the second time phase image based on the combining position. The deformation information estimating unit 160 acquires deformation information between the multiple time phases for the combined image generated at the multiple time phases. The smoothness calculating unit 170 calculates the smoothness near the contour in the entire region of the organ to be observed based on the deformation information. The display control unit 180 performs display control to display the first moving image, the second moving image, and the smoothness on the display unit 12.
[0020] Each of the components of the image processing device 10 functions according to a computer program. For example, the CPU uses the RAM as a work area to read and execute a computer program stored in the ROM or a storage unit, thereby realizing the function of each component. Note that some or all of the functions of the components of the image processing device 10 may be realized by using a dedicated circuit. Also, some of the functions of the components of the CPU may be realized by using a cloud computer.
[0021] For example, a computing device located at a different location from the image processing device 10 may be communicatively connected to the image processing device 10 via a network, and the image processing device 10 and the computing device may send and receive data to realize the functions of the components of the image processing device 10 or the control unit.
[0022] Next, an example of processing by the image processing device 10 in FIG. 1 will be described with reference to FIG.
[0023] FIG. 2 is a flowchart showing the overall processing procedure performed by the image processing device 10. As shown in FIG.
[0024] (S200) (Data Acquisition) In step S200, the data acquisition unit 110 acquires a first video and a second video obtained by imaging an object from different positions using an imaging device input to the image processing device 10. That is, step S200 corresponds to an example of a video acquisition process of acquiring a first video and a second video obtained by imaging an object from different positions using an imaging device. Specifically, for example, when the observation object is a lung, the first video and the second video are acquired by dividing the lung in the craniocaudal direction so that at least a part of the lung overlaps. That is, the first video is an image including the apex of the lung, and the second video is an image including the base of the lung. Note that the observation object is not limited to the lung, and may be, for example, the heart, whose movement indicates the presence or absence of a disease, as with the lung, or other organs.
[0025] Then, the acquired first and second video sequences are output to the extraction unit 115 , the phase parameter calculation unit 120 , and the connection position acquisition unit 140 .
[0026] (S205) (Extraction of organ regions) In step S205, the extraction unit 115 extracts the region of the target organ in each phase of the first and second video sequences, and outputs information on the extracted region of the target organ to the phase parameter calculation unit 120, the binding position acquisition unit 140, and the smoothness calculation unit 170.
[0027] A known image processing method can be used for the process of extracting the region of the organ from the image. Any threshold process using a pixel value threshold may be used, or known segmentation processes such as graph cut processing may be used. In addition, the region of the target organ may be manually extracted by the user using graphic drawing software (not shown), or the region of the target organ extracted using a known image processing method may be manually corrected by the user.
[0028] (S210) (Calculation of phase parameters) In step S210, the phase parameter calculation unit 120 calculates a first phase parameter representing phase information of the periodic motion of the target organ by analyzing the first video sequence. Also, the phase parameter calculation unit 120 calculates a second phase parameter representing phase information of the periodic motion of the target organ by analyzing the second video sequence. The periodic motion of the target organ is, for example, the respiratory motion of the lungs or the pulsation of the heart.
[0029] That is, the phase parameter calculation unit 120 corresponds to an example of a phase acquisition means that acquires a first phase parameter indicating phase information of the movement of the object in each phase of the first video sequence and a second phase parameter in each phase of the second video sequence indicating phase information of the movement of the object in each phase of the second video sequence. In particular, the phase parameter calculation unit 120 acquires the first phase parameter based on the state of the object captured in the first video sequence, and acquires the second phase parameter based on the state of the object captured in the second video sequence.
[0030] Then, the calculated first phase parameter and second phase parameter are output to the time phase corresponding information acquisition section 130.
[0031] In this embodiment, the phase parameter calculation unit 120 calculates a value that is correlated with the periodic motion of the target organ from each time phase image (still image of each time phase) included in the first moving image as the first phase parameter. For example, when the target organ is the lung, the volume of the lung changes in conjunction with the respiratory motion, so the volume of the lung included in each time phase image is measured using the lung region extracted in step S205, and the size of the lung volume in each time phase image is set as the first phase parameter. Alternatively, the size of the area of the lung in a predetermined slice plane may be set as the first phase parameter. Also, it is known that in CT images, the pixel value of the air region in the lung region changes according to the volume of the lung. Therefore, the first phase parameter may be calculated based on the distribution information of the pixel value in the lung region included in each time phase image. Note that the information is not limited to the information that can be obtained from the lung, and information of the surrounding parts that are linked to the respiratory motion may be used. For example, the first phase parameter may be calculated based on the movement of the body surface of the chest and abdomen or other organs. Furthermore, the first phase parameter may be calculated by integrating multiple pieces of information that are linked to the respiratory motion.
[0032] The second phase parameter is also obtained by analyzing the second video image using various methods, similar to the first phase parameter, although the first and second phase parameters do not necessarily have to be calculated using the same method.
[0033] Also, the phase parameter may be acquired from an external device (not shown). For example, when the target organ is a lung, the ventilation volume of the subject may be measured using a spirometer at the same time as the moving image is captured, and the measured value may be used as the phase parameter. Alternatively, the movement of the subject's body surface during the capturing of the moving image may be measured using a laser sensor or a pressure sensor, and the measured value may be used as the phase parameter.
[0034] (S220) (Acquisition of phase correspondence information) In step S220, the phase correspondence information acquisition unit 130 associates phase images of the first and second videos having similar phases based on the first and second phase parameters. Also, the unit acquires information used in the association as phase correspondence information. Here, the phase correspondence information is information indicating which phase of the first video and which phase of the second video have the most similar phases.
[0035] That is, the phase correspondence information acquiring section 130 corresponds to an example of a correlation means for correlating a phase image obtained from the first video sequence with a phase image obtained from the second video sequence based on the first phase parameter and the second phase parameter. In particular, the phase correspondence information acquiring section 130 correlates phase images having similar first and second phase parameters with each other.
[0036] The time phase correspondence information will be specifically described by taking as an example a case where the target organ is the lung and the first phase parameter and the second phase parameter are calculated based on the lung volume. Since the first and second moving images include different lung regions, the ventilation volumes are not necessarily similar even if the lung volumes are similar. Therefore, the parameters are normalized and calculated so that the first phase parameter value at the maximum inspiration position where the lung volume in the first moving image is maximum is 1, and the first phase parameter value at the maximum expiration position where the volume is minimum is 0. Similarly, the second phase parameter is normalized and calculated so that it is in the range from 0 to 1.
[0037] This allows a predetermined phase image having the first normalized phase parameter in the first video sequence to be associated with a predetermined phase image having the second phase parameter similar to the first normalized phase parameter in the second video sequence, and a pair of phase images of the first video sequence and the second video sequence associated in this manner is acquired as phase correspondence information.
[0038] Specifically, for example, the phase correspondence information acquisition unit 130 associates time phase images of the maximum inspiration position in the first video and the second video in which the values of the first phase parameter and the second phase parameter after normalization are 1, and acquires a pair of the time phase image of the maximum inspiration position in the first video and the time phase image of the maximum inspiration position in the second video as one piece of phase correspondence information. The phase correspondence information acquisition unit 130 also associates time phase images of the maximum expiration position in the first video and the second video in which the values of the first phase parameter and the second phase parameter after normalization are 0, and acquires a pair of the time phase image of the maximum expiration position in the first video and the time phase image of the maximum expiration position in the second video as one piece of phase correspondence information.
[0039] That is, the phase correspondence information acquiring unit 130 associates a phase image having a maximum value of the first phase parameter with a phase image having a maximum value of the second phase parameter, and associates a phase image having a minimum value of the first phase parameter with a phase image having a minimum value of the second phase parameter. Similarly, for phases other than the maximum inspiration and maximum expiration, the first phase image and the second phase image having similar phase parameters are associated with each other.
[0040] Next, a specific example of the result of matching multiple time-phase images using the normalized phase parameters is shown in Fig. 3. In Fig. 3, the horizontal axis represents time and the vertical axis represents phase parameters, with the first phase parameters as black dots and the second phase parameters as white dots, arranged in order of time phase from the left.
[0041] Here, an example is taken in which the number of phases in the first video sequence and the second video sequence is five, and the first phase parameters are {0.0, 0.3, 0.6, 0.8, 1.0} from the first phase, and the second phase parameters are {0.0, 0.5, 0.6, 0.8, 1.0}.
[0042] As shown in FIG. 3A, when the first phase parameter is used as a reference and associated with the most similar time phase of the second phase parameter, the first video sequence and the second video sequence are associated one-to-one starting from the first time phase, resulting in a total of five associated time phases.
[0043] Furthermore, when the second phase parameter is used as a reference as shown in FIG. 3B, the second phase (0.5) and the third phase (0.6) of the second phase parameter both correspond to the third phase (0.6) of the first phase parameter, and the number of corresponding phases is five.
[0044] In addition, when the phase parameters of multiple time phases on one side are all most similar to the same time phases on the other side, only one pair of the most similar phase parameters may be associated. In this case, the other time phases that are not associated may be associated with the second or subsequent time phases having similar phase parameters, or may not be associated and used in the subsequent steps.
[0045] That is, as shown in FIG. 3C, if the unmatched phases are not used in the subsequent steps, then when the second phase parameter is used as a reference, the second phase of the first phase parameter and the second phase parameter are not matched, and the number of matched phases is four.
[0046] As described above, the normalized parameters can be used to associate a plurality of time phase images obtained from the first video sequence with a plurality of time phase images obtained from the second video sequence.
[0047] The above number of phase parameters and the number of corresponding pairs are merely examples and are not limited to the above.
[0048] Also, the time phases may be associated based on the first and second phase parameters without normalization. For example, when the first and second phase parameters are calculated using approximately the same part of the subject between the first and second videos, the time phases with similar phase parameters may be associated as time phases with similar ventilation volumes without normalization. Specifically, by using a device coordinate system that can be obtained from the header information of the videos, the body surface positions of the thorax and abdomen at the same device coordinate positions on the first and second videos are obtained as the first and second phase parameters. In this case, the time phases with similar phase parameters (close body surface positions) between the first and second videos may be associated without normalizing the phase parameters.
[0049] In the following, the maximum inspiration position is referred to as the reference phase, and the maximum expiration position is referred to as the comparison phase. Also, the time-phase images of the first video and the second video associated with the maximum inspiration position, which is the reference phase, are referred to as the first reference image and the second reference image, respectively. Also, the time-phase images of the first video and the second video associated with the maximum expiration position, which is the comparison phase, are referred to as the first comparison image and the second comparison image, respectively. Also, the maximum expiration position may be the reference phase, and the maximum inspiration position may be the comparison phase, or another phase acquired from each of the first video and the second video may be assigned as the reference phase or the comparison phase.
[0050] (S230) (Calculation of binding position information) In step S230, the combination position acquisition unit 140 calculates, as combination position information, a position where the first time phase image obtained from the first video sequence and the second time phase image obtained from the second video sequence are combined in each time phase associated in step S220. That is, the combination position acquisition unit 140 corresponds to an example of a combination position information acquisition means that acquires combination position information between the first reference image obtained from the first video sequence and the second reference image obtained from the second video sequence. Then, the combination position acquisition unit 140 outputs the calculated combination position information to the combination image generation unit 150.
[0051] As the joining position, a position on the second time phase image corresponding to a predetermined position on the first time phase image is calculated. For example, when the i-th slice position of the first time phase image is P1_i, a j-th slice position P2_j of the second time phase image corresponding to the slice position P1_i of the first time phase image is searched for. Then, the slice position P2_j of the second time phase image corresponding to the slice position P1_i of the first time phase image is held as the joining position. Here, it is preferable that the slice position P1_i is selected from an overlapping region of the first and second moving images. The joining position may not be a slice position, but may be a position of a pixel on the second time phase image corresponding to each pixel on the slice position P1_i of the first time phase image.
[0052] The combination position can be calculated by searching for pixels that represent the same position between the first time phase image and the second time phase image so that they substantially coincide with each other using a known image processing method. For example, a slice position where the image similarity between the tomographic images at a predetermined slice position is high is searched for. As the image similarity, known methods such as the commonly used Sum of Squared Difference (SSD), mutual information, and cross-correlation coefficient can be used.
[0053] In addition, not only the binding position but also the reliability of the binding position can be calculated as the binding position information. The movement of the lung in the craniocaudal direction from the reference phase to the comparison phase is approximately unidirectional, and when a slice position P2_j corresponding to a slice position P1_i in each time phase is searched for, it is expected that the value of j changes monotonically in one direction or hardly changes when i is a fixed value. That is, for example, when a binding position significantly different from other time phases is shown only in a certain time phase, the calculation result of the binding position of that time phase is likely to be erroneous, and it can be determined that the reliability of the binding position is low. The reliability of the binding position can be calculated based on the value of the second derivative of the binding position calculated in the consecutive time phases, for example, a value inversely proportional to the value of the second derivative (a value proportional to the degree of reliability). That is, the binding position acquisition unit 140 corresponds to an example of a calculation means for calculating the reliability of the binding position of the time phase image obtained from the first video image and the time phase image obtained from the second video image associated by the association means.
[0054] (S240) (Generation of combined image) In step S240, the combined image generating unit 150 generates a combined image by combining the first phase image and the second phase image based on the combined positions of each phase calculated in step S230, and outputs the generated combined image to the deformation information estimating unit 160 and the display control unit 180.
[0055] The combined image generating unit 150 generates a combined image by combining the first time phase image so that a position on the first time phase image and a corresponding position on the second time phase image overlap based on the combining position calculated in step S230. A specific description will be given with reference to FIG. 4. In FIG. 4, an image 400 represents a coronal slice of the first time phase image, and a dashed line 410 represents a slice position P1_i. An image 420 represents a coronal slice of the second time phase image, and a dashed line 430 represents a slice position P2_j corresponding to the slice position P1_i calculated in step S230. An image 440 represents a combined image obtained by translating the second time phase image in the slice direction and combining the first time phase image so that the slice position P1_i of the first time phase image overlaps the slice position P2_j of the second time phase image. Region 450 indicates a region that is only imaged in the first phase image, region 460 indicates a region that is only imaged in both the first phase image and the second phase image, and region 470 indicates a region that is only imaged in the second phase image. The pixel values of the combined image in region 450 and region 470 are the pixel values of one of the first phase image and the second phase image. The pixel values of the combined image in region 460 may be either the pixel value of the first phase image or the pixel value of the second phase image, or may be the average value. In addition, a weighted average value may be used in which the weight of the pixel value of the first phase image is increased for pixels on the apical side of the lung and the weight of the pixel value of the second phase image is increased for pixels on the bottom side of the lung, depending on the position in region 460.
[0056] In this embodiment, whether or not to generate a combined image may be determined based on the reliability of the combined position calculated in step S230. When the reliability of the combined position is calculated, it may be determined whether or not it is equal to or greater than a predetermined threshold, and only when it is equal to or greater than the threshold, it may be determined that the reliability of the combined position is high and a combined image may be generated. In other words, from among the pairs of time phase images associated in step S220, a pair of time phase images whose calculated reliability of the combined position is equal to or greater than a predetermined threshold may be selected to generate a combined image. The pair of time phase images from which a combined image is to be generated may be selected by a user or automatically based on the reliability.
[0057] That is, the combined image generating section 150 corresponds to an example of a generating means that selects at least two or more pairs of time phase images having a reliability equal to or higher than a threshold value and generates a combined image.
[0058] This reduces the possibility that a combined image with misaligned combining positions will be generated. Also, by using only combined images with high reliability, the smoothness can be calculated with high accuracy in the subsequent processing.
[0059] In this embodiment, the first time phase image is translated so that the slice position P1_i of the first time phase image and the slice position P2_j of the corresponding second time phase image overlap, but the method of generating the combined image is not limited to this. For example, when the position of each pixel is calculated as the combining position information instead of the slice position, the second deformed image may be deformed so that each pixel overlaps at the combining position, and then combined. In addition, when the first time phase image is translated so that the slice position P1_i of the first time phase image and the slice position P2_j of the corresponding second time phase image overlap, the second time phase image may be deformed so that a predetermined portion such as a body surface position or a branching position of a bronchus is aligned before combining.
[0060] This makes it possible to generate combined images in multiple time phases. In other words, it is possible to obtain a time-series combined image that includes the entire region of the observation target. In the following, a combined image obtained by combining a first reference image and a second reference image, which are the reference phase, is called a reference combined image (or a first combined image). Also, a combined image obtained by combining a first comparison image and a second comparison image, which are the comparison phases, is called a comparison combined image (or a second combined image).
[0061] (S250) (Estimation of deformation information) In step S250, the deformation information estimating unit 160 aligns the combined images of the multiple time phases, estimates deformation information that represents the time-series deformation of the subject, and outputs the estimated deformation information to the smoothness calculating unit 170.
[0062] In this embodiment, the deformation information is obtained by a known image processing method. For example, the deformation information is obtained by deforming one image so that the image similarity between the images after deformation is high. As the image deformation model, a deformation model based on a radial basis function such as Thin Plate Spline (TPS) or a known deformation model such as Free Form Deformation (FFD) can be used.
[0063] Instead of image similarity, deformation information may be estimated so that anatomical feature points approximately match. For example, feature points may be set at the branches of bronchi and blood vessels in the lung region and at the ribs and sternum outside the lung region, and the amount of movement of the feature points may be estimated as deformation information.
[0064] The deformation information estimates the time series movement between the reference combined image, which is the combined image in the reference phase, and the comparison combined image, which is the combined image in the comparison phase, by performing all the positioning of the combined images between adjacent time phases. Note that, for example, if at least one of the reference combined image and the comparison combined image is a combined image in a time phase other than the reference phase and the comparison phase in step S240, the target for estimating the time series movement is not limited to the time series between the reference combined image and the comparison combined image.
[0065] As a result, even if the deformation between the reference phase and the comparison phase is large, the deformation between the combined images of adjacent phases is small, making the registration process easier. On the other hand, since the processing time required for the registration process increases in proportion to the number of phases, deformation information may be estimated using only a portion of phases thinned out at a predetermined interval. Also, the deformation between the reference combined image and the comparison combined image may be directly obtained. The deformation information is represented, for example, by a displacement vector field from the reference combined image to the comparison combined image.
[0066] In this embodiment, a combined image to be used for estimating deformation information may be selected based on the reliability of the combined position calculated in step S230. If the combined position is incorrect, there is a possibility that an anatomical inconsistency occurs near the combined position of the combined image. Therefore, when the reliability of the combined position is calculated, it is determined whether or not it is equal to or greater than a predetermined threshold value, and only when it is equal to or greater than the threshold value, the image is selected as the image to be used for estimating deformation information, thereby enabling deformation information to be estimated with high accuracy.
[0067] In this case, if at least one combined image is thinned out based on the reliability, deformation information cannot be estimated unless there are two or more combined images, so it is desirable that three or more combined images are generated in step S240.
[0068] That is, the deformation information estimation section 160 selects two or more combined images having a reliability equal to or greater than a threshold value from among the three or more combined images, and estimates deformation information from the selected combined images.
[0069] In addition, when the combined image is not necessarily thinned out, the combined image generated in step S240 may be two.
[0070] This allows deformation information to be estimated using only highly reliable combined images, making it possible to calculate the degree of smoothness with high accuracy in subsequent processing.
[0071] (S260) (Calculation of slipperiness) In step S260, the smoothness calculation unit 170 calculates the smoothness at each position near the contour of the target organ based on the deformation information estimated in step S250. That is, the smoothness calculation unit 170 corresponds to an example of a smoothness calculation means that calculates the smoothness of the target object based on the deformation information estimated from the first combined image and the second combined image. Then, the calculated smoothness is output to the display control unit 180.
[0072] In this embodiment, the smoothness is obtained by calculating the degree to which pixels near the contour of the target organ in the region of the target organ extracted in step S205 slip relative to nearby pixels outside the region of the target organ. For example, as shown in FIG. 5, two pixels, pixel 520 in the region of the lung adjacent in the normal direction to the contour 510 of the lung region in a combined image 500 (reference combined image) in the reference phase and pixel 530 outside the region, are set as reference points. Then, the norm of the difference in deformation information (vector) between the combined image (reference combined image) in the reference phase of the two set reference points and the combined image (comparison combined image) in the comparison phase can be calculated as the smoothness at the corresponding position of the target organ. That is, the smoothness corresponds to an example of the relative movement amount between a reference point set inside the lung region and a reference point set outside the lung region with the contour line as a reference. The norm of the deformation information may be the norm of the difference in vector along the surface of the lung contour. In addition, the number of reference points to be set is an example and is not limited to the above. Furthermore, one of the two corresponding reference points may be set on the contour line. Furthermore, the degree of smoothness does not necessarily have to be calculated from deformation information between the reference phase and the comparison phase, and deformation information calculated from a time-phase image of any phase between the reference phase and the comparison phase can be used. Note that the method of calculating the degree of smoothness is not limited to this, and a known image processing method can also be used.
[0073] (S270) (Indication of slipperiness) In step S270, the display control unit 180 visualizes the smoothness calculated in step S260 and displays it on the display unit 12.
[0074] As a method for visualizing the smoothness, a smoothness map can be generated and displayed on the display unit 12. The smoothness map is an image in which, for example, pixels at the contour positions of the lungs have pixel values of grayscale values or color scale values corresponding to the smoothness, and may be displayed in parallel with or superimposed on a combined image of a reference phase (reference combined image). Also, it may be displayed in parallel with or superimposed on a combined image other than the reference phase. In this case, it is desirable to perform coordinate conversion of the smoothness map so that the position is consistent with the combined image other than the reference phase. Also, it may be displayed in parallel with or superimposed on combined images of multiple time phases. Furthermore, they may be displayed as a moving image in which they are continuously switched. Furthermore, it may be displayed in parallel with or superimposed on the first moving image and the second moving image, which are input images. In this case, it is also desirable to display the smoothness map in spatial consistency with the first moving image and the second moving image. Note that the process of displaying the smoothness is not necessarily required, and the information on the smoothness calculated in step S260 may be output to the data server 11 and stored.
[0075] In this manner, the processing of the image processing device 10 is carried out.
[0076] According to the above, by generating a combined image of a plurality of time phases including the entire region of the observation target, and aligning the combined images between the time phases, it is possible to accurately calculate the degree of smoothness near the contour of the entire region of the observation target. More specifically, if two reference points set on either side of the contour are set near the boundary of the moving image, the position of the reference point after the phase change may not fall within the imaging range. For example, when imaging the lungs using a CT device, the sensor size of the CT device is about the size of the heart, so in order to image the entire region of the lungs, which has a larger volume than the heart, it is necessary to divide the image into upper and lower parts in the cranio-caudal direction and image them. Therefore, it was difficult to calculate the degree of smoothness near the boundary to be combined of the divided images.
[0077] However, according to the above, by combining multiple moving images captured by dividing the region of an organ, the position of the reference point after the phase change falls within any combined image. This makes it possible to accurately calculate the smoothness based on the deformation information even if two reference points are set near the boundary between moving images. Then, the user can observe the combined image containing the entire region of the observation target and the smoothness. Furthermore, since the above allows the smoothness of the entire region of the observation target (lungs) to be calculated with high accuracy, the possibility that the user will mistakenly recognize adhesion can be reduced.
[0078] (Variation 1-1) In this embodiment, in step S230, one slice position P2_j of the second time phase image corresponding to the slice position P1_i of the first time phase image is calculated, but it is not necessarily one position.
[0079] In the respiratory motion of the lungs, the direction of movement inside the lung region is different from that outside the lung region. Therefore, when the slice position P2_j of the second phase image corresponding to the slice position P1_i of the first phase image is searched for using image similarity, the combined position may be calculated correctly for either the inside or outside of the lung region, but may not be calculated correctly for the other. Therefore, the lung region extracted in step S205 may be used to search for combined positions inside and outside the lung region. In this case, the slice position P2_k of the second phase image having high image similarity between the slice position P1_i of the first phase image and the inside of the lung region, and the slice position P2_l having high image similarity outside the lung region are set as combined positions. In this case, in step S240, a combined image combined at each combined position can be generated. That is, two combined images combined at each combined position inside the lung region and outside the lung region are generated in each phase. Then, in step S250, deformation information of each combined image is estimated. The deformation information estimated within the lung region and outside the lung region may be integrated into one deformation information by using the lung region extracted in step S205, and integrating the deformation information estimated using the combined image of the inside of the lung region and the deformation information estimated using the combined image of the outside of the lung region. Then, in step S260, the smoothness at each position near the contour of the target organ is calculated based on the deformation information estimated within the lung region and the deformation information estimated outside the lung region. Alternatively, the smoothness at each position near the contour of the target organ is calculated based on the deformation information obtained by integrating the deformation information estimated within the lung region and the deformation information estimated outside the lung region.
[0080] This allows a combined image to be generated from time-phase images with the highest similarity between the inside and outside of the region of the organ being observed, thereby making it possible to calculate the degree of smoothness with even greater accuracy.
[0081] (Variation 1-2) In step S250 in this embodiment, when estimating deformation information between combined images of multiple time phases, deformation information may be estimated inside and outside the lung region. That is, the smoothness calculation unit 170 may calculate the smoothness of the object based on deformation information inside the object and deformation information outside the object estimated from the first combined image and the second combined image. For example, when estimating deformation information of a combined image combined at a combined position within the lung region, deformation information may be estimated by performing preprocessing such as pixel value conversion under conditions suitable for observation inside the lung region (e.g., window level (WL) is -600, window width (WW) is 1500). When estimating deformation information of a combined image combined at a combined position outside the lung region, deformation information may be estimated by performing preprocessing such as pixel value conversion under conditions suitable for observation outside the lung region (e.g., WL is 60, WW is 400).
[0082] This allows deformation information to be estimated from conditions suitable for observation inside and outside the region of the organ to be observed, respectively, thereby making it possible to calculate the degree of smoothness with even greater accuracy.
[0083] (Variation 1-3) In this embodiment, the combined image is generated in step S240, but by replacing steps S230 to S250 with the following processing, deformation information may be estimated without generating a combined image. First, the combined position acquisition unit 140 acquires relative position information of a first reference image included in the first video and a second reference image included in the second video as combined position information. Then, the deformation information estimation unit 160 estimates first deformation information between the first reference image and a first comparison image included in the first video, and estimates second deformation information between the second reference image and a second comparison image included in the second video. Furthermore, based on the combined position information, third deformation information between the first reference image and the second comparison image is estimated. Alternatively, deformation information between the first comparison image and the second reference image is estimated as the third deformation information. For example, deformation information of a part commonly captured in the first reference image and the second comparison image can be calculated by integrating the combined position information and the second deformation information. The first deformation information, the second deformation information, and the third deformation information calculated by the above process are integrated to estimate deformation information similar to the deformation information described in step S250 of the first embodiment. Specifically, for example, the deformation information of an area captured in common to the first reference image and the first comparative image can be integrated using a common area to set the deformation information as the first deformation information. In this case, the deformation information of the common area between the second reference image and the second comparative image can be integrated as the second deformation information, and the deformation information of the common area between the first reference image and the second comparative image can be integrated as the third deformation information.
[0084] This allows the smoothness to be calculated without generating a combined image, thereby reducing the amount of computer memory required to hold the combined image.
[0085] (Variation 1-4) In addition, in this embodiment, a combined image of a plurality of time phases is generated in step S240, but by replacing steps S230 to S250 with the following processing, a combined image of only one time phase may be generated and deformation information may be estimated. Relative position information of the first reference image and the second reference image is acquired as combined position information, and a reference combined image is generated by combining the first reference image and the second reference image based on the combined position information. Then, a fourth deformation information between the reference combined image and the first comparison image is estimated, and a fifth deformation information between the reference combined image and the second comparison image is estimated. The fourth deformation information and the fifth deformation information calculated by the above processing are integrated to estimate deformation information similar to the deformation information described in step S250 of the first embodiment. Specifically, the deformation information of the common area between the reference combined image and the first comparison image is set as the fourth deformation information, and the deformation information of the common area between the combined image and the second comparison image is set as the fifth deformation information, so that integration can be performed.
[0086] This allows for a reduction in the computer memory required to hold the combined image, since the combined image is generated for only one time phase.
[0087] <Second embodiment> In the first embodiment, a combined image is generated by combining time phase images associated with the same phase in the first and second video sequences. On the other hand, the image processing device according to the present embodiment generates an interpolation image between time phase images as necessary, and generates a combined image using the interpolation image.
[0088] Specifically, for example, if there is no time phase image in the second video sequence having a second phase parameter similar to the first phase parameter of the first time phase image in the first video sequence, an interpolated image having a phase corresponding to that of the first time phase image is generated from two time phase images included in the second video sequence, and then a combined image is generated from the first time phase image and the interpolated image.
[0089] This makes it possible to generate an interpolated image whose phase corresponds to that of the first time phase image from the second video sequence, and allows the user to observe a combined image obtained by combining the time phase images whose phases correspond to each other.
[0090] In the following description, it is assumed that the organ to be observed is the lung, and the first and second moving images are four-dimensional CT images, as in the first embodiment, however, the organ to be observed and the imaging device are not limited to this.
[0091] The configuration and processing of this embodiment will be described below with reference to FIGS.
[0092] 6 shows the configuration of the image processing device according to this embodiment. The functions of the time phase correspondence information acquisition unit 130 and the interpolation image generation unit 610 are explained below. The other components are the same as those in the first embodiment, so the explanation is omitted.
[0093] The time phase correspondence information acquisition section 130 acquires an interpolated time phase having a phase corresponding to a predetermined time phase image (first time phase image) of either the first or second video sequence. The interpolated time phase represents a time phase between consecutive captured time phases. The interpolated image generation section 610 generates a time phase image (interpolated image) corresponding to the interpolated time phase from the time phase image of the other time phase.
[0094] Fig. 7 shows a flowchart of the overall processing procedure performed by the image processing device 60. Steps S700 to S710 and S730 to S770 are similar to steps S200 to S210 and S230 to S270 in the first embodiment, and therefore their explanations are omitted. Only the differences from the flowchart in Fig. 2 will be explained below.
[0095] (S720) (Acquisition of time phase correspondence information) In step S720, similar to the process of step S220 in the first embodiment, the time-phase correspondence information acquisition unit 130 associates the time-phase images of the first moving image and the second moving image corresponding to the phase based on the first phase parameter and the second phase parameter. Then, information indicating the similarity of the phases is acquired as the time-phase correspondence information. However, when there is no phase of the other phase parameter that matches one phase parameter in a predetermined (targeted) time-phase image, a process of associating with the interpolation time-phase is performed.
[0096] This will be specifically described with reference to FIG. 8. FIG. 8 shows, with the horizontal axis representing time and the vertical axis representing the phase parameter, the first phase parameter as black dots and the second phase parameter as white dots, arranged in the order of time-phase from left to right. Here, when the time-phase images with the most similar phase parameters are associated with each other based on the second phase parameter, the time-phase image 800 at the 5th time-phase of the first moving image will not be associated with any time-phase image of the second moving image. At this time, in this embodiment, an interpolation time-phase to be associated with the time-phase image 800 at the 5th time-phase of the first moving image is acquired. For example, the time-phase images at the 4th time-phase of the first moving image and the second moving image are associated with each other, and the time-phase image 810 at the 6th time-phase of the first moving image and the time-phase image 815 at the 5th time-phase of the second moving image are associated with each other. From this, it can be estimated that the time-phase image 800 at the 5th time-phase of the first moving image represents the state of the subject between the time-phase image 805 at the 4th time-phase and the time-phase image 815 at the 5th time-phase of the second moving image. Therefore, the time-phase image 800 at the 5th time-phase of the first moving image is associated with the interpolation time-phase (for example, 4.5 time-phases) between the time-phase image 805 at the 4th time-phase and the time-phase image 815 at the 5th time-phase of the second moving image. The interpolation time-phase can also be calculated as follows.
[0097] For example, taking the case where the time-phase images at the a-th time-phase of the first moving image and the b-th time-phase of the second moving image are associated with each other, and further the time-phase image at the (a + c)-th time-phase (0 < c) of the first moving image and the time-phase image at the (b + d)-th time-phase (0 < d) of the second moving image are associated with each other as an example. In this case, the interpolation time-phase (b + f) (0 < f < d) of the second moving image associated with the time-phase image at the (a + e)-th time-phase (0 < e < c) of the first moving image can be obtained using Equation (1). b + f = b + d×e / c ··· (1)
[0098] Note that the method for obtaining the interpolation time phase is not limited to the above method. For example, the interpolation time phase may be obtained based on the value of the phase parameter.
[0099] For example, take the case where the value of the first phase parameter of the phase image at the (a + e)-th time phase of the first moving image is p_e, and the values of the second phase parameters of the phase images at the b-th time phase and the (b + d)-th time phase of the second moving image are p_b and p_d (p_b < p_e < p_d), respectively. In this case, the interpolation time phase b + f can be obtained using Equation (2) according to the ratio of the phase parameters. b + f = b + d×(p_e - p_b) / (p_d - p_b) ··· (2)
[0100] In this embodiment, a method for obtaining the interpolation time phase of the second moving image corresponding to a predetermined (targeted) time phase image of the first moving image has been described. However, the interpolation time phase of the first moving image corresponding to a predetermined (targeted) time phase image of the second moving image may be obtained by a similar method.
[0101] (S725) (Generation of interpolation image) In step S725, when the interpolation time phase is obtained in step S720, the interpolation image generation unit 610 generates an interpolation image at the interpolation time phase. Then, the generated interpolation image is output to the combination position acquisition unit 140 and the combination image generation unit 150 as the time phase image of the second moving image corresponding to the targeted time phase image of the first moving image.
[0102] The interpolation image is generated based on the temporal movement of the subject in the time phase images near the interpolation time phase.
[0103] For example, a case where an interpolated image of the 4.5th time phase, which is an interpolated time phase of the second video sequence corresponding to the 5th time phase image of the first video sequence, is generated is taken as an example. In this case, an interpolated image is generated based on the time series movement between the 4th and 5th time phase images, which are time phase images before and after the 4.5th time phase of the second video sequence. That is, deformation information between the 4th and 5th time phases is estimated, and an interpolated image is generated by, for example, deforming one of the images. At this time, an interpolated image that represents the state of the subject between the 4th and 5th time phase images is generated by interpolating the deformation information. Specifically, when deforming the 5th time phase image, the interpolated image is generated by deforming the 5th time phase image using deformation information obtained by multiplying the amount of deformation from the 5th time phase image to the 4th time phase image by the interpolation rate obtained from the interpolated time phase. In the above case, an interpolation rate of, for example, 5 (time phase) - 4.5 (time phase) = 0.5 is used. This makes it possible to generate an interpolated image that represents the state of the subject between the fourth and fifth phases.
[0104] Note that the time phases used to generate an interpolated image do not have to be images of adjacent time phases before and after the interpolated time phase. For example, an interpolated image can be generated by estimating deformation information between a time phase b before the interpolated time phase b+f and a time phase b+d after the interpolated time phase b+f, and deforming the time phase image of the time phase b+d using deformation information obtained by multiplying the deformation amount by the value of (df) / d. Also, a value (interpolation rate) by which the deformation information is multiplied can be calculated based on phase parameters in the two time phases for which the deformation information is estimated.
[0105] The two phases used when estimating deformation information do not have to be phases before and after the interpolated phase. For example, a case will be taken as an example where an interpolated image of the first video sequence having a similar phase to the seventh phase image 825 of the second video sequence shown in Fig. 8 is generated. In this case, deformation information between the sixth phase image 810 and the seventh phase image 820 of the first video sequence may be estimated, and the interpolated image may be generated by extrapolating the deformation information.
[0106] 8, the value of the second phase parameter of the seventh phase image 825 of the second video sequence is a value between the first phase parameters of the second phase image and the third phase image of the first video sequence. Therefore, it is possible to estimate that the phase is between these two phases. Therefore, it is possible to generate an interpolated image of an interpolated phase of the first video sequence corresponding to the seventh phase image 825 of the second video sequence by estimating deformation information of the phase images between the second and third phases of the first video sequence and deforming one of the phase images.
[0107] In this embodiment, the deformation information can be estimated using a method similar to that used in step S250 in the first embodiment.
[0108] In this manner, the processing of the image processing device 60 is carried out.
[0109] This allows for the generation of a combined image by combining time-phase images with similar ventilation volumes in the lungs to be observed. That is, even if the pace of respiratory movement during the capture of the first and second moving images is different, time-phase images with any ventilation volume can be combined. Then, the user can easily observe the movement of the entire lung region by observing the combined image in which time-phase images with similar ventilation volumes are combined in chronological order. In addition, since the smoothness can be calculated based on the combined image with the phases more appropriately associated, the smoothness can be calculated with higher accuracy.
[0110] <Other embodiments> Furthermore, the technology disclosed in this specification can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium), etc. Specifically, the technology may be applied to a system composed of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus composed of a single device.
[0111] Needless to say, the object of the technology disclosed in this specification can be achieved by the following: That is, a recording medium (or storage medium) on which is recorded a program code (computer program) of software that realizes the functions of the above-mentioned embodiments is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the technology disclosed in this specification.
[0112] Furthermore, the disclosure of this specification is not limited to the above-mentioned embodiment, and various modifications (including organic combinations of the respective examples) are possible based on the spirit of the disclosure of this specification, and are not excluded from the scope of the disclosure of this specification. In other words, configurations that combine the above-mentioned respective examples and their modifications are all included in the embodiments disclosed in this specification.
[0113] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.
[0114] This application claims priority based on Japanese Patent Application No. 2018-245138, filed on December 27, 2018, the entire contents of which are incorporated herein by reference.
Claims
1. a moving image acquisition means for acquiring a first moving image and a second moving image obtained by capturing images of an object from different positions using an imaging device; a generating means for generating a first combined image by combining a first reference image obtained from the first video sequence with a second reference image obtained from the second video sequence, and for generating a second combined image by combining a first comparison image obtained from the first video sequence with a second comparison image obtained from the second video sequence; a smoothness calculation means for calculating a smoothness of the object based on deformation information estimated from the first combined image and the second combined image; 13. An image processing device comprising:
2. 2. The image processing device according to claim 1, wherein the degree of smoothness is a relative amount of movement calculated based on the positions in the first combined image and the positions in the second combined image of two reference points respectively set inside and outside the object on either side of the contour line of the object.
3. The object is an organ that performs periodic motion, a phase acquiring means for acquiring a first phase parameter indicating phase information of a motion of the object in each time phase of the first video sequence, and a second phase parameter indicating phase information of a motion of the object in each time phase of the second video sequence, at each time phase; a matching means for matching a plurality of time phase images including the first reference image and the first comparison image obtained from the first video sequence with a plurality of time phase images including the second reference image and the second comparison image obtained from the second video sequence, based on the first phase parameter and the second phase parameter; 3. The image processing apparatus according to claim 1, wherein the first and second reference images, and the first and second comparison images are images associated by the association means.
4. The image processing device according to claim 3, characterized in that the phase acquisition means acquires the first phase parameter based on the state of the object captured in the first moving image, and acquires the second phase parameter based on the state of the object captured in the second moving image.
5. 5. The image processing apparatus according to claim 3, wherein the association means associates time phase images in which the first phase parameter and the second phase parameter are similar to each other.
6. 6. The image processing device according to claim 3, wherein the matching means matches time phase images inside the object in which the first phase parameter and the second phase parameter are similar, and matches time phase images outside the object in which the first phase parameter and the second phase parameter are similar.
7. 7. The image processing device according to claim 3, further comprising a calculation means for calculating a reliability of a joining position of a time phase image obtained from the first video sequence and a time phase image obtained from the second video sequence that have been associated by the association means.
8. 8. The image processing apparatus according to claim 7, wherein the generating means selects at least two or more sets of time phase images having the reliability equal to or greater than a threshold value and generates a combined image.
9. the generating means generates at least three or more combined images from a time phase image obtained from the first video sequence and a time phase image obtained from the second video sequence; The image processing device described in claim 7 or 8, characterized in that the smoothness calculation means selects two or more combined images from the three or more combined images whose reliability is equal to or greater than a threshold value, and calculates the smoothness of the object based on deformation information estimated from the selected combined images.
10. 10. The image processing device according to claim 1, wherein the generating means generates at least one of the first reference image and the first comparison image from at least one time phase image obtained from the first moving image.
11. 11. The image processing device according to claim 1, wherein the generating means generates at least one of the second reference image and the second comparison image from at least one time phase image obtained from the second moving image.
12. the generating means further generates a third combined image by combining the first reference image and a third reference image obtained from the second video sequence; The image processing device according to any one of claims 1 to 11, characterized in that the smoothness calculation means calculates the smoothness of the object based on first deformation information estimated from the first combined image and the second combined image and second deformation information estimated from the first combined image and the third combined image.
13. The image processing device according to any one of claims 1 to 12, characterized in that the smoothness calculation means calculates the smoothness of the object based on deformation information inside the object and deformation information outside the object estimated from the first combined image and the second combined image.
14. a moving image acquisition means for acquiring a first moving image and a second moving image obtained by capturing images of an object from different positions using an imaging device; a combination position information acquisition means for acquiring combination position information between a first reference image obtained from the first video sequence and a second reference image obtained from the second video sequence; The binding position information; first deformation information estimated from a first comparison image obtained from the first reference image and the first video sequence; second deformation information estimated from a second comparison image obtained from the second reference image and the second video sequence; and a smoothness calculation means for calculating a smoothness of the object based on third deformation information estimated from at least one of a set of the first reference image and the second comparison image or a set of the first comparison image and the second reference image; 13. An image processing device comprising:
15. a moving image acquisition means for acquiring a first moving image and a second moving image obtained by capturing images of an object from different positions using an imaging device; a generating means for generating a combined image by combining a first reference image obtained from the first video sequence and a second reference image obtained from the second video sequence; a smoothness calculation means for calculating a smoothness of the object based on first deformation information estimated from a first comparison image obtained from the combined image and the first video sequence, and second deformation information estimated from the second comparison image obtained from the combined image and the second video sequence; 13. An image processing device comprising:
16. 16. The image processing apparatus according to claim 1, wherein the imaging device is an X-ray CT device, and the first moving image and the second moving image are four-dimensional CT images.
17. 17. The image processing device according to claim 1, wherein the object is a lung, and the first moving image and the second moving image are moving images obtained by dividing the lung in the cranial-caudal direction and imaging the divided lung.
18. 18. The image processing device according to claim 17, wherein the matching means matches a time phase image at a maximum inspiration position obtained from the first video with a time phase image at the maximum inspiration position obtained from the second video, and matches a time phase image at a maximum expiration position obtained from the first video with a time phase image at the maximum expiration position obtained from the second video.
19. a moving image acquiring step of acquiring a first moving image and a second moving image obtained by capturing images of an object from different positions using an imaging device; a generating step of generating a first combined image by combining a first reference image obtained from the first video sequence and a second reference image obtained from the second video sequence, and generating a second combined image by combining a first comparison image obtained from the first video sequence and a second comparison image obtained from the second video sequence; a smoothness calculation step of calculating a smoothness of the object based on deformation information estimated from the first combined image and the second combined image; 13. An image processing method comprising:
20. 19. A program for causing a computer to execute each of the means of the image processing apparatus according to claim 1.
Citation Information
Patent Citations
Radiation image processor, radiographic image processing method, and program
JP2011255055A
Medical image processor, and medical image processing method
JP2016067832A
Breast image display system and image processing device
JP2017018317A
Radiographic system
JP2017225724A
