IMAGE PROCESSING APPARATUS, IMAGE DIAGNOSIS SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM

Abstract

This allows for determining an appropriate size of the comparison area. [Solution] The apparatus includes an image acquisition means for acquiring a first image and a second image, which are two three-dimensional images of a subject captured at different times; a pixel size acquisition means for acquiring the larger pixel size of the pixel sizes in a predetermined axis direction between the two three-dimensional images as a first pixel size; a smoothing means for smoothing the three-dimensional image having a second pixel size that is smaller than the first pixel size based on the first pixel size to generate a smoothed image; and a determination means for determining the size of the predetermined axis direction of a comparison region for comparing density values ​​between the two three-dimensional images based only on the first pixel size of the first and second pixel sizes.

Description

[Technical Field]

[0001] The disclosure of this specification relates to an image processing device, an image diagnostic system, an image processing method, and a program. [Background technology]

[0002] In the medical field, image subtraction techniques are known that align two images taken at different times and display a difference image that visualizes the differences between the images to assist in comparing the images. However, there are problems with noise appearing in the difference image due to alignment errors and differences in density values ​​of the same area between the images. Non-Patent Document 1, "Voxel Matching," proposes a technique to solve this problem. It calculates the differences between a pixel of interest in a first image and a corresponding pixel in a second image, as well as its neighboring pixels, and uses the minimum of these differences as the density value of the difference image. This technique selects a pixel closest in value to the pixel of interest from among the neighboring pixels and uses the difference value between the pixel of interest and that pixel, thereby reducing noise in the difference image. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Yoshinori Itai, Hyoungseop Kim, Seiji Ishikawa, Shigehiko Katsuragawa, Kunio Doi, “Development of a voxel-matching technique for substantial reduction of subtraction artifacts in temporal subtraction images obtained from thoracic MDCT.” Journal of digital imaging, vol. 23, No. 1, pp. 31-38, 2010. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Non-Patent Document 1, the user arbitrarily determines the range of the comparison region in which the density value of a pixel of interest in a first image is compared with the density values ​​of the corresponding pixel and its neighboring pixels in a second image in the voxel matching method. As a result, there are cases where noise cannot be sufficiently reduced, or conversely, necessary signals are deleted. In other words, there is a problem in that the size of the comparison region cannot be appropriately determined.

[0005] In addition to the above-mentioned problem, the effects derived from the various configurations shown in the description of the embodiments of the invention described below, which cannot be obtained by conventional technologies, can also be positioned as another problem of the present disclosure. [Means for solving the problem]

[0006] The image processing device disclosed in this specification is an image processing device that performs image processing to reduce noise based on a difference in pixel size contained in a difference image generated based on two images captured with different pixel sizes of a common subject, and includes an image acquisition means that acquires a first image captured at a predetermined time and a second image captured at a time different from the predetermined time, a pixel size acquisition means that acquires pixel sizes in a predetermined axis direction in the first image and the second image, respectively, and a pixel size acquisition means that acquires one of the first image and the second image, which has a smaller pixel size, based on the pixel size of the other of the first image and the second image. Using the kernel size a determining means for determining the size of a comparison region in the predetermined axis direction based on the pixel size of the other image, for a comparison region including a plurality of positions on the smoothed image whose density values ​​are compared with a position of interest on the other image; a position obtaining means for obtaining the position of interest on the other image and a corresponding position on the smoothed image corresponding to the position of interest; and a setting means for setting the position of the comparison region based on the corresponding position. a difference acquisition means for acquiring a difference value between a density value at the target position and an interpolated value obtained by interpolating density values ​​at a plurality of positions on the smoothed image by at least one of nearest neighbor interpolation, linear interpolation, and cubic interpolation; and a difference image generation means for generating the difference image based on the difference value. It has. The image processing device disclosed in this specification is an image processing device that performs image processing to reduce noise due to a discretization position shift between two images captured with different pixel sizes of a common subject, an image acquisition means for acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition means for acquiring pixel sizes in a predetermined axis direction in the first image and the second image, respectively; a smoothing means for smoothing one of the first image and the second image, which has a smaller pixel size, using a kernel size based on the pixel size of the other of the first image and the second image to generate a smoothed image; a determining means for determining a size of a comparison region in the predetermined axis direction based on a pixel size of the other image, the size of the comparison region including a plurality of positions on the smoothed image whose density values ​​are compared with the position of interest on the other image; a position acquisition means for acquiring the position of interest on the other image and a corresponding position on the smoothed image that corresponds to the position of interest; a difference obtaining means for obtaining a difference between the density value of the target position and an interpolated value obtained by interpolating density values ​​around at least a plurality of positions on the smoothed image; a setting means for setting the position of the comparison area based on the corresponding position; It has. The image processing method disclosed in this specification is an image processing method for performing image processing to reduce noise due to the difference in pixel size contained in a difference image generated based on two images captured with different pixel sizes of a common subject. method the method includes an image acquisition step of acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition step of acquiring pixel sizes in a predetermined axis direction in the first image and the second image; and a pixel size acquisition step of acquiring one of the first image and the second image having a smaller pixel size based on the pixel size of the other of the first image and the second image. Using the kernel size a determining step of determining a size of a comparison region in the predetermined axis direction based on the pixel size of the other image, for a comparison region including a plurality of positions on the smoothed image whose density values ​​are compared with positions of interest on the other image; a position obtaining step of obtaining the positions of interest on the other image and corresponding positions on the smoothed image corresponding to the positions of interest; and a setting step of setting the position of the comparison region based on the corresponding positions. a difference obtaining step of obtaining a difference value between the density value of the target position and an interpolated value obtained by interpolating density values ​​of a plurality of positions on the smoothed image by at least one of nearest neighbor interpolation, linear interpolation, and cubic interpolation; and a difference image generating step of generating the difference image based on the difference value. It has. The image processing method disclosed in this specification also includes: 1. An image processing method for performing image processing to reduce noise due to a discretization position shift between two images captured with different pixel sizes of a common subject, comprising: an image acquisition step of acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition step of acquiring pixel sizes in a predetermined axis direction in the first image and the second image, respectively; a smoothing step of smoothing one of the first image and the second image, which has a smaller pixel size, based on the pixel size of the other of the first image and the second image to generate a smoothed image; a determining step of determining a size of a comparison region in the predetermined axis direction based on a pixel size of the other image, the size of the comparison region including a plurality of positions on the smoothed image whose density values ​​are compared with the position of interest on the other image; a position acquiring step of acquiring the position of interest on the other image and a corresponding position on the smoothed image that corresponds to the position of interest; a setting step of setting the position of the comparison region based on the corresponding position; a difference obtaining step of obtaining a difference value between the density value of the target position and an interpolated value obtained by interpolating density values ​​of a plurality of positions on the smoothed image by at least one of nearest neighbor interpolation, linear interpolation, and cubic interpolation; It has. [Effects of the Invention]

[0007] According to the disclosure of the present specification, it is possible to determine an appropriate size of the comparison region. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the device configuration of an image processing apparatus according to a first embodiment. [Figure 2] FIG. 3 is a flowchart showing an example of an overall processing procedure in the first embodiment. [Figure 3] 1A, 1B, and 1C are diagrams showing examples of discretization position deviations in the first embodiment; [Figure 4] FIG. 4 is a diagram for explaining an example of a method for setting a comparison region in the first embodiment. [Figure 5] FIG. 10 is a flowchart showing an example of an overall processing procedure in the second embodiment. [Figure 6] FIG. 11 is a diagram showing an example of the device configuration of an image processing apparatus according to a third embodiment. [Figure 7] FIG. 11 is a flowchart showing an example of an overall processing procedure in the third embodiment. [Figure 8] FIG. 13 is a flowchart showing an example of an overall processing procedure in the fourth embodiment. [Figure 9] FIG. 13 is a diagram showing an example of the device configuration of an image processing apparatus according to a fifth embodiment. [Figure 10] FIG. 13 is a flowchart showing an example of an overall processing procedure in the fifth embodiment. [Figure 11] FIG. 13 is a diagram showing an example of the device configuration of an image processing apparatus according to a sixth embodiment. [Figure 12] FIG. 20 is a flowchart showing an example of an overall processing procedure in the sixth embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the device configuration of an image processing apparatus according to the seventh embodiment. [Figure 14] FIG. 20 is a flowchart showing an example of an overall processing procedure in the seventh embodiment. [Figure 15] FIG. 23 is a diagram showing an example of deviation of discretization positions in the seventh embodiment. [Figure 16] FIG. 13 is a flowchart showing an example of a processing procedure of an image processing apparatus according to the eighth embodiment. [Figure 17] FIG. 20 is a diagram for explaining an example of a method for setting a comparison region in the eighth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, preferred embodiments of the image processing device according to the disclosure of this specification will be described in detail with reference to the accompanying drawings, although the scope of the invention is not limited to the illustrated examples. [Example]

[0010] First Embodiment The image processing device according to this embodiment is a device that generates a 3D difference image between multiple 3D images (first and second images). To calculate differences in the details between the images, the image processing device acquires first and second converted images by converting the resolution of the first and second images so that their pixel size (voxel size) differs from that of the original images. The image processing device then acquires a corresponding position on the second converted image that corresponds to a position of interest on the first converted image, and sets a comparison region on the second converted image centered on the corresponding position. Taking advantage of the fact that the position of the pixel that best reflects the original image signal shifts by up to half a pixel size due to a discretization position shift between the images being compared, the size of the comparison region is determined based on the pixel sizes of the first and second images before resolution conversion. The image processing device then calculates the difference between the position of interest based on the density value of the position of interest on the first converted image and the density values ​​of multiple pixels within a comparison region set around the corresponding position on the second converted image, and generates a 3D difference image in which the calculated value is used as the density value of the position of interest on the 3D difference image. By calculating the difference value from a comparison region of the minimum necessary size, the user can observe a 3D difference image in which noise due to discretization position shifts is reduced while minimizing the elimination of signals necessary for diagnosis, such as differences due to changes over time. The configuration and processing of this embodiment will be described below with reference to Figures 1 to 4.

[0011] 1 shows the configuration of an image diagnostic system according to this embodiment. As shown in the figure, an image processing device 100 according to this embodiment is connected to a data server 110 and a display unit 120. The image diagnostic system may include a modality (imaging device), and images captured by the modality may be transmitted to the image processing device without going through the data server. In this case, the data server may not be included in the system.

[0012] The data server 110 stores a first image and a second image designated by a user as targets for generating a subtraction image. The first image and the second image are three-dimensional tomographic images (volume data) obtained by imaging a subject in advance using the same modality under different conditions (date and time, contrast conditions, imaging parameters, etc.). The modality for imaging the three-dimensional tomographic images may be an MRI device, an X-ray CT device, a three-dimensional ultrasound imaging device, a photoacoustic tomography device, a PET / SPECT device, an OCT device, etc. The first image and the second image may be images of the same patient captured with the same modality in the same position on different dates and times for follow-up observation, or may be images of the same patient captured under different contrast conditions or imaging parameters. Furthermore, the first image and the second image may be images of different patients, or a patient image and a standard image. The first image and the second image are input to the image processing device 100 via the data acquisition unit 1010.

[0013] The display unit 120 is a monitor that displays an image generated by the image processing device 100. Specifically, the display unit 120 is a monitor that displays a difference image generated using the calculated difference value as a density value.

[0014] The image processing device 100 is composed of the following components: The data acquisition unit 1010 acquires a first image and a second image input to the image processing device 100; the converted image acquisition unit 1020 acquires a first converted image and a second converted image obtained by converting the resolution of the first image and the second image, respectively; the deformation information acquisition unit 1030 acquires deformation information representing the correspondence between positions on the first converted image and the second converted image; the comparison region size determination unit 1040 determines the size of the comparison region based on the pixel sizes of the first image and the second image; the position acquisition unit 1050 acquires a focus position on the first converted image and, using the deformation information acquired by the deformation information acquisition unit 1030, acquires a corresponding position on the second converted image that corresponds to the focus position on the first converted image; and the comparison region setting unit 1060 sets a comparison region having the size of the comparison region around the corresponding position on the second converted image. The combination determination unit 1070 determines a combination of the density value of the target position on the first converted image and the density values ​​of multiple positions in the comparison area on the second converted image as the target for comparison processing (difference calculation). The difference calculation unit 1080 calculates a difference value of the target position based on the combination of density values ​​determined by the combination determination unit 1070, i.e., the density value of the target pixel on the first converted image and the density values ​​of multiple pixels in the comparison area on the second converted image. The difference image generation unit 1090 generates a difference image in which the calculated difference value is the density value of the target position. The display control unit 1100 performs display control to display the first image, the second image, and the difference image side by side on the display unit 120.

[0015] FIG. 2 is a flowchart showing the overall processing procedure performed by the image processing device 100.

[0016] (S2000) (Data Acquisition) In step S2000, the data acquisition unit 1010 acquires the first image and the second image input to the image processing device 100. Then, the acquired first image and the second image are output to the converted image acquisition unit 1020. The data acquisition unit 1010 also outputs information about the pixel sizes of the first image and the second image to the comparison region size determination unit 1040.

[0017] (S2010) (Acquisition of converted image) In step S2010, the converted image acquisition unit 1020 acquires a first converted image obtained by converting the resolution of the first image so that the pixel sizes are different, and a second converted image obtained by converting the resolution of the second image so that the pixel sizes are different. For example, if the pixel size of the original image is anisotropic, an image with an isotropic pixel size is acquired so that the image registration performed later can be performed with high accuracy. For example, if the first and second images are general CT images, the resolution within the slice plane is high relative to the distance between the slices, so a process of upsampling pixels in the inter-slice direction to match the resolution within the slice plane is performed. Similarly, if the pixel sizes of the first and second images do not match, a process of converting the resolution of at least one of the images to match the pixel sizes of the respective images is performed. Typically, the resolution is converted to match the image with the higher resolution. The generated converted images are then output to the deformation information acquisition unit 1030, position acquisition unit 1050, comparison region setting unit 1060, combination determination unit 1070, and difference calculation unit 1080.

[0018] If resolution conversion processing is not required (for example, if the pixel sizes of the first image and the second image are isotropic and equal), this step is not performed, and the original image is treated as the converted image and subsequent processing is performed.

[0019] In this embodiment, a known image processing method can be used to interpolate density values ​​during resolution conversion, such as nearest neighbor interpolation, linear interpolation, or cubic interpolation.

[0020] (S2020) (Acquisition of deformation information) In step S2020, the deformation information acquisition unit 1030 acquires deformation information so that pixels representing the same part between the first converted image and the second converted image substantially match. That is, it performs a registration process (deformation estimation process) between the first converted image and the second converted image. Then, it outputs the acquired deformation information to the position acquisition unit 1050. That is, when the first converted image is used as a reference image, the deformation information acquisition unit 1030 acquires deformation information between the reference image and the second converted image.

[0021] In this embodiment, the deformation information is obtained using a known image processing method. For example, the deformation information is obtained by deforming one image so that the image similarity between the deformed images is increased. Image similarity can be calculated using known methods such as commonly used Sum of Squared Difference (SSD), mutual information, or cross-correlation coefficient. Furthermore, image deformation models can be used, such as a radial basis function-based deformation model like Thin Plate Spline (TPS), or a known deformation model like Free Form Deformation (FFD) or Large Deformation Diffeomorphic Metric Mapping (LDDMM). Note that if only differences in position and orientation exist between the first and second images (or can be approximated as such), rigid registration between the images may be performed, and transformation parameters for the position and orientation may be acquired as deformation information. Alternatively, affine transformation parameters between the images may be acquired as deformation information. Note that if there is no positional misalignment between the images (or if there is no positional misalignment between the images), this step is unnecessary.

[0022] (S2030) (Determining the size of the comparison area) In step S2030, comparison region size determination unit 1040 determines the size of the comparison region to be used in calculating the difference value based on the pixel size of the first image and the pixel size of the second image, and outputs the determined size of the comparison region to comparison region setting unit 1060.

[0023] In this embodiment, the size of the comparison region is determined by utilizing the property that, due to a shift in the discretization position during image generation between the first and second images, the pixel positions at which the original imaging signals of the subject are best reflected between the first and second converted images shift by up to the sum of half pixel sizes of the respective original images. That is, the sum of half pixel sizes of the first and second images is determined as the size of the comparison region.

[0024] FIG. 3 illustrates how a discretization position shift between images can cause a discrepancy in the observed values ​​of the same part of a subject. In FIG. 3(a), graphs 3000 and 3040 show how imaging signals appear when the imaging position of the subject relative to the modality is shifted in opposite directions. The vertical axis represents the signal value when the subject is imaged with the modality, and the horizontal axis represents the position in the x-axis direction. Signals 3020 and 3060 are signals acquired by imaging the same subject, and dotted lines 3010 and 3050 indicate the same position on the x-axis. Here, signals 3010 and 3050 are signals acquired when images are taken at the same position, which is conveniently referred to as the reference position. FIG. 3(a) illustrates that when images are generated from signals 3020 and 3060 acquired from the same subject, the discretization positions are shifted in different directions relative to the reference position. Furthermore, the lines 3030 arranged at equal intervals in FIG. 3(a) are the boundary positions of the discretization, and when generating an image, the signals indicated by 3020 and 3060 are combined in the area separated by the lines to generate one pixel.

[0025] In Figure 3(b), graph 3070 represents pixel information generated by discretizing signal 3020 in Figure 3(a), with the vertical axis representing density values ​​and the horizontal axis representing position in the x-axis direction. For example, pixel 3080 in Figure 3(b) is a pixel generated from an area containing a large amount of signal 3020, and is represented by a large density value. Similarly, in Figure 3(b), graph 3090 represents pixel information generated by discretizing signal 3060, with the vertical axis representing density values ​​and the horizontal axis representing position in the x-axis direction. In this case, as described above, the discretization positions of signals 3020 and 3060 are shifted in different directions, so the density values ​​at the same position differ between graphs 3070 and 3090.

[0026] In FIG. 3(c), graphs 3100 and 3120 respectively represent the density values ​​and positions of pixels obtained by converting the resolution of the pixels represented by graphs 3070 and 3090 in FIG. 3(b). That is, graphs 3070 and 3090 can be considered as graphs of the first and second images, and graphs 3100 and 3120 as graphs of the signal values ​​of the first and second converted images. In this case, pixels 3110 and 3130 are pixels that best reflect the object signals 3020 and 3060, respectively. When signals 3020 and 3060 are shifted in opposite directions, the maximum shift in the discretized positions of the signals relative to reference positions 3010 and 3050 is half a pixel size of the original image. When signals 3020 and 3060 are shifted in opposite directions, the shift between pixels 3110 and 3130 is the sum of the respective half-pixel sizes.

[0027] In Figure 3, a one-dimensional graph in only the x-axis direction is used for simplicity. However, in an actual three-dimensional image, the discretization position may shift in each of the x-, y-, and z-axis directions. Therefore, the size of the comparison region is determined based on the pixel size in each axial direction. In general CT images, the resolution within a slice (i.e., pixel size in the x and y directions) is often sufficient, so noise reduction may not be performed in the x and y directions. In this case, the size of the comparison region in the x and y directions may be set to 0, and only the size of the comparison region in the z-axis direction may be determined using the above method. Alternatively, the size of the comparison region in the x and y directions may be set to a predetermined fixed value (e.g., 1 mm), and only the size of the comparison region in the z-axis direction may be determined using the above method. This can speed up calculations.

[0028] In this embodiment, the size of the comparison region is the sum of the half-pixel sizes of the first and second images. However, the sum of the pixel sizes of the first and second images may be multiplied by a predetermined constant (e.g., the expected value of the discretization position offset) or may be added to the sum. The predetermined constant may also be changed based on the registration method used to align the first and second images. For example, when registration is performed to reduce the difference in density values ​​between the images, pixel position offsets due to discretization position offsets may be corrected, and the positions of pixels representing the same part between the images may become closer. Therefore, the predetermined constant may be reduced, and a value smaller than the maximum discretization position offset may be used as the size of the comparison region. This reduces the size of the comparison region and eliminates the need to search excessive areas, thereby improving processing speed and accuracy. Alternatively, the size of the comparison region may be determined by mapping a value obtained by a predetermined nonlinear function using the pixel sizes of the first and second images as input variables. The size of the comparison region may also be determined based on the pixel size of only one of the first and second images. For example, the size of the comparison area may be determined based only on the pixel size of the first image, which is the reference image. Alternatively, the size of the comparison area may be determined based on the pixel size of the larger of the first and second images. In this case, too, the pixel size to be used can be multiplied by a predetermined constant to use as the size of the comparison area.

[0029] Furthermore, in a registration method that reduces the distance between corresponding positions (feature points) between images rather than the difference in density values, pixel misalignment due to discretization position misalignment does not necessarily decrease. Generally, the accuracy of registration is high near the feature points between images used for registration, and the further away from the feature points, the lower the accuracy of registration. When the accuracy of registration is low, the misalignment of pixels between images may increase due to the misalignment of discretization positions plus a registration error. In such cases, the predetermined constant may be changed for each position on the image depending on the distance from the feature points on the image. For example, the predetermined constant may be decreased at positions near the feature points on the image and increased at positions farther away from the feature points.

[0030] (S2040) (Get location) In step S2040, the position acquisition unit 1050 acquires a position of interest on the first converted image (a position of interest on the reference image) and acquires a corresponding position on the second converted image that corresponds to the position of interest using the deformation information acquired in step S2020. That is, the position acquisition unit 1050 corresponds to an example of a position acquisition means that acquires a position of interest on one image and a corresponding position on the other image that corresponds to the position of interest. The position acquisition unit 1050 then outputs the acquired positions to the comparison area acquisition unit 1060, the combination determination unit 1070, and the difference calculation unit 1080.

[0031] (S2050) (Setting the comparison area) In step S2050, the comparison area setting unit 1060 sets a comparison area having the size determined in step S2030 around the corresponding position on the second converted image as its center, and outputs information about the set comparison area to the combination determination unit 1070.

[0032] FIG. 4 is a diagram illustrating the comparison region set on the transformed image. In FIG. 4, graphs 3100 and 3120 represent the same graphs as graphs 3100 and 3120 in FIG. 3(c), respectively. Here, an example is shown in which the pixel size in the x-axis direction is 5 mm in both the first and second images, and the images are upsampled to a pixel size of 1 mm. In this case, the size of the comparison region in the x-axis direction is the sum of the half pixel sizes of the first and second images, resulting in 2.5 mm + 2.5 mm = 5 mm. In FIG. 4, assume that pixel 3110 on the first transformed image is the position of interest, and the corresponding position (calculated based on the deformation information) on the second transformed image is pixel 4030. In this case, 4040 represents a comparison region of 5 mm in size set at pixel 4030, the corresponding position. Considering the possibility that the discretization position may shift by up to 5 mm in the x-axis direction, comparison regions of 5 mm are set in both the +x and -x directions, centered on pixel 4030. Here, only the x-axis direction has been described to simplify the illustration, but in an actual three-dimensional image, the comparison area is a rectangle whose size is determined based on the pixel size in each of the x-, y-, and z-axis directions.

[0033] (S2060) (Determining Combinations) In step S2060, the combination determination unit 1070 determines a combination of density values ​​to be subjected to the comparison process (difference calculation). First, the combination determination unit 1070 interpolates the density value of the position of interest on the first converted image acquired in step S2040 and the density values ​​of pixels in the comparison region on the second converted image. A known image processing method can be used to interpolate the density values. For example, nearest neighbor interpolation, linear interpolation, cubic interpolation, etc. can be used. Interpolation is not necessarily required. Note that the combination determination unit 1070 may interpolate the density values ​​of pixels in the comparison region on the second converted image without interpolating the density value of the position of interest on the first converted image. The combination determination unit 1070 may determine a combination of density values ​​such that a difference is calculated between the density value of the position of interest on the first converted image and the density values ​​of all pixels in the comparison region on the second converted image. The combination determination unit 1070 may also sample at least one pixel from among the pixels included in the comparison region and determine the combination of the density value of that pixel and the density value of the focus position as the combination for which the difference is to be calculated. For example, the combination determination unit 1070 may sample the maximum and minimum density values ​​from among the pixels included in the comparison region. Then, the combination of the maximum density value included in the comparison region and the density value of the focus position and the combination of the minimum density value included in the comparison region and the density value of the focus position are determined as the combinations for which the difference is to be calculated. The sampled density values ​​are not limited to the maximum and minimum, and three or more values ​​may be sampled, or a single value, such as the maximum or minimum density value, may be sampled. Alternatively, a density range with the maximum and minimum density values ​​in the comparison region as its upper and lower limits may be obtained, and the density value of the focus position and the density range in the comparison region may be obtained as the combination for which the difference is to be calculated. The density range in the comparison region may be other than the maximum and minimum density values. For example, it may be the maximum and minimum values ​​after removing outliers from the density values.

[0034] The combination determination unit 1070 outputs information indicating the determined combination of density values ​​to be used for calculating the difference to the difference calculation unit 1080. The information indicating the combination of density values ​​output by the combination determination unit 1070 may be information indicating all combinations of density values ​​of the position of interest on the first converted image and density values ​​of pixels in the comparison area on the second converted image, or may output only information indicating combinations of density values ​​of pixels sampled from the pixels included in the comparison area and density values ​​of the position of interest.

[0035] (S2070) (Calculation of the difference value) In step S2070, the difference calculation unit 1080 obtains a difference value to be assigned to the difference image based on a combination of the density value of the position of interest on the first converted image determined in step S2060 and the density value in the comparison region on the second converted image. The difference calculation unit 1080 calculates the difference between the density value of the position of interest on the reference image and each of the density values ​​at multiple positions in the comparison region on the second converted image, or the difference between the density value at the position with the closest density value among the multiple positions. The result is then output to the difference image generation unit 1090.

[0036] 4, the difference between the density value of pixel 3110 of the first converted image and each density value in comparison region 4040 on the second converted image is calculated, thereby obtaining the difference between pixel 3110 of the first difference image and pixel 3130 of the second converted image where the difference is the smallest. In other words, it is possible to calculate the difference between the density values ​​between pixels that best reflect the signal from the subject in FIG. 3.

[0037] In this embodiment, the pixels in the comparison region on the second converted image for which the difference with the density value of the position of interest on the first converted image is calculated may be all pixels output from the combination determination unit 1070. Alternatively, they may be pixels obtained by sampling the pixels output from the combination determination unit 1070 at predetermined intervals, or pixels obtained by randomly sampling a predetermined number of pixels. Furthermore, they may be pixels within a spherical region inscribed in the comparison region. This reduces the number of times the difference is calculated, thereby enabling faster processing.

[0038] As another method for using the difference between the density value of the target position on the first converted image and the density value of each pixel in the comparison area on the second converted image, instead of the minimum difference, the average difference or the second smallest difference may be calculated and used as the difference value of the difference image. This may leave more noise than obtaining the minimum difference. However, on the other hand, it is possible to prevent the difference between the target pixel on the first converted image and a pixel at a non-corresponding position (such as an artifact or a pixel in another part) from being obtained as the minimum value, even if the difference between the target pixel on the second converted image and the pixel at the first converted image is small. This prevents the difference value from being significantly smaller than the difference between the pixel at the corresponding position.

[0039] Alternatively, distribution information of density values ​​within the comparison region on the second converted image may be calculated, and the difference value may be determined by comparing the distribution information with the density value of the position of interest on the first converted image. For example, if the density range within the comparison region is acquired in step S2060, the difference value may be determined by comparing the density range with the density value of the position of interest on the first converted image. For example, if the density value of the position of interest is included in the density range within the comparison region, the difference value may be set to 0. Furthermore, if the density value of the position of interest is greater than the maximum value (upper limit) of the density range, the difference value from the maximum value (upper limit) may be set as the difference value provided to the difference image. If the density value of the position of interest is smaller than the minimum value (lower limit) of the density range, the difference value from the minimum value (lower limit) may be set as the difference value provided to the difference image. In this case, the density range within the comparison region on the second converted image represents the range in which the density value of the position of interest on the first converted image can change depending on the discretization position. Therefore, even if the density values ​​of the same part between images do not match perfectly due to the effects of discretization and a slight difference remains, the difference can be set to 0 as long as it is within the range that can change depending on the discretization position, thereby further reducing noise.

[0040] However, if the comparison region contains pixels with significantly different density values ​​due to lesions or artifacts, the difference between the maximum and minimum density values ​​in the comparison region may become too large, resulting in the difference at the target position being erroneously set to zero. Therefore, the difference may be set to zero only when the density value at the target position is between the maximum and minimum density values ​​in the comparison region and the difference between the maximum and minimum density values ​​at the target position is equal to or less than a threshold. Alternatively, the distribution of density values ​​in the comparison region may be classified into multiple clusters, and processing similar to the above may be performed based on a comparison of the minimum and maximum values ​​of each cluster. That is, if the density value at the target position is within a cluster, the difference value is set to zero; if it is outside the cluster, the difference value is set to the (signed) distance from the nearest cluster. The density values ​​in the comparison region may be classified based solely on density values, or based on pixel position and density value. In the former case, for example, clustering processing may be performed on a histogram of density values ​​in the comparison region. In the latter case, clustering may be performed by dividing the comparison region into regions based on the continuity of density values ​​between pixels.

[0041] In this embodiment, the comparison region may be set around the corresponding position on the original second image, rather than on the second converted image, and the difference between the density value of the target position on the first converted image and the density value within the comparison region on the second image may be calculated. In this case, density values ​​may be obtained at multiple positions within the comparison region at intervals smaller than the pixel size. The density values ​​at these positions are obtained by interpolating from the surrounding density values. If the interpolation method is the same as when the second converted image was obtained in step S2010, it is possible to calculate difference values ​​equivalent to those when the comparison region is set on the second converted image.

[0042] Alternatively, the difference value calculated based on the density value of the position of interest on the first converted image and the density values ​​of each pixel in the comparison region on the second converted image as described above may be used as the first difference value, and the difference (second difference value) between the density value of the position of interest on the first converted image and the density value of the corresponding position on the second converted image may be calculated separately, and the difference value to be given to the difference image may be calculated based on the first difference value and the second difference value. For example, the weighted average of the first difference value and the second difference value may be used as the difference value of the difference image. This reduces the risk of erasing signals that are not noise.

[0043] (S2080) (Change focus position?) In step S2080, the position acquisition unit 1050 determines whether or not difference values ​​have been calculated at all positions (all pixels) on the first converted image. If difference values ​​have been calculated at all positions, the process proceeds to step S2090. On the other hand, if difference values ​​have not been acquired at all positions, the process returns to step S2040.

[0044] In this embodiment, the difference values ​​may be calculated not only for all positions on the first converted image but also for some positions on the first converted image that have been extracted in advance using a known image processing technique, thereby reducing the processing time required for noise reduction.

[0045] (S2090) (Generation of difference image) In step S2090, the difference image generating unit 1090 generates a difference image (first difference image) in which the difference value at each position on the first converted image is used as the density value. The obtained difference image is then stored in the data server 110. Alternatively, it may be stored in a memory within the image processing device 100. It is also output to the display control unit 1100. Note that a general difference image (second difference image) in which the second difference value calculated in step S2060 (the difference between the density value at the position of interest on the first converted image and the density value at the corresponding position on the second converted image) is used as the density value may also be generated.

[0046] (S2100) (Display of difference image) In step S2100, the display control unit 1100 controls the display unit 120 to display the difference image (first difference image) generated in step S2090.

[0047] As an example of the display, for example, a single screen may be divided vertically or horizontally to display the first image, the second image, and the difference image side by side. Alternatively, a difference image (first difference image) drawn in a different color from the first image or the second image may be superimposed and displayed, or only one of the first image, the second image, and the difference image may be selected and displayed (freely switched to the same position). Alternatively, one image may be enlarged or reduced to match the resolution of the other image, or they may be displayed side by side so that a corresponding position in the second image and a corresponding position in the difference image are aligned with a position of interest on the first image. Alternatively, the first difference image and the second difference image may be displayed by switching between them.

[0048] In this way, the processing of the image processing device 100 is carried out.

[0049] As a result, by calculating the difference value from a comparison area of ​​the minimum necessary size that takes into account the shift in discretization position, the user can observe a difference image in which the necessary signals are retained in the difference image and noise resulting from differences in density values ​​due to the shift in discretization position between images is reduced.

[0050] (Variation 1-1) (Calculating image similarity taking into account discretization position shift) In this embodiment, a method for calculating the difference between pixels of interest by setting a comparison region around the pixel based on the pixel size and using the density values ​​within that region was used to reduce noise during the generation of the difference image. However, this method can also be used in other situations where pixels of interest are compared. For example, in step S2020, the deformation information acquisition unit 1030 acquires image similarity using a known image processing method. However, this method can also be used in the process of acquiring image similarity for image registration. That is, when calculating the density value difference between a pixel pair between images, as in SSD, the pixel pair may be set as a target position and a corresponding position, and the difference value may be calculated using processes similar to those in steps S2050, S2060, and S2070. In registration, which repeatedly optimizes deformation information to increase image similarity, the image similarity obtained at each optimization step is evaluated based on the difference value calculated using the same steps as the noise-reduced image ultimately viewed by the user. This allows image similarity to be calculated taking into account the discretization position shift, resulting in a difference image with reduced noise.

[0051] (Variation 1-2) (Reducing noise at a position specified by the user) In this embodiment, in step S2080, all positions (all pixels) on the first converted image or a portion of positions extracted in advance using a known image processing technique are processed. However, the user may specify the positions. That is, the noise-reduced difference image generation process may be performed for all positions or a portion of positions within a region specified in advance by the user. Alternatively, only the second difference image may be generated and displayed, and the processes of steps S2040 to S2090 may be performed only on the region adjacent to the position of interest interactively specified by the user, and only this region may be replaced with the first difference image and displayed. This reduces the time required for noise reduction by reducing noise only at the minimum necessary positions. Furthermore, positions that the user determines are not noise can be excluded from the noise reduction process, preventing excessive signal erasure.

[0052] Second Embodiment The image processing device according to this embodiment, like the first embodiment, generates a 3D difference image between a first image and a second image. The image processing device according to this embodiment is characterized by determining whether to perform noise reduction processing based on the pixel size of the input image and generating a 3D difference image according to the determination. Like the first embodiment, the image processing device according to this embodiment determines the size of a comparison region based on the pixel sizes of the first and second images. If the pixel sizes of the first and second images are less than a predetermined threshold, the device determines that the discretization position misalignment is small and generates a difference image without performing noise reduction processing. On the other hand, if the pixel sizes are equal to or greater than the predetermined threshold, the device generates a difference image after performing noise reduction processing, like the first embodiment. This prevents excessive elimination of the density value difference between images when the discretization position misalignment is small. Furthermore, the processing time required for noise reduction can be shortened. The configuration and processing of this embodiment will be described below using Figures 1 and 5.

[0053] The configuration of the image processing device according to this embodiment is the same as that of the first embodiment. However, the comparison region size determination unit 1040 and the difference calculation unit 1080 have different functions from those of the first embodiment, and therefore, their functions will be explained below. The other components have the same functions as those of the first embodiment, and therefore, explanations thereof will be omitted.

[0054] The comparison region size determination unit 1040 determines whether or not to perform noise reduction processing based on the pixel sizes of the first and second images. As in the first embodiment, the size of the comparison region is determined based on the pixel sizes of the first and second images. If noise reduction processing is not to be performed, the difference calculation unit 1080 calculates the difference value between the density value of the target position on the first converted image and the density value of the corresponding position on the second converted image as the difference value of the target position. On the other hand, if noise reduction processing is to be performed, the same processing as in the first embodiment is performed.

[0055] 5 shows a flowchart of the overall processing procedure performed by the image processing device 100. Steps S5000 to S5020 and S5030 to S5100 perform the same processing as steps S2000 to S2020 and S2030 to S2100 in the first embodiment, respectively, and therefore explanations thereof will be omitted. Only the differences from the flowchart in FIG. 2 will be explained below.

[0056] (S5025) (Do you perform noise reduction processing?) In step S5025, the comparison region size determination unit 1040 determines whether or not to perform noise reduction processing based on the pixel sizes of the first and second images.

[0057] Here, if the pixel sizes of both the first image and the second image are less than a predetermined threshold, it is determined that the discretization position deviation is small, it is determined that noise reduction is not necessary, and the process proceeds to step S5100. On the other hand, if the pixel sizes are equal to or greater than the predetermined threshold, the process proceeds to step S5030, where noise reduction processing similar to that in the first embodiment is performed. For example, the threshold is set to a value according to the lesion size or the resolution of the modality, or a value predetermined by the user.

[0058] In this embodiment, the determination of whether to perform noise reduction processing is based on the pixel size of each of the first and second images. However, the determination may also be based on the sum of the pixel sizes of each of the first and second images. The predetermined threshold may be different for each of the x, y, and z axes of the image. The necessity of noise reduction may also be determined for each axis, and the implementation or non-implementation of noise reduction processing may be controlled for each axis. For example, if the pixel sizes in the x and y axes of each of the first and second images are less than a threshold value and the pixel size in the z axis direction is equal to or greater than the threshold value, it may be determined that noise reduction is unnecessary in the x and y axes, and the size of the comparison region may be set to 0. It may also be determined that noise reduction is necessary only in the z axis direction, and the size of the comparison region may be determined as in the first embodiment. If it is determined that noise reduction is unnecessary in all three axes, the process proceeds to step S5100. Otherwise, the process proceeds to step S5030.

[0059] In general, since the resolution within a slice (= pixel size in the x and y directions) is often sufficient in CT images, it is possible to always not perform noise reduction in the x and y directions without making a determination based on the image size. In other words, it is only necessary to determine whether or not to perform noise reduction in the z direction based on the slice interval (= pixel size in the z direction) of the input image. In this way, it is possible to determine not to perform noise reduction if the input image is thin sliced, and to perform noise reduction if it is thick sliced.

[0060] (S5110) (Calculation of the difference value at the corresponding position) In step S5110, the difference calculation unit 1080 calculates the difference value (the second difference value in the first embodiment) between the density values ​​of corresponding positions between the first converted image and the second converted image. At this time, the corresponding positions between the images are obtained using the deformation information, as in step S5040.

[0061] According to this embodiment, by determining whether noise reduction processing is necessary depending on the pixel size, when the pixel size of the input image is sufficiently small and the noise due to discretization is small, unnecessary calculations can be omitted. Also, compared to the first embodiment, there is an advantage in that it is possible to prevent excessive erasure of the original difference value.

[0062] <Third embodiment> The image processing device according to this embodiment, like the first embodiment, is a device that generates a 3D difference image between a first image and a second image. The image processing device according to this embodiment is characterized by generating a noise-reduced 3D difference image using a deformed comparison area obtained by projecting a comparison area onto a deformed image that has been deformed and aligned so that identical portions of the images substantially coincide. Like the first embodiment, the image processing device according to this embodiment acquires a first converted image and a second converted image obtained by converting the first image and the second image so that their resolutions differ from those of the original images. The image processing device then sets a comparison area around a corresponding position on the second converted image that corresponds to a position of interest on the first converted image. The image processing device then projects the comparison area using deformation information onto the second deformed and aligned image obtained by deforming and aligning the second converted image so that the density values ​​of each pixel are similar to those of the first converted image. Then, the difference of the focus position is calculated based on the density value of the focus position on the first transformed image and the density values ​​of multiple pixels in a transformation comparison region set around the corresponding position on the second transformed transformed image, and a 3D difference image is generated with this value as the density value of the focus position on the 3D difference image. Because a transformed image and a difference image of a second image in which the positions of the same parts on the first image and the second image are approximately the same can be obtained, by displaying these images side by side, the user can easily confirm which positions in the first image and the transformed image were used to calculate the difference value on the difference image. The configuration and processing of this embodiment will be described below with reference to Figures 6 and 7.

[0063] 6 shows the configuration of an image diagnostic system according to this embodiment. Here, the data server 110 and the display unit 120 are the same as those in the first embodiment, and therefore their explanations will be omitted. The image processing device 600 is composed of the following components: a data acquisition unit 1010, a converted image acquisition unit 1020, a deformation information acquisition unit 1030, a comparison region size determination unit 1040, a position acquisition unit 1050, a comparison region setting unit 1060, a difference image generation unit 1090, and a display control unit 1100. Since these have the same functions as those in the first embodiment, their explanations will be omitted.

[0064] The transformed image acquisition unit 1610 acquires a second transformed / transformed image by transforming the second transformed image using the transformation information acquired by the transformation information acquisition unit 1030. The comparison area projection unit 1620 acquires a transformed comparison area by projecting the comparison area set on the second transformed image by the comparison area setting unit 1060 onto the second transformed / transformed image using the transformation information acquired by the transformation information acquisition unit 1030. The combination determination unit 1070 determines a combination of the density value of the target position on the first transformed image and the density values ​​of multiple positions in the comparison area on the second transformed image as targets for comparison processing (difference calculation). The difference calculation unit 1080 calculates a difference value of the target position based on the combination of density values ​​determined by the combination determination unit 1070, i.e., the density value of the target pixel on the first transformed image and the density values ​​of multiple pixels in the comparison area on the second transformed image. Figure 7 shows a flowchart of the overall processing procedure performed by the image processing device 600. Steps S7000 to S7050 and S7080 to S7100 are similar to steps S2000 to S2050 and S2080 to S2100 in the first embodiment, respectively, and therefore will not be described further. Only the differences from the flowchart in FIG.

[0065] (S7052) (Acquisition of deformation image) In step S7052, the deformed image acquisition unit 1610 acquires a second deformed and transformed image by deforming the second transformed image using the deformation information acquired in step S7020. The acquired second deformed and transformed image is then output to the comparison area projection unit 1720 and the difference calculation unit 1080.

[0066] (S7054) (Projection of comparison area) In step S7054, the comparison area projection unit 1620 projects the comparison area set on the second converted image in step S7050 onto the second transformed converted image as a transformed comparison area using the transformation information acquired in step S7020, and outputs it to the difference calculation unit 1080.

[0067] Here, the comparison area on the second transformed image to be projected may be the entire area within the comparison area, or multiple locations sampled from within the comparison area. When the entire area within the comparison area is projected, the entire area within the outline of the comparison area projected onto the second transformed image using the transformation information can be used as the transformed comparison area.

[0068] (S7060) (Determining Combinations) In step S7060, the combination determination unit 1070 determines a combination of density values ​​to be used for comparison (difference calculation). First, the combination determination unit 1070 interpolates the density value of the target position on the first converted image acquired in step S7040 and the density values ​​of pixels in the comparison area on the second converted image. Note that known image processing techniques can be used to interpolate the density values. For example, nearest neighbor interpolation, linear interpolation, cubic interpolation, etc. can be used. The density values ​​of pixels in the comparison area on the second converted image do not necessarily need to be interpolated. A combination of density values ​​may be determined so that the difference between the density value of the target position on the first converted image and the density values ​​of all pixels in the comparison area on the second converted image is calculated. Alternatively, at least one pixel may be sampled from the pixels included in the comparison area, and the combination of the density value of that pixel and the density value of the target position may be used as the combination for calculating the difference. For example, the combination determination unit 1070 samples the maximum and minimum density values ​​from the pixels included in the comparison area. Then, the combination of the maximum density value included in the comparison region and the density value of the position of interest, and the combination of the minimum density value included in the comparison region and the density value of the position of interest are determined as combinations for which the difference is to be calculated. The density values ​​to be sampled are not limited to the maximum and minimum, and three or more values ​​may be sampled, or a single value such as the maximum or minimum density value may be sampled. Alternatively, a density range with the maximum and minimum density values ​​in the comparison region as its two ends (upper and lower limits) may be obtained, and the density value of the position of interest and the density range in the comparison region may be obtained as the combination for which the difference is to be calculated. The density range in the comparison region may be other than the maximum and minimum density values. For example, it may be the maximum and minimum values ​​after removing outliers from the density values.

[0069] The combination determination unit 1070 outputs information indicating the determined combination of density values ​​to be used for calculating the difference to the difference calculation unit 1080. The information indicating the combination of density values ​​output by the combination determination unit 1070 may be information indicating all combinations of density values ​​at the position of interest on the first converted image and density values ​​of pixels in the comparison region on the second converted image. Alternatively, the combination determination unit 1070 may output only information indicating combinations of density values ​​at the position of interest and density values ​​of pixels sampled from the pixels included in the comparison region.

[0070] (S7070) (Calculation of the difference value) In step S7070, difference calculation unit 1080 obtains a difference value to be given to the difference image based on a combination of the density value of the position of interest on the first converted image obtained in step S7060 and the density value within the deformation comparison area on the second transformed image projected in step S7054. The only difference from the processing of step S2060 in the first embodiment is that the second converted image is changed to the second transformed image and the comparison area is changed to a deformation comparison area; otherwise, the processing is the same.

[0071] In this way, the processing of the image processing device 600 is carried out.

[0072] According to this embodiment, it is possible to easily compare and observe the same part between a deformed image of a second image in which the position of the same part on the first image and the image is approximately the same, and a noise-reduced subtraction image, which is a deformed image of the same part on the first image and the second image, and therefore, compared to the first embodiment, it has the advantage that the user can easily determine whether the difference value on the subtraction image is due to a lesion or not.

[0073] (Variation 3-1) (Calculating image similarity taking into account discretization position shift) As in Modification 1-1 of the first embodiment, image similarity may be obtained based on difference values ​​in which noise due to discretization position shifts is reduced when calculating image similarity. Here, the minimum value of the difference between the density value of the target position on the first transformed image and each density value within the deformation comparison region on the second transformed image is used. This makes it possible to obtain image similarity that takes into account the discretization position shifts, resulting in a difference image with reduced noise.

[0074] <Fourth embodiment> Similar to the first embodiment, the image processing device according to this embodiment generates a difference image between three-dimensional images. The image processing device according to this embodiment is characterized by setting comparison regions at both a position of interest in a first converted image and a corresponding position in a second converted image, and calculating a difference value at the position of interest based on the density values ​​of pixels in each comparison region. As in the first embodiment, the direction in which a comparison region is set on the second converted image and the difference between the position of interest in the first converted image is calculated is referred to as the forward direction. Similarly, the direction in which a comparison region is set on the first converted image and the difference between the position of interest in the second converted image is calculated is referred to as the reverse direction. The image processing device according to this embodiment calculates difference values ​​in both the forward and reverse directions at the position of interest in the difference image, and combines these two difference values ​​to create a representative difference value, which serves as the density value of the generated difference image. This allows signals that cannot be captured by calculations in only one direction to be retained in the difference image. The configuration and processing of this embodiment are described below using Figures 1 and 8.

[0075] The configuration of the image processing device according to this embodiment is the same as that of the first embodiment. However, the functions of the comparison area setting unit 1060, combination determination unit 1070, and difference calculation unit 1080 are different from those of the first embodiment, and therefore, these functions will be explained below. The functions of the other components are the same as those of the first embodiment, and therefore explanations will be omitted.

[0076] The comparison area setting unit 1060 sets comparison areas each having the size of the comparison area around a position of interest on the first converted image and around a corresponding position on the second converted image. The combination determination unit 1070 determines a combination of the density value of the position of interest on the first converted image and the density values ​​of multiple positions within the comparison area on the second converted image as targets for comparison processing (difference calculation). The difference calculation unit 1080 calculates difference values ​​in the forward direction and in the reverse direction based on the combination of density values ​​determined by the combination determination unit 1070, and acquires a representative difference value by integrating the two difference values.

[0077] 8 shows a flowchart of the overall processing procedure performed by the image processing device 100. Steps S8000 to S8040 and S8080 to S8100 perform the same processing as steps S2000 to S2040 and S2080 to S2100 in the first embodiment, respectively, and therefore explanations thereof will be omitted. Only the differences from the flowchart in FIG. 2 will be explained below.

[0078] (S8050) (Setting the comparison area) In step S8050, comparison area setting unit 1060 sets a first comparison area having the size of the comparison area determined in step S8030, around the position of interest on the first converted image. Similarly to step S2050 in the first embodiment, it also sets a second comparison area having the size of the comparison area determined in step S8030, around the corresponding position on the second converted image. Information about the set first and second comparison areas is then output to combination determination unit 1070.

[0079] (S8060) (Determining bidirectional combinations) In step S8060, the combination determination unit 1070 interpolates the density value of the focus position on the first converted image acquired in step S8040 and the density value of the pixel in the comparison region on the second converted image to determine a combination (forward direction). Furthermore, the combination determination unit 1070 interpolates the density value of the corresponding position on the second converted image and the density value of each pixel in the first comparison region on the first converted image to determine a combination (reverse direction). This process is simply a matter of swapping the focus position and the corresponding position, and can be performed in the same manner as step S2060 in the first embodiment. Note that the various methods described in step S2060 can be used to determine a combination.

[0080] (S8070) (Calculation of bidirectional differential values) In step S8070, the difference calculation unit 1080 calculates a (forward) difference value at the target position based on a combination of the density value at the target position on the first converted image and each pixel in the second comparison region on the second converted image, similar to step S2070 in the first embodiment. Furthermore, the difference calculation unit 1080 calculates a (reverse) difference value at the target position based on the density value at the corresponding position on the second converted image and each pixel in the first comparison region on the first converted image. This process is simply a matter of swapping the target position and the corresponding position, and can be performed in the same manner as step S2070 in the first embodiment. Note that the calculation of the difference value can use the various methods described in step S2070.

[0081] (S8075) (Calculation of representative difference value) In step S8075, the difference calculation unit 1080 combines the forward difference value and the reverse difference value calculated in step S8070 to calculate a representative difference value. For example, the absolute values ​​of both are compared, and the difference value with the larger absolute value is acquired as the representative difference value. The acquired representative difference value is then output to the difference image generation unit 1080 as the density value of the difference image at the target position.

[0082] In the case of a small lesion signal that exists only in one of the first and second images, the differential value may be captured in either the forward or reverse direction, but not in the other direction. Although the signal may disappear in the differential value in one direction, by calculating a representative differential value using the differential values ​​in both directions, such differential values ​​can be retained on the differential image.

[0083] In this embodiment, the difference value with the larger absolute value is used as the representative difference value, but the difference value with the smaller absolute value may also be used as the representative difference value. Also, the average value of the forward difference value and the reverse difference value may also be used as the representative difference value. This reduces the difference value with respect to signals that exist only in one image, while further suppressing noise signals.

[0084] In this embodiment, the size of the comparison region does not necessarily have to be adaptively determined based on the pixel size of the input image. That is, a comparison region size suitable for a typical pixel size may be set as a default value in advance. In this case, the comparison region size determination unit 1040 and the processing of step S8030 are unnecessary.

[0085] This embodiment has the advantage of reducing the risk of erasing signals such as small lesions that exist only in one image, or of lesions being displayed smaller than their actual size, compared to the first embodiment.

[0086] Fifth Embodiment The image processing device according to this embodiment is a device that generates a three-dimensional difference image between a plurality of three-dimensional images (first and second images), similar to the first embodiment. In this embodiment, an example of implementing the present invention with a simpler configuration will be described. The configuration and processing of this embodiment will be described below with reference to FIGS. 9 and 10.

[0087] 9 shows the configuration of an image diagnostic system according to this embodiment. The data server 110 and the display unit 120 have the same functions as those in the first embodiment, so a description thereof will be omitted.

[0088] The image processing device 900 is composed of the following components: a data acquisition unit 1010, a comparison region size determination unit 1040, a difference image generation unit 1090, and a display control unit 1100. The functions of the data acquisition unit 1010, the comparison region size determination unit 1040, the difference image generation unit 1090, and the display control unit 1100 are the same as those in the first embodiment, and therefore descriptions thereof will be omitted. The functions of the other components will be described below.

[0089] The deformation information acquisition unit 1030 acquires deformation information that represents the correspondence between positions on the first image and the second image. The position acquisition unit 1050 acquires a focus position on the first image and acquires a corresponding position on the second image that corresponds to the focus position on the first image using the deformation information acquired by the deformation information acquisition unit 1030. The comparison area setting unit 1060 sets a comparison area having the size of the comparison area around the corresponding position on the second image. The combination determination unit 1070 determines a combination of the density value of the focus position on the first converted image and the density values ​​of multiple positions within the comparison area on the second converted image as targets for comparison processing (difference calculation). The difference calculation unit 1080 calculates a difference value of the focus position based on the combination of density values ​​determined by the combination determination unit 1070, i.e., the density value of the focus pixel on the first converted image and the density values ​​of multiple pixels within the comparison area on the second converted image.

[0090] 10 is a flowchart showing the overall processing procedure performed by the image processing device 900. Step S10100 performs the same processing as step S2100 in the first embodiment, and therefore its explanation will be omitted. Only the differences from the flowchart in FIG. 2 will be explained below.

[0091] (S10000) (Data Acquisition) In step S10000, data acquisition unit 1010 acquires the first image and second image to be input to image processing device 900. Then, data acquisition unit 1010 outputs the acquired first image and second image to deformation information acquisition unit 1030, position acquisition unit 1050, comparison area setting unit 1060, combination determination unit 1070, and difference calculation unit 1080. Information regarding the pixel sizes of the first image and second image is also output to comparison area size determination unit 1040.

[0092] (S10020) (Acquisition of deformation information) In step S10020, deformation information acquisition unit 1030 acquires deformation information so that pixels representing the same part between the first image and the second image are approximately the same. That is, it performs a registration process (deformation estimation process) between the first image and the second image. Then, it outputs the acquired deformation information to position acquisition unit 1050.

[0093] In this embodiment, the deformation information can be obtained by a known image processing method, similar to the first embodiment.

[0094] (S10030) (Determining the size of the comparison area) In step S10030, comparison region size determination unit 1040 determines the size of the comparison region to be used in calculating the difference value based on the pixel size of the first image and the pixel size of the second image, and outputs the determined size of the comparison region to comparison region setting unit 1060.

[0095] In this embodiment, similarly to the first embodiment, the property that the pixel position that best reflects the original imaging signal of the subject shifts by a maximum of the sum of half pixel sizes of the respective original images due to a shift in the discretization position between the first image and the second image when generating the images is utilized. That is, the sum of half pixel sizes of the first image and the second image is determined as the size of the comparison region.

[0096] (S10040) (Getting location) In step S10040, the position acquisition unit 1050 acquires a position of interest (pixel of interest) on the first image, and acquires a corresponding position on the second image that corresponds to the position of interest using the deformation information acquired in step S10020. The acquired position is then output to the comparison area setting unit 1060, the combination determination unit 1070, and the difference calculation unit 1080.

[0097] (S10050) (Setting the comparison area) In step S10050, comparison area setting unit 1060 sets a comparison area having the size determined in step S10030 around the corresponding position on the second image as its center, and outputs information about the set comparison area to combination determination unit 1070.

[0098] (S10060) (Determining combinations) In step S10060, the combination determination unit 1070 determines a combination of density values ​​to be used for comparison (difference calculation). First, the combination determination unit 1070 interpolates the density value of the target position on the first converted image acquired in step S10040 and the density values ​​of pixels in the comparison area on the second converted image. Note that known image processing techniques can be used to interpolate the density values. For example, nearest neighbor interpolation, linear interpolation, cubic interpolation, etc. can be used. The density values ​​of pixels in the comparison area on the second converted image do not necessarily need to be interpolated. A combination of density values ​​may be determined so that the difference between the density value of the target position on the first converted image and the density values ​​of all pixels in the comparison area on the second converted image is calculated. Alternatively, at least one pixel may be sampled from among the pixels included in the comparison area, and the combination of the density value of that pixel and the density value of the target position may be used as the combination for calculating the difference. For example, the combination determination unit 1070 samples the maximum and minimum density values ​​from among the pixels included in the comparison area. Then, the combination of the maximum density value included in the comparison region and the density value of the position of interest, and the combination of the minimum density value included in the comparison region and the density value of the position of interest are determined as combinations for which the difference is to be calculated. The density values ​​to be sampled are not limited to the maximum and minimum, and three or more values ​​may be sampled, or a single value such as the maximum or minimum density value may be sampled. Alternatively, a density range with the maximum and minimum density values ​​in the comparison region as its two ends (upper and lower limits) may be obtained, and the density value of the position of interest and the density range in the comparison region may be obtained as the combination for which the difference is to be calculated. The density range in the comparison region may be other than the maximum and minimum density values. For example, it may be the maximum and minimum values ​​after removing outliers from the density values.

[0099] (S10070) (Calculation of the difference value) In step S10070, the difference calculation unit 1080 obtains a difference value to be assigned to the difference image based on a combination of the density value of the position of interest on the first image determined in step S10060 and density values ​​of multiple positions in the comparison region on the second image, and outputs the difference value to the difference image generation unit 1090.

[0100] In the first embodiment, the difference value is calculated from the density values ​​of the reference image (first image or first converted image) and the second converted image. In this embodiment, the difference value is calculated from the density values ​​of the first image and the second image, as in step S2070 of the first embodiment. That is, the difference between the density value of the target position on the first image and the density values ​​at each of multiple positions in the comparison area on the second image is calculated, and the smallest value among these is obtained as the difference value to be assigned to the difference image. Note that when obtaining density values ​​from multiple positions in the comparison area on the second image, the positions of all pixels in the comparison area may be used as the multiple positions, and density values ​​may be obtained from each pixel. Alternatively, measurement points may be set within the comparison area at a predetermined interval finer than the pixel pitch, and the density value at each measurement point may be obtained by interpolating the density values ​​of neighboring pixels.

[0101] (S10080) (Change focus position?) In step S10080, position acquisition unit 1050 determines whether difference values ​​have been calculated for all positions (all pixels) on the first image. If difference values ​​have been calculated for all positions, processing proceeds to step S10090. On the other hand, if difference values ​​have not been acquired for all positions, processing returns to step S10040.

[0102] In this embodiment, the difference values ​​may be calculated not only for all positions on the first image but also for some positions on the first image that have been extracted in advance using a known image processing technique, thereby reducing the processing time required for noise reduction.

[0103] (S10090) (Generation of difference images) In step S10090, the difference image generating unit 1090 generates a difference image (first difference image) in which the difference value at each position (pixel) on the first image is used as the density value. The obtained difference image is then stored in the data server 110. It is also output to the display control unit 1100. Note that a general difference image (second difference image) in which the second difference value calculated in step S10070 (the difference between the density value at the position of interest on the first image and the density value at the corresponding position on the second image) is used as the density value may also be generated.

[0104] In this way, the processing of the image processing device 900 is carried out.

[0105] As a result, the same effect as in the first embodiment can be obtained without acquiring the first converted image and the second converted image. That is, by calculating the difference value from a comparison region of the minimum necessary size that takes into account the shift in discretization position, the user can observe a difference image in which the necessary signals remain in the difference image and noise caused by the difference in density values ​​due to the shift in discretization position between images is reduced. Note that each of the second to fifth embodiments can also be implemented without acquiring the first converted image and the second converted image, as in this embodiment.

[0106] Sixth Embodiment The image processing device according to this embodiment is a device that generates a three-dimensional difference image between a first image and a second image, similar to the first embodiment. However, the image processing device according to this embodiment is characterized in that, when the pixel sizes of the images differ, it generates a difference image in which noise caused by differences in density values ​​between the images due to the different pixel sizes is reduced. The image processing device according to this embodiment will be described below.

[0107] Typically, when reconstructing an image by converting continuous signal data obtained from a subject, such as a CT image, into discrete pixel density values, a weighted average of signal data within a predetermined interval (e.g., the pixel size within the plane of the slice image or the slice thickness) is used. That is, the density values ​​of pixels with larger pixel sizes are calculated by smoothing a wider range of signal data compared to pixels with smaller pixel sizes. In the image processing device of this embodiment, when the pixel size of the second image is smaller than that of the first image, the density values ​​of the second image are approximately regarded as signal data. The image processing device of this embodiment then smooths the density values ​​of pixels within a region on the second image that is the same size as one pixel of the first image, thereby approximately calculating density values ​​generated from signal data within the region that is the same size as one pixel of the first image, and generates a second smoothed image by smoothing the second image. The generated second smoothed image is then used instead of the second image to generate a difference image similar to that of the first embodiment. This selects a smoothed region on the second image that minimizes the difference in density value from the target pixel of the first image, thereby absorbing changes in density values ​​due to slight shifts in the smoothed region. On the other hand, if the pixel size of the first image is smaller than that of the second image, a process is performed in which a first smoothed image obtained by smoothing the first image is used instead of the first image to generate a difference image similar to that of the first embodiment. This allows the user to observe a difference image with less noise than in the first embodiment when two images with different pixel sizes are input. The configuration and process of this embodiment will be described below with reference to Figures 11 and 12.

[0108] 11 shows the configuration of an image diagnostic system according to this embodiment. The function of a smoothed image acquisition unit 11010 will be described below. The other components have the same functions as those in the first embodiment, so their description will be omitted.

[0109] The smoothed image acquisition unit 11010 determines which of the pixel size of the first image and the pixel size of the second image acquired from the data acquisition unit 1010 is larger, and if the pixel size of the second image is smaller than the pixel size of the first image, acquires a second smoothed image obtained by smoothing the second image. On the other hand, if the pixel size of the first image is smaller than the pixel size of the second image, acquires a first smoothed image obtained by smoothing the first image.

[0110] 12 shows a flowchart of the overall processing procedure performed by the image processing device 1100. Steps S12030 to S12110 perform the same processing as steps S2020 to S2100 in the first embodiment, respectively, and therefore explanations thereof will be omitted. Only the differences from the flowchart in FIG. 2 will be explained below.

[0111] (S12000) (Data Acquisition) In step S12000, the data acquisition unit 1010 acquires the first image, the second image, and the pixel size of each image input to the image processing device 1100. Then, the data acquisition unit 1010 outputs the acquired first image, the second image, and the pixel size of each image to the smoothed image acquisition unit 11010. In addition, the data acquisition unit 1010 outputs information regarding the pixel sizes of the first image and the second image to the comparison region size determination unit 1040.

[0112] (S12010) (Acquisition of smoothed image) In step S12010, the smoothed image acquisition unit 11010 determines which of the pixel sizes of the first image and the second image acquired from the data acquisition unit 1010 is larger. If the first image has the larger first pixel size of the two pixel sizes (if the pixel size of the second image is smaller than the pixel size of the first image), the smoothed image acquisition unit 11010 smoothes the second image to acquire a second smoothed image. Then, the smoothed image acquisition unit 11010 outputs the acquired second smoothed image and the first image to the converted image acquisition unit 1020. In this case, the image processing device 1100 replaces the second image with the second smoothed image and performs subsequent processing. On the other hand, if the first image has the smaller second pixel size of the two pixel sizes, the smoothed image acquisition unit 11010 acquires a first smoothed image by smoothing the first image. Then, the smoothed image acquisition unit 11010 outputs the acquired first smoothed image and second image to the converted image acquisition unit 1020. In this case, the image processing device 1100 replaces the first image with the first smoothed image and performs the subsequent processes. Note that the smoothed image acquisition unit 11010 is able to perform the above processes because it acquires the pixel size of each image from the data acquisition unit 1010.

[0113] Specifically, an example in which the first image has a first pixel size will be described. In this embodiment, the smoothed image acquisition unit 11010 performs processing on the second image, which has a smaller pixel size, similar to the processing performed when reconstructing the first image, which has a larger pixel size, from signal data. Therefore, the smoothed image acquisition unit 11010 smoothes the second image using a kernel of the same size as the pixel size of the first image. For example, assume that the pixel sizes of the first image in the x, y, and z axes are 1.5 mm, 1.5 mm, and 5 mm, respectively, and the pixel sizes of the second image are 0.5 mm, 0.5 mm, and 1 mm, respectively. In this case, the kernel size is 1.5 mm × 1.5 mm × 5 mm, which corresponds to a size of 3 × 3 × 5 pixels on the second image. A known smoothing process is performed using this kernel. The smoothing process may be a process of uniformly averaging the density values ​​of 3 × 3 × 5 pixels, or a process of calculating a weighted average, such as smoothing using a Gaussian filter. In general CT images, the resolution within a slice (= pixel size in the x and y directions) is often sufficiently high, so noise reduction in the x and y directions may not be required. In this case, the kernel size in the x and y directions is set to 1 pixel, and only the density value in the z direction is smoothed.

[0114] In the above example, the pixel size ratio between the images is an odd multiple. That is, the kernel size is an odd number of pixels in each axial direction on the second image. Below, an example in which the pixel size ratio between the images is an even multiple will be described. For example, if the pixel sizes in the x, y, and z axial directions of the first image are 0.5 mm, 0.5 mm, and 4 mm, respectively, and the pixel sizes in the second image are 0.5 mm, 0.5 mm, and 1 mm, respectively, the kernel size is 1 x 1 x 4 pixels on the second image. Generally, when performing filtering using a kernel, the kernel size in the axial direction is an odd number of pixels, and the processing is performed by using the pixel to be filtered and the pixels in the positive and negative directions of the axis centered on that pixel (for five pixels, two pixels each). However, when the kernel size is an even number of pixels, such as four pixels, it is not possible to use the pixels in the positive and negative directions of the axis equally. Therefore, an image is generated by converting the resolution of the second image so that the pixel size of the first image is an odd multiple of that of the second image. More specifically, the pixel size of the second image is converted so that the pixel size of the first image is an odd multiple of the image obtained by converting the resolution of the second image, and is the closest value less than or equal to the pixel size of the original second image. In the above example, since the pixel size is an even multiple only in the z-axis direction, the resolution is converted only in the z-axis direction. That is, by converting the pixel size of the second image to 0.5 mm, 0.5 mm, and 0.8 mm, the kernel size in the resolution-converted image becomes 1 x 1 x 5 pixels, and the density values ​​can be smoothed using pixels in both the positive and negative directions of the axis equally. In this case, the pixel size of the first image may be converted so that the pixel size of the first image is an odd multiple of the image obtained by converting the resolution of the second image, and is the closest value less than or equal to the pixel size of the original first image.

[0115] In this embodiment, the density values ​​of the second image are approximately regarded as signal data, and the density values ​​of the smoothed image are calculated from this approximate signal data. The method for calculating density values ​​from this signal data is preferably similar to the method for actually generating density values ​​of the first image from signal data and reconstructing the image. In other words, if the reconstruction algorithm for the first image is known, the density values ​​of the smoothed image may be calculated according to that algorithm.

[0116] In the above example, an example was described in which the first image has a first pixel size, but if the second image has a first pixel size, a first smoothed image can be obtained by smoothing the first image in a similar manner.

[0117] When calculating the smoothed density values ​​of pixels within the range indicated by the kernel, all pixels within the range may be used, or density values ​​of pixels sampled at any interval may be smoothed. Sampling pixels can speed up the smoothing process.

[0118] In this embodiment, if the pixel sizes of the first image and the second image are equal, or if the difference in pixel size is equal to or less than a threshold, the first image and the second image may be output to the converted image acquisition unit 1020, and this step may be omitted. In this case, the same processing as in the first embodiment is performed.

[0119] (S12020) (Acquisition of converted image) In step S12020, the converted image acquisition unit 1020 acquires a first converted image obtained by converting the resolution of the first image (or the first smoothed image) and a second converted image obtained by converting the resolution of the second image (or the second smoothed image) so that the pixel sizes of the two images are the same, as in the first embodiment. For example, if the first image has a first pixel size that is the larger of two pixel sizes and the second image has a second pixel size that is the smaller of the two pixel sizes, the conversion unit 1020 performs resolution conversion so that the pixel size of the first image becomes the second pixel size. The generated converted image is then output to the deformation information acquisition unit 1030, the position acquisition unit 1050, the comparison area setting unit 1060, the combination determination unit 1070, and the difference calculation unit 1080. Note that the converted image acquisition unit 1020 does not necessarily have to perform resolution conversion.

[0120] In this way, the processing of the image processing device 1100 is carried out.

[0121] In this embodiment, between images with different pixel sizes, the image with the smaller pixel size is smoothed so that it approximates the pixel size of the larger image, and then the difference is calculated. Therefore, according to this embodiment, the user can observe a difference image in which noise caused by differences in pixel size is reduced more than in the first embodiment.

[0122] In the above embodiment, when the pixel size of the second image is smaller than that of the first image, the smoothing process is performed on the second image. However, the same effect can be obtained by performing the smoothing process on the second converted image. Similarly, when the pixel size of the first image is smaller than that of the second image, the smoothing process may be performed on the first converted image instead of the first image. In this case, the smoothing kernel size may be determined based on the pixel size of the original image, not the pixel size of the converted image.

[0123] Note that the second to fifth embodiments can also perform processing similar to that of this embodiment. That is, after acquiring image data, a smoothed image is generated by smoothing the image with the smaller pixel size of the first and second images to match the image with the larger pixel size, and the smoothed image is then replaced with the original image for subsequent processing, thereby achieving a similar noise reduction effect. Here, in the third embodiment, if the pixel size of the second image is smaller than that of the first image, a smoothed image may be generated by performing a smoothing process on the second transformed / converted image, and this smoothed image may be used to replace the second transformed / converted image. Furthermore, in each embodiment, the first and second images (or their transformed images) that have not been smoothed may be used in the transformation information acquisition process performed by the transformation information acquisition unit 1030, and the smoothed image (or their transformed image) may be used only in the difference value calculation process performed by the difference calculation unit 1080.

[0124] (Variation 6-1) (Acquiring deformation information from an external source) In this embodiment, deformation information is acquired using the image after smoothing processing. However, if a similar transformation processing has been performed previously and deformation information has been acquired, the deformation information acquired in the transformation processing may be stored in data server 110, and the previously acquired deformation information may be acquired from data server 110, thereby skipping the process of acquiring deformation information. The deformation information may then be used to generate a difference image between the first image or the first transformed image and the second smoothed image. For example, if a user who observes the difference image generated by the image processing device of the first embodiment determines that noise caused by differences in pixel size needs to be reduced, the image processing device of this embodiment can generate a difference image with reduced noise using the already acquired deformation information. This allows the process of acquiring deformation information to be skipped, thereby speeding up processing.

[0125] (Variation 6-2) (Smoothing method not based on pixel division) In this embodiment, the resolution of the second image is converted and smoothed so that the pixel size of the first image is an odd multiple of that of the second image. However, the resolution does not necessarily have to be converted. For example, a first region of the same pixel size as the first image may be set around the pixel to be smoothed in the second image, and smoothing may be performed based on the density values ​​and volume ratio of the pixels included in the first region. More specifically, if a pixel of the second image is completely contained within the first region, the weighting coefficient associated with the density value of that pixel is set to 1. On the other hand, if only half of the pixel is contained within the first region, the weighting coefficient associated with the density value is set to 0.5. A second smoothed image can then be obtained whose density value is the weighted average of the density values ​​of all pixels included in the first region. This allows for the same effect to be achieved without converting the resolution of the second image so that the pixel size of the first image is an odd multiple of that of the second image.

[0126] Seventh Embodiment The image processing device according to this embodiment, like the sixth embodiment, is a device that generates a three-dimensional difference image between a first image and a second image. However, the image processing device according to this embodiment automatically determines the relative pixel sizes between the images, and if the pixel sizes between the images are different, acquires the size of the comparison region based on the larger pixel size. As a result, compared to the sixth embodiment, it is characterized by generating a difference image with reduced noise from the smoothed image. The image processing device according to this embodiment will be described below.

[0127] 13 shows the configuration of an image diagnostic system according to this embodiment. The functions of a pixel size determination unit 13010 and a smoothed image acquisition unit 11010 will be explained below. The other components have the same functions as those in the sixth embodiment, and therefore their explanation will be omitted.

[0128] The pixel size determination unit 13010 automatically determines which pixel size is larger between the first image and the second image, and acquires the larger pixel size as the first pixel size. That is, it determines whether the pixel sizes of the first image and the second image are different. If the pixel size determination unit 13010 determines that the pixel size of the second image is smaller than the pixel size of the first image, the smoothed image acquisition unit 11010 acquires a second smoothed image obtained by smoothing the second image. On the other hand, if the pixel size of the first image is smaller than the pixel size of the second image, it acquires a first smoothed image obtained by smoothing the first image.

[0129] 14 shows a flowchart of the overall processing procedure performed by the image processing device 1300. Steps S14020, S14030, S14050, and S14070 to S14110 perform the same processing as steps S12020, S12030, S12050, and S12070 to S12110 in the sixth embodiment, respectively, and therefore explanations thereof will be omitted. Only differences from the flowchart in FIG. 12 will be explained below.

[0130] (S14000) (Data Acquisition) In step S14000, the data acquisition unit 1010 acquires a first image and a second image to be input to the image processing device 1300. The data acquisition unit 1010 also acquires information about the pixel sizes of the first image and the second image in a predetermined axis direction. That is, the data acquisition unit 1010 corresponds to an example of an image acquisition means for acquiring the first image and the second image. The data acquisition unit 1010 also corresponds to an example of a pixel size acquisition means for acquiring a first pixel size, which is the pixel size in a predetermined axis direction of each of the first image and the second image captured at different times, and a second pixel size, which is different from the first pixel size. The data acquisition unit 1010 then outputs the acquired first image and second image and information about the pixel sizes of the first image and the second image to the pixel size determination unit 13010. The information about the pixel sizes of the first image and the second image is also output to the comparison region size determination unit 1040. At this time, only information about the pixel sizes of the first image and the second image in the predetermined axis direction may be output to the comparison region size determination unit 1040. For example, the data acquisition unit 1010 outputs only the pixel size in the z-axis direction to the comparison region size determination unit 1040. Note that, although the present embodiment shows a case where the image processing device 1300 includes the functions 1010 to 1100, the present invention is not limited to this.

[0131] (S14002) (Determining the difference in pixel size) In step S14002, the pixel size determination unit 13010 automatically determines whether the pixel sizes of the first image and the second image are different and the relative size of the pixel sizes of the first image and the second image. In other words, the pixel size determination unit 13010 corresponds to an example of a determination unit that determines whether the first pixel size and the second pixel size acquired by the pixel size acquisition unit are different. The pixel size determination unit 13010 further determines whether the difference in pixel size between the first image and the second image is greater than a threshold. If the difference in pixel size is greater than the threshold, the process proceeds to step S14005. On the other hand, if the difference is equal to or less than the threshold, the first image and the second image are output to the converted image acquisition unit 1020, and the processes of steps S14005 and S14010 are omitted. In this case, the same process as in the first embodiment is performed.

[0132] For example, the determination is made based on whether or not the difference in at least one of the pixel sizes on the x, y, and z axes (a value indicating the difference between the first pixel size and the second pixel size) is greater than a threshold value. Here, the threshold value may be set as the ratio between the first pixel size and the second pixel size (for example, 1.5 times), or as the difference in size (for example, 3 mm). Also, any method of comparing two numerical values ​​may be used. Note that, without setting the threshold value, whether or not there is a difference in pixel size may be used as the determination condition.

[0133] In this embodiment, the determination is made as to whether the difference in pixel size between the first image and the second image is greater than a threshold value, but the determination may be made based on the volume of one voxel, or based only on the pixel size in a predetermined axis direction. For example, in the case of a general CT image, the pixel size in the slice plane (x-axis, y-axis directions) is sufficiently small and the difference is small, so the difference in pixel size may be determined based only on the pixel size in the z-axis direction (slice thickness).

[0134] (S14005) (Obtaining the first pixel size) In step S14005, the pixel size determination unit 13010 acquires the larger pixel size as the first pixel size and the smaller pixel size as the second pixel size of the pixel sizes of the first image and the second image acquired from the data acquisition unit 1010. Then, the acquired first pixel size is output to the smoothed image acquisition unit 11010 and the comparison region size determination unit 1040.

[0135] For example, the pixel size of the image with the larger volume per voxel between the first and second images is acquired as the first pixel size. Alternatively, the pixel size of the image with the larger pixel size in a predetermined axis direction (e.g., z-axis direction) may be acquired as the first pixel size. Note that in general CT images, the z-axis direction corresponds to the body axis direction. Alternatively, the larger pixel size may be selected for each of the x, y, and z axes, and the combination may be used as the first pixel size.

[0136] (S14010) (Acquisition of smoothed image) In step S14010, the smoothed image acquisition unit 11010 acquires a smoothed image by smoothing the first image or the second image based on the first pixel size acquired in step S14005. Specifically, if the first image has the first pixel size, the same processing as in step S12010 is performed. That is, the second image is smoothed using the first pixel size as the smoothing kernel size. Note that if the images with larger pixel sizes differ on the x, y, and z axes, the first and second images may be defined for each axis, and smoothing processing may be performed on the second image in the corresponding axis direction. On the other hand, if the first image has a second pixel size smaller than the first pixel size, the first image may be smoothed. That is, the first image is smoothed using the first pixel size as the smoothing kernel size.

[0137] (S14040) (Determining the size of the comparison area) In step S14040, the comparison region size determination unit 1040 determines the size of the comparison region. If the first pixel size was acquired in step S14005, the comparison region size determination unit 1040 determines the size of the comparison region based on the first pixel size. In other words, when the first pixel size is different from the second pixel size, the comparison region size determination means corresponds to an example of a determination means that, when the first pixel size is different from the second pixel size, determines the size in a predetermined axis direction of the comparison region, which is a region including multiple density values ​​in the other image of the first and second images that is to be compared with the density value of the position of interest in one of the first and second images, based on the larger of the first pixel size and the second pixel size. The comparison region size determination unit 1040 outputs the determined size of the comparison region to the comparison region setting unit 1060.

[0138] For example, the comparison region size determination unit 1040 determines only the size of the comparison region in a predetermined axis direction based on the first pixel size. In general CT images, the resolution within a slice (i.e., pixel size in the x and y directions) is often sufficient, so the comparison region in the x and y directions may not be determined based on the first pixel size. For example, in the x and y axis directions, density values ​​are compared only between the density values ​​of the position of interest in the first image and the density values ​​of the corresponding position in the second image. Alternatively, the size of the comparison region in the x and y axes may be set to a predetermined fixed value (e.g., 1 mm), and only the size of the comparison region in the z axis direction may be determined using the above method. This can speed up calculations. Similarly to the first embodiment, in the x and y axis directions, the comparison region size determination unit 1040 may determine the size of the comparison region as the sum of the half pixel sizes of the first and second images. In addition, the comparison area size determination unit 1040 may determine the comparison area to be 0 in the x-axis and y-axis directions so that only the difference between the density value of the target position in the first image and the density value of the corresponding position in the second image is calculated.

[0139] The reason why the size of the comparison region is obtained based on the first pixel size will be described below.

[0140] For example, assume that the predetermined axis direction is the z-axis, and there are a first image having a first pixel size and a second image having a second pixel size, each of which has a pixel size of D mm and d mm (D = n * d). Figure 15 shows an example of how the difference in coordinates of the endpoints of the graphs changes when the discretization position of the first image is shifted, focusing on pixels reflecting the same lesion in the first and second images, each of which has a different pixel size. For simplicity, D = 3 * d, and the lesion in the original image is described as a sphere with a diameter of d mm. However, the size of the first pixel size and the shape and size of the lesion are not limited to these. In Figure 15, the first and second images are, for example, images of a body, and the predetermined axis direction, the z-axis, indicates the body axis direction. In other words, the pixel size in the z-axis direction indicates the slice thickness. The processing in this embodiment is not limited to processing in the z-axis direction, but can also be applied to pixels in the x and y directions.

[0141] (1) When the bottom edges of the discretization positions of the two pixels are aligned 15(a) is a diagram schematically illustrating an example in which the lower ends of the discretization positions of two pixels are aligned, where the vertical axis represents the position in the z-axis direction.

[0142] First, a first image and a second image are obtained by capturing (discretizing) the original image. This step corresponds to step S14000 in this embodiment. Graphs 15010 and 15020 respectively show the pixel values ​​of pixels on the second image, which is an image of the lesion 15005 in the original image that has been discretized, and the pixel values ​​of pixels on the first image. That is, they show the density values ​​of the first pixel in the first image and the density values ​​of the second pixel in the second image, which show a predetermined region of the subject. Graph 15010 shows the pixel in the second image that best reflects the lesion 15005 in the original image. Graph 15020 shows the pixel in the first image that best reflects the lesion 15005 in the original image. Reference symbol 15005 is not limited to a lesion, but may also represent a structure such as a bone.

[0143] Next, the smoothed image acquisition unit 11010 performs a smoothing process corresponding to step S14010 of this embodiment. Graph 15030 shows values ​​obtained by smoothing the density values ​​of graph 15010 based on the pixel size of the first image. Specifically, it shows values ​​obtained by smoothing the density values ​​of the second image using the pixel size of the first image as the kernel size. In other words, the smoothed image acquisition unit 11010 corresponds to an example of a smoothing means that smooths an image of the smaller pixel size of the first and second pixel sizes using a kernel size based on the larger pixel size.

[0144] Next, the converted image acquisition unit 1020 performs density value interpolation corresponding to step S14070 of this embodiment (step S12020 in the sixth embodiment). Graphs 15040 and 15050 show values ​​obtained by linearly interpolating the density values ​​of graphs 15030 and 15020 with adjacent pixels, respectively. That is, graph 15050 corresponds to an example of a first interpolated value obtained by interpolating the density value of a first pixel in a first image and the density value of a pixel adjacent to the first pixel. Graph 15040 corresponds to an example of a second interpolated value obtained by interpolating the density value of a second pixel after smoothing the density value of a second pixel in a second image using a first pixel size and the density value of a pixel adjacent to the pixel exhibiting the density value of the second pixel after smoothing. The height along the horizontal axis in graphs 15040 and 15050 indicates the density value. A known image processing method can be used to interpolate the density values. For example, you can use nearest neighbor interpolation, linear interpolation, cubic interpolation, etc. In this case, the coordinates of the endpoints in the positive z-axis direction in each graph are (5 / 2)*d for graph 15040 and (3 / 2)*D for graph 15050.

[0145] In this case, the difference 15060 between the coordinates of the endpoints of each of the graphs described above is {(3 / 2)*D-(5 / 2)*d}=2*d.

[0146] (2) In the first image, the lesion 15005 in the original image is discretized at the center. FIG. 15(b) is a diagram schematically showing an example in which the lesion of the original image is discretized at the center in the first image. In FIG. 15(b), the vertical axis represents the position in the z-axis direction.

[0147] In this case, since the object is discretized at the center, the graph 15020, which is the pixel in the first image that most reflects the lesion 15005 of the original image, consists of two consecutive pixels.

[0148] Next, when linearly interpolating these two pixels 15020 in the same manner as in FIG. 15(a), the coordinate of the end point in the positive z-axis direction in the graph 15050 is {(1 / 2)*d+(3 / 2)*D}=(5 / 3)*D. At this time, the difference 15060 between the coordinates of the end points of each graph is {(5 / 3)*D-(5 / 2)*d}=2.5*d.

[0149] (3) When the lesion 15005 of the original image is discretized at the end in the first image FIG. 15(c) is a diagram schematically showing an example in which the lesion of the original image is discretized at the end (upper end) in the first image. In FIG. 15(c), the vertical axis represents the position in the z-axis direction. Note that in FIG. 15(c), it is assumed that the discretization occurs at an end d’ (<d) that is infinitely close to the contact point (z = d) of the lesion 15005 of the original image.

[0150] In this case, since the object is discretized at the end, the graph 15020, which is the pixel that contains more of the lesion within the discretization interval among the two pixels reflecting the lesion 15005 of the original image, becomes the pixel that most reflects the lesion ********** of the original image.

[0151] It should be noted that there seems to be an incomplete expression in the original text at the end of ID=18, where "lesion 15005 of the original image" is repeated twice in the English translation for the sake of consistency with the original structure. If this is an error in the original, it may need to be corrected in the source material.Next, two pixels reflecting the lesion 15005 in the original image are linearly interpolated. At this time, the coordinate of the endpoint in the positive z-axis direction of graph 15050 is {d'+(3 / 2)*D} (<{d+(3 / 2)*D}=(11 / 6)*D). Specifically, the value decreases by the amount of the discretization position shift between the first and second images. However, since FIG. 15(c) assumes that the discretization position shift is infinitesimal, this value can be approximated to 0. In this case, the difference 15060 between the coordinates of the endpoints of each graph is {(11 / 6)*D-(5 / 2)*d}≒3*d. This value matches the first pixel size.

[0152] (4) When the top edges of the discretization positions of the two pixels are aligned Fig. 15(d) is a diagram schematically illustrating an example in which the upper ends of the discretization positions of two pixels are aligned, where the vertical axis represents the position in the z-axis direction.

[0153] Continuing the process in the same manner as in FIG. 15(a), the coordinates of the endpoints of graphs 15040 and 15050 are compared, and the difference 15060 is |[{d-(3 / 2)*D}-{-(3 / 2)*d}]|=|-(2*d)|=2*d.

[0154] 15(a) to 15(d) show that the difference in coordinates of the endpoints when the pixel that best reflects the lesion 15005 in the original image is linearly interpolated with adjacent pixels varies between 2*d and 3*d. Therefore, by adaptively determining the size of the comparison region to 3*d=D, i.e., the first pixel size, it is possible to search for a density value that reflects the lesion 15005 in the original image between two images with the minimum required size. From a different perspective, by setting a comparison region in one of the first and second images based on the amount of deviation between the corresponding positions of non-maximum values ​​in the first interpolated value (graph 15050) and non-maximum values ​​in the second interpolated value (graph 15040), it becomes possible to search for corresponding values ​​other than the maximum values ​​of the interpolated values. Note that the amount of deviation between the minimum value of the first interpolated value and the corresponding minimum value of the second interpolated value may be maximized, and the size of the comparison region may be determined based on this deviation. Alternatively, the size of the comparison region may be determined based on the amount of deviation between the non-maximum and non-minimum values ​​of the first interpolated value and the corresponding non-maximum and non-minimum values ​​of the second interpolated value.

[0155] The comparison region size determination unit 1040 may determine the size of the comparison region by multiplying the first pixel size (the larger pixel size) by a predetermined coefficient. The predetermined coefficient may be 0.5 or another value. In this embodiment, the predetermined coefficient is set to 0.5 because the degree of difference between the first interpolated value and the second interpolated value is random, and the various cases described above appear randomly.

[0156] As can be seen from the processing of steps S14002, 14005, and 14010, when the value indicating the difference between the first pixel size and the second pixel size is greater than or equal to a threshold value, the comparison area size determination unit 1040 determines the size of the comparison area in the specified axis direction based on the larger pixel size.

[0157] On the other hand, if the first pixel size has not been acquired in step S14005, the comparison region size determination unit 1040 determines the size of the comparison region based on, for example, the pixel sizes of the first and second images, as in the sixth embodiment. Specifically, the comparison region size determination unit 1040 determines the size of the comparison region to be the sum of half the pixel size of the first image and half the pixel size of the second image. In other words, if the value indicating the difference between the first pixel size and the second pixel size is less than the threshold, the comparison region size determination unit 1040 determines the size of the comparison region in the predetermined axis direction based on the first pixel size and the second pixel size.

[0158] If the first pixel size is not acquired in step S14005, the comparison region size determination unit 1040 may determine the comparison region to be 0 so that only the difference between the density value of the position of interest in the first image and the density value of the corresponding position in the second image is calculated. The determined size of the comparison region is then output to the comparison region setting unit 1060. As in the above embodiment, the comparison region size determination unit 1040 may determine the size of the comparison region by multiplying the first pixel size (the larger pixel size) by a predetermined coefficient. The predetermined coefficient may be 0.5 or another value. In this embodiment, the predetermined coefficient is set to 0.5 because the degree of difference between the first interpolated value and the second interpolated value is random, and various cases such as those described above will appear randomly. Furthermore, if it is determined in S14002 that the difference between the two pixel sizes is greater than the threshold, step S14040 is not limited to this processing procedure, and may be performed before any of S14030, S14020, and S14010.

[0159] (S14060) (Setting the comparison area) In step S14060, the comparison region setting unit 1060 sets a comparison region having the size determined in step 14040 based on the corresponding position on the second image. For example, the comparison region is set based on the corresponding position on the smoothed image. Note that the region in which the comparison region is set is not limited to the second image. For example, if a focus position is set on the second image, the comparison region may be set on the first image based on the corresponding position on the first image that corresponds to the focus position. The comparison region setting unit 1060 corresponds to an example of a setting unit that sets a comparison region on the other image based on the corresponding position on the other image. For example, the comparison region setting unit 1060 sets a comparison region around the corresponding position on the second image. That is, the comparison region setting unit 1060 sets a comparison region of the size determined by the setting unit, centered on the corresponding position on the other image. At this time, the comparison region may be set so as not to exceed a position that is the first pixel size away from the corresponding position in at least one of the three-dimensional axis directions of x, y, and z. That is, in an n-dimensional image, the comparison area setting unit 1060 sets the comparison area in at least one of the n-dimensional axis directions so that the comparison area does not exceed a position that is the larger pixel size away from the corresponding position on the other image.

[0160] For example, consider a case where a rectangular comparison region is set around a corresponding position on the second image. First, assume that the coordinates of the corresponding position are (x, y, z) = (1, 1, 1) and the size of the comparison region is D = 4. In this case, if the center of the comparison region is set to (x, y, z) = (1, 1, 2), the coordinates of the endpoints of the rectangular comparison region are (x, y, z) = (1 + 4, 1 + 4, 2 + 4) = (5, 5, 6). This exceeds the position (z = 5) that is the first pixel size away from the corresponding position in the z direction, but does not exceed the position (x = 5, y = 5) that is the first pixel size away from the corresponding position in the x and y directions. Therefore, this corresponds to a comparison region that does not exceed the position that is the first pixel size away from the corresponding position in at least one of the above-mentioned axial directions. Alternatively, the comparison region may be set so that it does not exceed the position that is the first pixel size away from the corresponding position in all three-dimensional axial directions. Information about the set comparison region is then output to difference calculation unit 1080.

[0161] In this embodiment, when smoothing processing of the second image is performed using a kernel of the same size as the first pixel size, the smoothed image obtained by smoothing the second image can be regarded as an image discretized in an area of ​​the same size as the first pixel size, just like the first image. In other words, since the discretization position is shifted between the smoothed image and the first image by up to half the size of the first pixel, noise caused by the shift in the discretization position between the smoothed image and the first image can be reduced by setting the size of the comparison area to the large pixel size (= half the size of the first pixel size + half the size of the first pixel size).

[0162] The size of the comparison region in the x and y directions, which are different from the predetermined axis direction, can be set to a predetermined value. Specifically, for example, the comparison region size determination unit 1040 may determine the size of the comparison region in the x and y directions as the sum of half the pixel size of the first image and half the pixel size of the second image in the corresponding directions, or may set a predetermined constant. Furthermore, the size of the comparison region in the x and y directions, which are different from the predetermined axis direction, may be set to 0 so that only the density value of the target position and the density value of the corresponding position are compared.

[0163] (S14070) (Determining the combination) In step S14070, the combination determination unit 1070 determines a combination of density values ​​to be used for comparison (difference calculation). First, the combination determination unit 1070 interpolates the density value of the target position on the first converted image acquired in step S14050 and the density values ​​of pixels in the comparison area on the second converted image. Note that known image processing techniques can be used to interpolate the density values. For example, nearest neighbor interpolation, linear interpolation, cubic interpolation, etc. can be used. The density values ​​of pixels in the comparison area on the second converted image do not necessarily need to be interpolated. The combination of density values ​​may be determined so that the difference between the density value of the target position on the first converted image and the density values ​​of all pixels in the comparison area on the second converted image is calculated. Alternatively, at least one pixel may be sampled from the pixels included in the comparison area, and the combination of the density value of that pixel and the density value of the target position may be used as the combination to calculate the difference. For example, the combination determination unit 1070 samples the maximum and minimum density values ​​from the pixels included in the comparison area. Then, the combination of the maximum density value included in the comparison region and the density value of the position of interest, and the combination of the minimum density value included in the comparison region and the density value of the position of interest are determined as combinations for which the difference is to be calculated. The density values ​​to be sampled are not limited to the maximum and minimum, and three or more values ​​may be sampled, or a single value such as the maximum or minimum density value may be sampled. Alternatively, a density range with the maximum and minimum density values ​​in the comparison region as its two ends (upper and lower limits) may be obtained, and the density value of the position of interest and the density range in the comparison region may be obtained as the combination for which the difference is to be calculated. The density range in the comparison region may be other than the maximum and minimum density values. For example, it may be the maximum and minimum values ​​after removing outliers from the density values.

[0164] The combination determination unit 1070 outputs information indicating the determined combination of density values ​​to be used for calculating the difference to the difference calculation unit 1080. The difference calculation unit 1080 calculates the difference between the density value of the position of interest on one image and the density values ​​of multiple positions in the comparison area on the other image, for example, as described above. The information indicating the combination of density values ​​output by the combination determination unit 1070 may be information indicating all combinations of the density values ​​of the position of interest on the first converted image and the density values ​​of pixels in the comparison area on the second converted image, or may output only information indicating combinations of the density values ​​of pixels sampled from the pixels included in the comparison area and the density values ​​of the position of interest.

[0165] In this way, the processing of the image processing device 1300 is carried out.

[0166] In this embodiment, the pixel size relationship between images with different pixel sizes is automatically determined, and a smoothing process is performed to approximate the image with the smaller pixel size to the image with the larger pixel size, thereby obtaining a smoothed image. This allows both the smoothed image and the first image to be considered as having been discretized into regions of the same size as the first pixel size. By using a value based on the size of the discretization region (first pixel size) as the size of the comparison region, the user can observe a difference image in which noise resulting from differences in pixel size is reduced more than in the sixth embodiment. In other words, the size of the comparison region that can further reduce noise can be adaptively determined. Furthermore, according to this embodiment, the size of the comparison region is determined based on the first pixel size, which is larger than the sum of half the pixel sizes of the two images, so that even relatively weak signals representing a predetermined object are less likely to be visualized as noise.

[0167] Eighth Embodiment In the above-described embodiment, a single image processing device acquires two images taken at different times and performs a series of processes up to generating a difference image. However, the image processing described herein can also be performed when the comparison region size determination unit 1040 is provided in a separate image processing device. In this case, the components provided in each of the two image processing devices are shown in FIG. 16. The functions of the image acquisition unit 16010, pixel size determination unit 16020, pixel size determination unit 16030, and pixel size determination unit 16030' will be described below. Description of the other components will be omitted.

[0168] The image acquisition unit 16010 acquires the first and second images input from the data server 110 to the image processing device 1610 .

[0169] The pixel size acquisition unit 16020 acquires the first pixel size and the second pixel size input to the image processing device 1620 from the data server 110. At this time, the first image and the second image themselves do not need to be acquired.

[0170] The pixel size determination unit 16030 determines which of the pixel sizes of the first image and the second image input from the image acquisition unit 16010 is larger, and sets the larger pixel size as the first pixel size.

[0171] The pixel size determination unit 16030' determines which of the two pixel sizes input from the pixel size acquisition unit 16020 is larger and sets the larger pixel size as the first pixel size. In this case, between the two devices, only the pixel size determination unit 16030 may determine which of the pixel sizes is larger and send the determination result to the separate image processing device 1620, or only the pixel size determination unit 16030' may determine which of the pixel sizes is larger and send the determination result to the separate image processing device 1610.

[0172] FIG. 17 is a flowchart showing the overall processing procedure performed by the image processing device 1620.

[0173] Each process will be explained below.

[0174] (S17000) (Data Acquisition) In step S17000, the pixel size acquisition unit 16020 outputs information relating to the pixel sizes of the first image and the second image input to the image processing device 1620 to the pixel size determination unit 16030'.

[0175] (S17100) (Determine pixel size) In step S17100, the pixel size determination unit 16030′ acquires as the first pixel size the larger pixel size of the first image and the pixel size of the second image acquired from the pixel size acquisition unit 16020, and acquires as the second pixel size the smaller pixel size. Then, the acquired first pixel size is output to the comparison region size determination unit 16040.

[0176] For example, the pixel size of the image with the larger volume per voxel between the first and second images is acquired as the first pixel size. Alternatively, the pixel size of the image with the larger pixel size in a predetermined axis direction (e.g., z-axis direction) may be acquired as the first pixel size. Note that in general CT images, the z-axis direction corresponds to the body axis direction. Alternatively, the larger pixel size may be selected for each of the x, y, and z axes, and the combination may be used as the first pixel size.

[0177] (S17200) (Determining the size of the comparison area) In step S17200, the comparison region size determination unit 16040 determines the size of the comparison region. If the first pixel size has been acquired in step S17200, the comparison region size determination unit 16040 determines the size of the comparison region based on the first pixel size.

[0178] Then, it is output to the comparison region setting unit 1060 of the image processing device 1610.

[0179] Ninth Embodiment Although the above-described embodiment has described the case where a difference image is generated by the difference calculation unit 1080 and the difference image generation unit 1090, the image generated by the image processing device according to the present invention is not limited to a difference image, as long as it is an image showing the difference between two images captured at different times. For example, the image processing device may calculate the ratio of density values ​​between two images and generate an image showing the density value ratio (division result) between the two images based on that ratio. In this case, the ratio is 1 for areas without change and a value other than 1 for areas with change, so that changes over time can be visualized as in the above-described difference image. Note that, when generating an image showing the density value ratio, a ratio calculation unit that calculates the ratio and an image generation unit that generates an image showing the ratio may be provided instead of the difference calculation unit 1080 and the difference image generation unit 1090. Here, since calculating the difference between multiple density values ​​and calculating the ratio between multiple density values ​​can both be considered to be comparing multiple density values, the difference calculation unit 1080 and the ratio calculation unit may be collectively referred to as a comparison unit. That is, the difference calculation unit 1080 or the ratio calculation unit corresponds to an example of a comparison means that compares the density value of a position of interest on one image with the density values ​​of multiple positions in a comparison region on the other image.

[0180] <Other embodiments> Although the exemplary embodiments have been described above in detail, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, a recording medium (storage medium), etc. Specifically, the present invention may be applied to a system made up of multiple devices (for example, a host computer, an interface device, an imaging device, a Web application, etc.), or may be applied to an apparatus made up of a single device.

[0181] Needless to say, the object of the present invention can be achieved by the following: A recording medium (or storage medium) on which software program code (computer program) that realizes the functions of the above-described embodiments is recorded is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. A computer (or a CPU or MPU) of the system or device reads and executes the program code stored on the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiments, and the recording medium on which the program code is recorded constitutes the present invention. Note that each function of the image processing device in the above-described embodiments is realized by at least one processor included in the image processing device executing a program stored in at least one memory. The type of processor is not important, and multiple types of processors may be used.

[0182] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims.

[0183] Any of the above-described embodiments may be combined. [Explanation of symbols]

[0184] 1010 Data acquisition unit 1040 Comparison area size determination unit 13010 Pixel size determination unit

Claims

1. 1. An image processing device that performs image processing to reduce noise due to a difference in pixel size contained in a difference image generated based on two images captured with different pixel sizes of a common subject, an image acquisition means for acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition means for acquiring pixel sizes in a predetermined axis direction in the first image and the second image, respectively; a smoothing means for smoothing one of the first image and the second image, which has a smaller pixel size, by using a kernel size based on the pixel size of the other of the first image and the second image to generate a smoothed image; a determining means for determining a size of a comparison region in the predetermined axis direction based on a pixel size of the other image, the size being determined for a comparison region including a plurality of positions on the smoothed image, the density values ​​of which are compared with the position of interest on the other image; a position acquisition means for acquiring the position of interest on the other image and a corresponding position on the smoothed image that corresponds to the position of interest; a setting means for setting a position of the comparison region based on the corresponding position; a difference acquisition means for acquiring a difference between a density value at the target position and an interpolated value obtained by interpolating density values ​​at a plurality of positions on the smoothed image by at least one of nearest neighbor interpolation, linear interpolation, and cubic interpolation; a difference image generating means for generating the difference image based on the difference value; 13. An image processing device comprising:

2. 1. An image processing device that performs image processing to reduce noise caused by a discretization position shift between two images captured with different pixel sizes of a common subject, an image acquisition means for acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition means for acquiring pixel sizes in a predetermined axis direction in the first image and the second image, respectively; a smoothing means for smoothing one of the first image and the second image, which has a smaller pixel size, by using a kernel size based on the pixel size of the other of the first image and the second image to generate a smoothed image; a determining means for determining a size of a comparison region in the predetermined axis direction based on a pixel size of the other image, the size being determined for a comparison region including a plurality of positions on the smoothed image, the density values ​​of which are compared with the position of interest on the other image; a position acquisition means for acquiring the position of interest on the other image and a corresponding position on the smoothed image that corresponds to the position of interest; a difference acquiring means for acquiring a difference between a density value of the target position and an interpolated value obtained by interpolating density values ​​of at least a plurality of positions on the smoothed image; a setting means for setting a position of the comparison region based on the corresponding position; 13. An image processing device comprising:

3. The image processing device according to claim 1 , wherein the first image and the second image include a three-dimensional image.

4. 4. The image processing apparatus according to claim 1, wherein the determining means determines the size of the comparison region so as not to be based on a pixel size of the one image.

5. The image processing apparatus according to claim 2 , further comprising a difference image generating means for generating a difference image based on the difference value.

6. 6. The image processing apparatus according to claim 1, wherein the first image and the second image are images of a body, and the predetermined axis direction is a body axis direction.

7. 7. The image processing apparatus according to claim 1, wherein the determining means determines the size of the comparison area by multiplying a pixel size of the other image by a predetermined coefficient.

8. 8. The image processing apparatus according to claim 7, wherein the predetermined coefficient is 0.

5.

9. the determining means determines a size of the comparison region in the predetermined axis direction based on the pixel size of the other image when a ratio of a pixel size of the other image to a pixel size of the one image is greater than a threshold value; 9. The image processing device according to claim 1, wherein, when the ratio is equal to or smaller than a threshold value, the size of the comparison region in the specific axis direction is determined based on the pixel size of the other image and the pixel size of the one image.

10. 10. The image processing apparatus according to claim 9, wherein the determining means determines the size of the comparison region to be a predetermined value in an axial direction different from the predetermined axial direction.

11. 10. The image processing apparatus according to claim 9, wherein the determining unit determines to compare the density value of the target position with the density value of the corresponding position in an axial direction different from the predetermined axial direction.

12. each of the first image and the second image is an n-dimensional image, The image processing device according to any one of claims 1 to 11, characterized in that the setting means sets the comparison area so that, in at least one of the n-dimensional axial directions, the comparison area does not exceed a position away from the corresponding position on the other image by a pixel size of the other image.

13. 13. The image processing apparatus according to claim 1, wherein the setting means sets the comparison area having the size and centered at the corresponding position on the other image.

14. 10. The image processing device according to claim 9, wherein the threshold value is 1.

5.

15. 15. An image diagnostic system comprising: the image processing device according to claim 1; and an imaging device that captures the first image and the second image.

16. The imaging diagnostic system of claim 15, further comprising a data server that stores the first image and the second image.

17. An image diagnostic system having an image processing device as described in claim 1 or 5 and a monitor that displays the difference image.

18. 1. An image processing method for performing image processing to reduce noise due to a difference in pixel size contained in a difference image generated based on two images captured with different pixel sizes of a common subject, the method comprising: an image acquiring step of acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition step of acquiring pixel sizes in a predetermined axis direction in the first image and the second image, respectively; a smoothing step of smoothing one of the first image and the second image, which has a smaller pixel size, using a kernel size based on the pixel size of the other of the first image and the second image to generate a smoothed image; a determining step of determining a size of a comparison region in the predetermined axis direction based on a pixel size of the other image, the size being determined for a comparison region including a plurality of positions on the smoothed image, the density values ​​of which are compared with the position of interest on the other image; a position acquiring step of acquiring the position of interest on the other image and a corresponding position on the smoothed image that corresponds to the position of interest; a setting step of setting a position of the comparison region based on the corresponding position; a difference acquisition step of acquiring a difference value between the density value of the target position and an interpolated value obtained by interpolating density values ​​of a plurality of positions on the smoothed image by at least one of nearest neighbor interpolation, linear interpolation, and cubic interpolation; a difference image generating step of generating the difference image based on the difference value; 13. An image processing method comprising:

19. 1. An image processing method for performing image processing to reduce noise caused by a discretization position shift between two images captured with different pixel sizes of a common subject, comprising: an image acquiring step of acquiring a first image captured at a predetermined time and a second image captured at a time different from the predetermined time; a pixel size acquisition step of acquiring pixel sizes in a predetermined axis direction in the first image and the second image, respectively; a smoothing step of smoothing one of the first image and the second image, which has a smaller pixel size, using a kernel size based on the pixel size of the other of the first image and the second image to generate a smoothed image; a determining step of determining a size of a comparison region in the predetermined axis direction based on a pixel size of the other image, the size being determined for a comparison region including a plurality of positions on the smoothed image, the density values ​​of which are compared with the position of interest on the other image; a position acquiring step of acquiring the position of interest on the other image and a corresponding position on the smoothed image that corresponds to the position of interest; a setting step of setting a position of the comparison region based on the corresponding position; a difference acquisition step of acquiring a difference value between the density value of the target position and an interpolated value obtained by interpolating density values ​​of a plurality of positions on the smoothed image by at least one of nearest neighbor interpolation, linear interpolation, and cubic interpolation; 13. An image processing method comprising:

20. 20. The image processing method according to claim 19, further comprising a difference image generating step of generating a difference image based on the difference value.

21. a comparison step of acquiring a difference value between a first interpolated value obtained by interpolating the density value of the target position and a second interpolated value obtained by interpolating density values ​​of a plurality of positions within the comparison region; 21. The image processing method according to any one of claims 18 to 20, further comprising:

22. A program for causing a computer to execute each step of the image processing method according to any one of claims 19 to 21.

Images