Image processing system, image processing method, and program

Abstract

To suppress unnecessary information included in a difference image.SOLUTION: An image processing system according to an embodiment comprises: image acquisition means which acquires a first image and a second image of a subject captured at different times; identification information acquisition means which acquires identification information of portions at a plurality of coordinates of the first image; and difference image generation means which when a target pixel is identified as a first portion with the identification information, generates a difference image between the first image and the second image so as to regard as a pixel value of the target pixel a corrected difference value obtained by subjecting an initial difference value expressing a difference of pixel values between a first coordinate and a second coordinate to predetermined correction processing according to a result of determining whether or not the pixel values of the first coordinate of the first image corresponding to the target pixel and the second coordinate of the second image corresponding to the first coordinate satisfy a predetermined condition.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing device, an image processing method, and a program. [Background technology]

[0002] By collecting medical images from multiple times, it is possible to evaluate changes over time in lesions, etc. Furthermore, by generating differential images that visualize changes over time in lesions, etc. based on medical images from multiple times, it is possible to more efficiently support diagnosis. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-192691 [Patent Document 2] Japanese Patent Application Publication No. 2023-128704 [Patent Document 3] Japanese Patent Application Publication No. 2018-38815 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to suppress unnecessary information contained in a difference image. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0005] The image processing device according to the embodiment includes an image acquisition means for acquiring a first image and a second image of a subject captured at different times; an identification information acquisition means for acquiring identification information of a region at multiple coordinates in the first image; and a difference image generation means for generating a difference image between the first image and the second image when a pixel of interest is identified as a first region based on the identification information, such that a predetermined correction process is applied to an initial difference value representing the difference in pixel values ​​between the first coordinate and the second coordinate in the second image corresponding to the pixel of interest according to a determination result as to whether or not the pixel values ​​of the first coordinate in the first image corresponding to the pixel of interest and the second coordinate in the second image corresponding to the first coordinate satisfy a predetermined condition, and the pixel value of the pixel of interest is set to a corrected difference value.

[0006] An image processing method according to an embodiment includes acquiring a first image and a second image of a subject captured at different times, acquiring identification information of regions at multiple coordinates in the first image, and, when a pixel of interest is identified as a first region based on the identification information, generating a difference image between the first image and the second image such that the pixel value of the pixel of interest is a corrected difference value obtained by applying a predetermined correction process to an initial difference value representing the difference in pixel values ​​between the first coordinate and the second coordinate, in accordance with a determination result as to whether or not a pixel value between a first coordinate in the first image corresponding to the pixel of interest and a second coordinate in the second image corresponding to the first coordinate satisfies a predetermined condition.

[0007] An image processing device according to an embodiment includes an image acquisition unit that acquires a first image and a second image captured at different times of the same subject; an image adjustment unit that adjusts at least one of the first image and the second image so as to correspond to the other; a difference image generation unit that calculates a difference value between the other image and the adjusted one of the images captured later in time and the earlier image in time to generate a difference image; an identification information acquisition unit that extracts an anatomical site of at least one of the later image and the difference image and acquires identification information of the site; and a difference image correction unit that corrects the difference image based on a correction method corresponding to the identification information, wherein the difference image correction unit performs correction according to whether the identification information of a pixel of interest in the difference image indicates a first site and whether the pixel value of the difference image is smaller than a predetermined threshold.

[0008] The image processing method according to the embodiment includes an image acquisition step of acquiring a first image and a second image of the same subject taken at different times; an image adjustment step of adjusting at least one of the first image and the second image so as to correspond to the other; a difference image generation step of calculating a difference value between the other image and the adjusted one of the images taken later in time and the earlier in time to generate a difference image; an identification information acquisition step of extracting an anatomical site of at least one of the later in time image and the difference image and acquiring identification information of the site; and a difference image correction step of correcting the difference image based on a correction method corresponding to the identification information, wherein the difference image correction step performs correction depending on whether the identification information of a pixel of interest in the difference image indicates a first site and whether the pixel value of the difference image is smaller than a predetermined threshold. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the configuration of an image processing system according to the first embodiment. [Figure 2] FIG. 2 is a flowchart showing the overall processing procedure in the first embodiment. [Figure 3A] FIG. 3A is a diagram showing an example of a two-dimensional cross-sectional image in the first embodiment. [Figure 3B] FIG. 3B is a diagram showing identification information of a region in the first medical image. [Figure 4] FIG. 4 is a flowchart showing the processing procedure of the difference image generating unit 104. [Figure 5] FIG. 5 is a diagram showing the configuration of an image processing system according to the second embodiment. [Figure 6] FIG. 6 is a flowchart showing the overall processing procedure in the second embodiment. [Figure 7] FIG. 7 is a diagram showing the configuration of an image processing system according to the third embodiment. [Figure 8] FIG. 8 is a flowchart showing the overall processing procedure in the third embodiment. [Figure 9A] FIG. 9A is a diagram illustrating conditional branches and correction processing operations when a differential image correction unit according to the third embodiment performs correction processing. [Figure 9B] FIG. 9B is a diagram illustrating an example of the third embodiment in which correction processing including attenuation processing is performed based on a predetermined threshold value of 0. [Figure 9C] FIG. 9C is a diagram illustrating an example of the third embodiment in which correction processing including attenuation processing is performed based on a negative predetermined threshold value. [Figure 9D] FIG. 9D is a diagram illustrating an example of the third embodiment in which correction processing including attenuation processing is performed based on a positive predetermined threshold value. [Figure 10] FIG. 10 is a diagram showing the configuration of an image processing system according to the fourth embodiment. [Figure 11A] FIG. 11A is a diagram illustrating conditional branches and correction processing operations when a differential image correction unit according to the fourth embodiment performs correction processing. [Figure 11B] FIG. 11B is a diagram illustrating a correction processing operation of the fourth embodiment, which performs differential image correction processing to change the display mode. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] First Embodiment The image processing device according to this embodiment is a device that generates a difference image between two images (a first medical image and a second medical image) captured at different times. For example, the image processing device performs deformation alignment of the two images and generates a difference image using a calculation method according to the region.

[0012] 1 is a diagram showing the configuration of an image processing system 10 according to the first embodiment. The image processing system 10 includes an image processing device 100, a data server 130, and a display unit 150.

[0013] The image processing device 100 is a device that generates a differential image between two images captured at different times, etc. The image processing device 100 is provided with a communication interface 10IF (not shown) for connecting to a data server 130 via a communication network 120.

[0014] The data server 130 stores a plurality of medical images. The data server 130 represents, for example, a PACS (Picture Archiving and Communication Systems) that receives medical image data captured by a modality and stores and manages the data via a network. In the following description, it is assumed that the data server 130 stores a plurality of three-dimensional tomographic images obtained by capturing images of a subject in advance at different times as a first medical image and a second medical image. In this embodiment, the first medical image and the second medical image are described as three-dimensional tomographic images (three-dimensional medical images) captured by an X-ray CT (Computed Tomography) device, as examples of medical images.

[0015] The modality for capturing the three-dimensional tomographic image may be an MRI (Magnetic Resonance Imaging) device, a three-dimensional ultrasound imaging device, a photoacoustic tomography device, a PET (Positron Emission Tomography) / SPECT (Single Photon Emission Computed Tomography), an OCT (Optical Coherence Tomography) device, or the like. Furthermore, the first medical image and the second medical image may be any three-dimensional tomographic images from which a difference image is to be generated. For example, the first medical image and the second medical image may be images captured on different dates and times of the same patient for follow-up observation. Alternatively, the first medical image and the second medical image may be images captured in a single examination but with different contrast phases.

[0016] For example, the first medical image and the second medical image are three-dimensional medical images (three-dimensional tomographic images) configured as a collection of two-dimensional tomographic images. The position and orientation of each two-dimensional tomographic image are converted into a reference coordinate system (a coordinate system in space based on the subject) and then stored in the data server 130. At this time, the first medical image and the second medical image expressed in the reference coordinate system are input to the image processing device 100 in accordance with instructions from a user operating the instruction unit 140. The instruction unit 140 includes various input devices that accept various commands from the user. For example, the instruction unit 140 includes various devices such as a mouse, keyboard, trackball, and touch panel, allowing the user to input various information and processing requests.

[0017] The image processing device 100 is a device that performs image processing upon receiving a processing request from the instruction unit 140. Specifically, based on an instruction from a user via the instruction unit 140, the image processing device 100 acquires a first medical image and a second medical image to be subjected to image processing from the data server 130 as a pair of images (image pair) to be subjected to image processing. The image processing device 100 then generates a difference image between the acquired first medical image and second medical image, and saves or displays the generated difference image on the display unit 150.

[0018] The image processing device 100 is composed of the components described below. The following components are realized by a processing circuit provided in the image processing device 100 executing code contained in a program. For example, the functions of the following components are realized by one or more central processing units (CPUs) functioning as control units of the image processing device 100 executing programs. The components of the image processing device 100 may be implemented on an integrated circuit as long as they perform similar functions.

[0019] For example, the image processing device 100 includes a storage unit (not shown), and processing functions corresponding to the image acquisition unit 101, the identification information acquisition unit 102, the alignment information acquisition unit 103, the difference image generation unit 104, and the display control unit 105 are stored in the storage unit in the form of computer-executable programs. The processing circuit included in the image processing device 100 is a processor that reads out the programs from the storage unit and executes them to realize the functions corresponding to the programs. In other words, the processing circuit in a state in which a program has been read out has the functions corresponding to the read program. The image acquisition unit 101, the identification information acquisition unit 102, the alignment information acquisition unit 103, the difference image generation unit 104, and the display control unit 105 may be realized by being distributed or integrated into a single or multiple processing circuits as appropriate.

[0020] The image acquisition unit 101 acquires information on the first medical image and the second medical image input to the image processing device 100. The identification information acquisition unit 102 acquires identification information of the part depicted in the input medical image. The alignment information acquisition unit 103 acquires alignment information representing the correspondence of coordinates between the first medical image and the second medical image. The difference image generation unit 104 generates a difference image between the first medical image and the second medical image based on the alignment information. The display control unit 105 performs display control to display the generated difference image on the display unit 150. The image acquisition unit 101 is an example of image acquisition means. The identification information acquisition unit 102 is an example of identification information acquisition means. The difference image generation unit 104 is an example of difference image generation means. The display control unit 105 is an example of display control means.

[0021] The display unit 150 is configured with any device such as an LCD (Liquid Crystal Display), CRT (Cathode Ray Tube), plasma display, or organic EL panel, and displays medical images, etc. Specifically, it displays cross-sectional images of the first medical image and the second medical image acquired from the image processing device 100. It also displays cross-sectional images and projection images of the difference image generated by the image processing device 100.

[0022] (Processing procedure of image processing device 100) FIG. 2 is a flowchart showing the overall processing procedure performed by the image processing device 100.

[0023] (S1010: Acquire input image) In step S1010, the image acquisition unit 101 acquires the first medical image and the second medical image specified by the user via the instruction unit 140 from the data server 130. Then, the image acquisition unit 101 outputs the first medical image and the second medical image to the identification information acquisition unit 102, the alignment information acquisition unit 103, the difference image generation unit 104, and the display control unit 105, respectively.

[0024] Note that image acquisition is not limited to being based on user instructions, and may be performed by any other method. For example, each time a new image is registered in the data server 130, it may be automatically determined based on a predetermined rule whether the image will be used as the first medical image (whether it will be used to generate a subtraction image). This determination may be performed based on, for example, the type of modality or the imaging range. For example, when an image of the chest and abdomen is captured using a CT scanner, the image may be used as the first medical image. In this case, the second medical image may be automatically acquired based on a predetermined rule from among past images stored in the data server 130. For example, an image from a previous examination of the same patient as the first medical image may be used as the second medical image. Alternatively, the first medical image may be designated by the user, and the second medical image may be automatically acquired using the above method.

[0025] (S1020: Acquisition of identification information) In step S1020, the identification information acquisition unit 102 acquires the identification information of the region depicted in the first medical image and outputs the acquired identification information to the difference image generation unit 104.

[0026] The identification information acquisition unit 102 acquires identification information of regions at multiple coordinates of the first image as identification information of regions. For example, the identification information acquisition unit 102 acquires information indicating to which region (organ) each of multiple pixels of the first medical image belongs. The identification information of regions is expressed, for example, in the form of a label image having, as pixel values, label values ​​that identify regions. The label image may be an image having the same resolution (pixel size) as the first medical image and corresponding one-to-one to each pixel of the first medical image (indicating to which region each pixel of the first medical image belongs), or may be an image having a different resolution (pixel size) from the first medical image. In the following, it is assumed that the identification information of regions is a label image having the same resolution as the first medical image.

[0027] Various processes can be applied to the first medical image as appropriate. For example, known adjustment processes such as deformation, rotation, alignment, resolution conversion, tone adjustment, masking, cropping, and angle-of-view expansion can be applied to the first medical image. The data format of the first medical image can also be converted, and additional information can be added to the first medical image. The term "first medical image" does not limit the presence or absence of such processes. That is, even if some processing is performed on the first medical image, if the processed image shows substantially the same information as the unprocessed image, the processed image is still the first medical image. Of course, if external processing such as subtraction or compositing with other images is performed on the first medical image, the processed image cannot be said to show substantially the same information as the first medical image. In such cases, the image will be described using a different name, such as a subtraction image or a composite image. The same applies to the second medical image.

[0028] In this embodiment, the identification information acquiring unit 102 acquires identification information identifying each of the first region, "spinal canal," and the second region, "bone," depicted in the first medical image. That is, a labeled image is acquired in which a label value representing "spinal canal" is assigned to pixels identified as depicting the spinal canal, and a label value representing "bone" is assigned to pixels identified as depicting a bone. Note that the bone labels may represent all bones as the same region (i.e., with the same label value), or may further identify bones (e.g., into skull, vertebrae, ribs, clavicle, sternum, pelvis, femur, etc.) and represent each region with a different label value. Furthermore, a labeled image may also identify regions other than "bone" and "spinal canal." Examples of other regions include organs and soft tissues.

[0029] The identification information of the region acquired by the identification information acquisition unit 102 will be described with reference to Figures 3A and 3B. Figure 3A shows a two-dimensional cross-sectional image cut out from a cross section of the first medical image. This example is a further enlarged cutout of the vertebrae and the vicinity of the vertebrae from a cross-sectional image showing an axial plane, depicting the vertebrae and the surrounding organs. Meanwhile, Figure 3B shows the identification information corresponding to this cross-sectional image. In Figure 3B, 310 indicates the area identified as the spinal canal, and 320 indicates the area identified as the bone.

[0030] Here, the identification of regions (organs) on the image is performed using a known technique. The identification information acquisition unit 102 in this embodiment identifies each region using a known region extraction method based on machine learning. To this end, an inference model (segmentation model) is constructed in advance, trained using pairs of CT images of numerous cases and their corresponding labeled images as training data. The identification information acquisition unit 102 inputs the first medical image to be processed into the inference model, thereby extracting labeled regions of each region (generating labeled images). Examples of known region extraction methods based on machine learning include U-Net based on a convolutional neural network (CNN) and its advanced forms. Note that the region extraction method is not limited to U-Net, and other known techniques may also be used. For example, methods based on a generative adversarial network (GAN) or a vision transformer are applicable, and any of these methods is applicable. In this case, an inference model created for each region may be used, or an inference model that simultaneously identifies multiple regions may be used. Furthermore, the results of classification by multiple inference models that classify the same region may be integrated and used. Region extraction may also be performed using a method other than machine learning, such as applying a statistical shape model of an organ to a medical image. Region extraction may also be performed using a rule-based image processing method such as that described in Patent Document 2, or any other region extraction method.

[0031] Alternatively, the identification information may be acquired by any other method. For example, the identification information acquisition unit 102 may be configured to read and acquire identification information (label image) previously associated with the first medical image and stored in the data server 130 without performing the identification process. Alternatively, the identification information acquisition unit 102 may perform a process to acquire identification information of the region of the second medical image, and then convert the region into the space of the first medical image using the alignment information described below to acquire the desired identification information. Furthermore, the identification information acquired from the second medical image in this manner may be integrated with the identification information acquired from the first medical image to acquire the identification information. For example, a pixel identified as "bone" in at least one of the two pieces of identification information may be identified as "bone." This allows for the identification of a region in one image (e.g., the first medical image) to be identified as "bone" even if it is difficult to identify the region in the other image (e.g., the second medical image) due to the progression of a disease, etc., and the correct identification information can be obtained if the region can be identified in the other image (e.g., the second medical image). For example, as osteolytic bone metastasis progresses, the area may become unrecognizable as bone in the first medical image, but if it can be recognized as bone in the second medical image, the area may be able to be identified as bone.

[0032] (S1030: Acquisition of alignment information) In step S1030, the registration information acquisition unit 103 acquires registration information representing the correspondence of coordinates between the first medical image and the second medical image, and outputs the acquired registration information to the difference image generation unit 104.

[0033] The registration information acquisition unit 103 in this embodiment performs deformation registration between the first medical image and the second medical image using a known inter-image registration method, and acquires registration information in the form of a displacement field between the images. For this deformation registration, known methods such as the Demons algorithm, Large Deformation Diffeomorphic Metric Mapping (LDDMM), and registration methods based on deep learning can be used.

[0034] Alternatively, the registration information may be acquired by any other method. For example, the registration information acquisition unit 103 may be configured to read and acquire registration information (displacement field) that is associated in advance with a pair of the first medical image and the second medical image and stored in the data server 130, without performing the registration process.

[0035] (S1040: Generate differential image) In step S1040, the difference image generating unit 104 generates a difference image between the first medical image and the second medical image and outputs the generated difference image to the display control unit 105.

[0036] FIG. 4 is a flowchart showing the procedure of the process performed by the difference image generating unit 104 in step S1040.

[0037] (S4005: Initialization of differential image) In step S4005, the difference image generating unit 104 initializes the difference image to be generated. Specifically, it secures a memory area for the difference image to be generated and assigns a pixel value of "0" to all pixels.

[0038] Here, it is desirable to set the image range of the difference image to the same range as the image range of the first medical image. Alternatively, the image range of the difference image may be a rectangular area or the like separately specified by the user. Furthermore, the resolution of the difference image may be the same as or different from that of the first medical image.

[0039] (S4010: Selecting the pixel of interest) In step S4010, the difference image generation unit 104 selects a pixel in the difference image for which a difference value has not been set as a pixel of interest. For example, the pixels of interest are determined sequentially by raster scanning. The difference image generation unit 104 also determines coordinates (first coordinates) on the first medical image corresponding to the selected pixel of interest based on the correspondence between the difference image and the first medical image. Furthermore, the difference image generation unit 104 determines coordinates (second coordinates) on the second medical image corresponding to the first coordinates based on the alignment information (displacement field) acquired in step S1030.

[0040] The pixel of interest may be each pixel on the difference image, or each pixel within a separately specified region of interest on the difference image (or on the first medical image). For example, the region of interest may be set to the overlapping range of the image range of the first medical image and the image range of the second medical image (a range in which a difference value can be calculated). Alternatively, the region of interest may be the inside of the patient's body on the first medical image, or a rectangular area or the like separately specified by the user.

[0041] (S4020: Determination of the first part) In step S4020, the difference image generation unit 104 determines whether the pixel of interest has been identified as belonging to the first region by referencing the identification information acquired in step S1020. More specifically, this determination is made based on whether the pixel value (label value) of the labeled image corresponding to the pixel of interest indicates the first region. In this embodiment, the first region is the spinal canal, so if the pixel value (label value) of the labeled image corresponding to the pixel of interest indicates the spinal canal, it is determined that "the pixel of interest belongs to the first region." If it is determined that "the pixel of interest belongs to the first region," the process proceeds to step S4021, where calculation processing of a difference value appropriate for the first region is performed. On the other hand, if it is determined that "the pixel of interest does not belong to the first region," the process proceeds to step S4030.

[0042] (S4021: Calculation of initial difference value) In step S4021, the difference image generating unit 104 calculates an initial difference value when the pixel of interest is identified as belonging to the first region (the spinal canal in this embodiment). In this embodiment, the initial difference value is a simple difference value obtained by simply subtracting the pixel value at the second coordinate on the second medical image from the pixel value at the first coordinate on the first medical image. This is to avoid signal loss due to noise removal processing, etc., because the characteristic changes between CT images that are of interest within the spinal canal are slight in terms of pixel values.

[0043] The pixel value of the first coordinate on the first medical image is the pixel value of the pixel located at the first coordinate among the pixels included in the first medical image. Alternatively, the pixel value of the first coordinate on the first medical image is a statistical value (average value, median value, etc.) based on a plurality of pixels included within a predetermined range from the first coordinate among the pixels included in the first medical image. The same applies to the pixel value of the second coordinate on the second medical image.

[0044] In the embodiments, the pixel values ​​of the first and second medical images are described assuming that the larger the CT value, the larger the pixel value. In this case, high-density regions, such as bones, have higher pixel values ​​than low-density regions. Of course, the definition of the pixel values ​​of each image can be arbitrarily modified, and it is also possible to set the pixel values ​​so that high-density regions have lower pixel values. Even in this case, the respective embodiments can be similarly applied, for example, by determining the direction of the inequality sign in Equation (1) described below.

[0045] (S4022: Correction and storage of differential values) In step S4022, the difference image generation unit 104 corrects the initial difference value when the pixel of interest is identified as belonging to the first region (the spinal canal in this embodiment).The difference image generation unit 104 then records the corrected difference value (corrected difference value) in the difference image as the pixel value of the pixel of interest.Then, the process proceeds to step S4050.

[0046] More specifically, the difference image generation unit 104 determines whether the initial difference value satisfies a predetermined condition, and determines the corrected difference value obtained by performing a predetermined correction process on the initial difference value according to the determination result as the pixel value of the pixel of interest. As an example, the difference image generation unit 104 determines the polarity (positive or negative) of the initial difference value, and changes (corrects) the difference value to "0" if the polarity is a predetermined one (negative in this embodiment). That is, the difference image generation unit 104 determines the corrected difference value using equation (1).

[0047]

number

[0048] Here, x represents the initial difference value, and x' represents the correction difference value. In other words, "the polarity of the initial difference value" is an example of a predetermined condition, and formula (1) is an example of a predetermined correction process according to the determination result. As shown in formula (1), it is preferable to set the correction value to "0", but approximately the same effect can be obtained by setting the correction value to a value close to "0" (for example, "-1" or "1").

[0049] Note that the predetermined correction process according to the determination result may be other than the above. For example, when the initial difference value satisfies a predetermined condition, the difference image generation unit 104 may perform a correction to attenuate the difference value by multiplying the initial difference value by a predetermined coefficient. That is, the difference image generation unit 104 determines the corrected difference value using equation (2).

[0050]

number

[0051] Here, a is a positive coefficient and takes a value greater than or equal to "0" and less than "1." For example, values ​​such as "0.05," "0.1," and "0.2" can be set.

[0052] As a predetermined correction process according to the determination result, a process of emphasizing a difference value other than the difference value to be attenuated (a positive difference value in this embodiment) may be used, instead of a process of attenuating a difference value to be attenuated (a negative difference value in this embodiment) as in equation (2). For example, the difference image generating unit 104 may determine a corrected difference value according to equation (3).

[0053]

number

[0054] Here, b is a positive coefficient and takes a value greater than "1." For example, values ​​such as "1.2," "1.5," and "2.0" can be set.

[0055] Furthermore, the predetermined condition is not limited to "the polarity of the initial difference value." For example, the predetermined condition may be "whether the initial difference value is less than a first threshold value near 0." In this case, the difference image generating unit 104 determines the corrected difference value using equations (4) to (6), respectively, instead of the above equations (1) to (3).

[0056]

number

[0057]

number

[0058]

number

[0059] Here, t represents a first threshold value, and preferably takes a value close to "0" (for example, "-1" or "1"). In equations (4) to (6), correction processing is performed when the initial difference value is less than the first threshold value, but correction processing may also be performed when the initial difference value is equal to or less than the first threshold value.

[0060] When a disease occurs in the spinal canal (mainly cancer tumor infiltration into the spinal canal), it is depicted as an increase in pixel values ​​on the CT image, and the disease tends to be depicted in the subtraction image as a positive difference value. On the other hand, negative difference values ​​are likely to be registration errors, artifacts on the CT image, or normal changes that doctors do not notice. Therefore, by removing or attenuating these signals from the subtraction image, changes that are useful for diagnosis can be highlighted.

[0061] As described above, when the threshold value is set to "0", the difference image generating unit 104 determines the initial difference value using equations (1) to (3), and when the first threshold value is used, which is not strictly "0" but is a value close to "0", the difference image generating unit 104 determines the initial difference value using equations (4) to (6). Here, the first threshold value may simply be a value that approximates "0", or may be adjusted according to the purpose.

[0062] For example, when a value greater than "0" is used as the first threshold, not only a decrease in pixel value but also a slight increase in pixel value is removed or attenuated from the difference image. A slight increase in pixel value may be due to a simple measurement error. By removing or attenuating such a signal from the difference image, changes useful for diagnosis are more prominent, making the difference image easier to see. On the other hand, when a value less than "0" is used as the first threshold, even if a decrease in pixel value is a slight change, the signal is not removed or attenuated from the difference image, making it less likely that a disease will be overlooked. The first threshold may be set taking into consideration the balance between the ease of viewing the difference image and preventing the disease from being overlooked. The first threshold may also be adjustable by the user.

[0063] (S4030: Determination of the second part) In step S4030, the difference image generation unit 104 determines whether the pixel of interest has been identified as belonging to the second region (bone in this embodiment) by the same process as in step S4020. If it is determined that the pixel of interest belongs to the second region, the process proceeds to step S4031, where a difference value suitable for the second region is calculated. On the other hand, if it is determined that the pixel of interest does not belong to the second region, the process proceeds to step S4040.

[0064] (S4031: Calculation of initial difference value) In step S4031, the difference image generating unit 104 calculates an initial difference value when the pixel of interest is identified as belonging to the second region (bone in this embodiment). In this embodiment, the voxel matching method described in Patent Document 3 is used to calculate a difference value between a first coordinate on the first medical image and a second coordinate on the second medical image (more precisely, between a region near the first coordinate and a region near the second coordinate), and this is set as the initial difference value. This makes it possible to reduce noise caused by the partial volume effect when the slices of the CT image are thick.

[0065] (S4032: Save differential value) In step S4032, the difference image generating unit 104 records the initial difference value calculated in step S4031 as the pixel value of the pixel of interest when the pixel of interest is identified as belonging to the second part (bone in this embodiment) in the difference image as is, and then proceeds to step S4050.

[0066] (S4040: Save differential value) In step S4040, the difference image generation unit 104 records a difference value of "0" in the difference image as the pixel value of a pixel of interest that is identified as belonging to neither the first nor the second region, and then proceeds to step S4050.

[0067] (S4050: Termination determination) In step S4050, the difference image generation unit 104 determines whether or not there are any pixels remaining for which a difference value has not been set. If it is determined that there are any pixels remaining for which a difference value has not been set, the process proceeds to step S4010, where the process of setting a pixel value for a new pixel of interest is executed again. On the other hand, if it is determined that the process has been completed for all pixels, the process proceeds to step S4060.

[0068] (S4060: Save differential image) In step S4060, the difference image generation unit 104 stores the generated difference image in a storage unit (not shown). By storing the generated difference image in the storage unit, when it is desired to acquire a difference image of the same input image again after the processing of the image processing device 100 is completed, the difference image generation unit 104 can easily acquire the difference image by reading the difference image from the storage unit. Note that the processing of this step of storing the difference image in the storage unit is not necessarily performed.

[0069] In this way, the process of step S1040 is carried out.

[0070] (S1050:Display) In step S1050, the display control unit 105 controls the display of the difference image etc. generated by the difference image generation unit 104 on the display unit 150. A GUI for receiving instructions from a user such as a doctor is arranged on the display unit 150. The user can freely switch between the cross-sectional images or projection images of the first medical image, the second medical image, and the difference image, or display them side by side.

[0071] In this way, the processing of the image processing device 100 is carried out.

[0072] In the medical field, attempts have been made to visualize changes over time, such as lesions, by generating a difference image between images captured at different times using various modalities. For example, Patent Document 1 discloses a technique for aligning two CT images to generate a difference image suitable for bone observation. Patent Document 2 discloses a technique for recognizing bones and spinal canals on CT images and calculating difference values ​​using methods suitable for each observation to generate a difference image. However, the techniques of Patent Documents 1 and 2 sometimes contain unnecessary information that can interfere with the observer's judgment. In contrast, according to this embodiment, unnecessary information contained in the difference image can be suppressed by calculating and correcting difference values ​​according to the region.

[0073] (Variation 1) The difference image generating unit 104 may further extract a region of interest based on the initial difference value or the corrected difference value. For example, as shown in equations (1) to (6), the difference image generating unit 104 determines whether the initial difference value for the pixel identified as the first region is equal to or less than a first threshold value close to "0," and then performs a correction process on the initial difference value according to the determination result to obtain a corrected difference value. As a result, negative difference values ​​that are unlikely to represent a disease are deleted or attenuated in the difference image, and positive difference values ​​are emphasized. Here, the difference image generating unit 104 may further extract a region of interest that is particularly likely to represent a disease from among regions having positive difference values.

[0074] For example, the difference image generation unit 104 extracts a region of interest by thresholding the initial difference value or the corrected difference value with a second threshold for multiple pixels of the difference image identified as the first region. Specifically, when the difference image generation unit 104 obtains a corrected difference value using equation (1), equation (2), equation (4), or equation (5), it extracts a region of interest by thresholding the initial difference value with the second threshold. Furthermore, when the difference image generation unit 104 obtains a corrected difference value using equation (3) or equation (6), it extracts a region of interest by thresholding the corrected difference value with the second threshold. The second threshold is not limited to a specific value, but is preferably larger than the first threshold. Furthermore, the second threshold may be arbitrarily adjustable by the user.

[0075] As described above, the subtraction image generating unit 104 according to the first modification eliminates or weakens negative subtraction values, and extracts, as a region of interest, a region with a particularly high probability of disease among the positive subtraction values. In other words, by using two criteria, the subtraction image generating unit 104 can suppress unnecessary information contained in the subtraction image while prioritizing necessary information, thereby achieving more efficient image diagnosis.

[0076] (Variation 2) In this embodiment, in the process of step S4040, the difference image generation unit 104 sets the difference value of a pixel of interest identified as a region that is neither the first region nor the second region (a background region) to "0." However, a configuration may also be adopted in which a difference value is set for a pixel of interest identified as a background region. This prevents the difference value from being completely erased when the accuracy of the region identification process is insufficient and an oversight occurs (i.e., when a pixel of interest that is actually the first region or the second region is mistakenly identified as a background region), thereby reducing the risk of overlooking a change. In this case, for example, the method described in Patent Document 1 can be used as a method for calculating the difference value. Note that even in this case, a configuration may be adopted in which the difference value of a pixel of interest identified as outside the body is set to "0."

[0077] (Variation 3) In this embodiment, the spinal canal is the first region. However, any region other than the spinal canal may be the first region as long as it is a region where a disease that is depicted as an increase in pixel values ​​on an image is the main disease.

[0078] For example, the area around the vertebrae (perivertebral area) may be the first region. The perivertebral area is, for example, the soft tissue adjacent to the vertebrae. Alternatively, both the spinal canal and the perivertebral area may be the first region. Extraosseous invasion of cancer bone metastasis may occur around the vertebrae. This extraosseous invasion is depicted as an increase in pixel values ​​on CT images, and the disease tends to be depicted as a positive difference value on the difference image, so removing or attenuating negative difference values ​​is effective.

[0079] At this time, in the process of step S1030, the identification information acquiring unit 102 further separates and recognizes the vertebrae from the bones that are the second region, thereby identifying the region around the vertebrae (for example, the region within a predetermined distance from the outer edge of the vertebra) as the “vertebra surrounding region.” At this time, the predetermined distance for the dorsal side of the vertebra may be set relatively smaller than that for the ventral side, thereby preventing the vertebra surrounding region from protruding from the body region.

[0080] When the vertebral region is the first region, in the process of step S4022, the difference image generating unit 104 may not use the calculated corrected difference value x' as the pixel value of the difference image as it is, but may set a new corrected difference value x'' obtained by further performing correction processing according to the distance from the outer edge of the vertebra as the pixel value of the difference image. Specifically, the corrected difference value x' of the pixel of interest and the difference value x_bg when the pixel of interest is in the background region (hereinafter referred to as the difference value of the background region) may be combined using a combination ratio that takes "1" at the outer edge of the vertebra and gradually decreases as the distance from the outer edge of the vertebra increases, and the combined value may be saved as the pixel value of the difference image (corrected difference value x''). That is, the corrected difference value x'' is determined using equation (7).

[0081]

number

[0082] Here, w represents the blending ratio. By adjusting the blending ratio to "0" at the boundary between the vertebral region and the background region further outside, the continuity of the difference value near the boundary can be maintained. Here, the difference value x_bg of the background region can be calculated, for example, by the method described in Patent Document 1, similar to Modification Example 2.

[0083] To explain this using a specific example, for example, when the correction difference value x' is defined by equation (1) (i.e., when a correction method is used in which the correction difference value is corrected to "1" when the initial difference value is negative), the new correction difference value x'' is set by equation (8) in which equation (1) is substituted into equation (7).

[0084]

number

[0085] That is, when x is a positive value (the lower part of equation (8), when the specified condition is not satisfied), the initial difference value x is gradually replaced with the background difference value x_bg according to the distance from the outer edge of the vertebrae, and a value obtained by gradually replacing the initial difference value x with the background difference value x_bg is saved as the corrected difference value x''. On the other hand, when x is a negative value (the upper part of equation (8), when the specified condition is satisfied), a value obtained by gradually replacing the value 0 with the background difference value x_bg according to the distance from the outer edge of the vertebrae is saved as the corrected difference value x''. That is, the initial difference value is corrected to 0 at the outer edge of the vertebrae. Then, the degree of correction weakens (the synthesis ratio of the background difference value x_bg increases) as the distance from the outer edge of the vertebrae increases, and the background difference value x_bg is applied as is at the boundary between the vertebral peripheral region and the background region.

[0086] Here, if the difference value x_bg of the background region is set to "0" (i.e., if the processing described in step S4040 of the first embodiment is performed), this processing is equivalent to processing in which the corrected difference value x' is subjected to attenuation processing adjusted so that the degree of attenuation gradually increases with increasing distance from the outer edge of the vertebrae, and the resulting value is set in the difference image. That is, equations (7) and (8) are simplified as equations (9) and (10).

[0087]

number

[0088]

number

[0089] That is, when x is a positive value (the lower part of equation (10), when the specified condition is not met), the initial difference value x is attenuated according to the distance from the outer edge of the vertebrae and saved as the corrected difference value x''. On the other hand, when x is a negative value (the upper part of equation (10), when the specified condition is met), the value "0" is saved as the corrected difference value x'' regardless of the distance from the outer edge of the vertebrae. As a result, the corrected difference value x'' approaches "0" as the distance from the outer edge of the vertebrae increases, and a difference image with a smooth boundary with the background region can be obtained.

[0090] On the other hand, when the initial difference value x calculated in the same manner as for the first part is given as the difference value x_bg of the background region (i.e., when x_bg=x), equation (8) is simplified to equation (11).

[0091]

number

[0092] This corresponds to gradually weakening the process of correcting x to x' as the distance from the outer edge of the vertebra increases. That is, when x is a positive value (the lower part of equation (11), when the specified condition is not met), the initial difference value x is always stored as the corrected difference value x'', regardless of the distance from the outer edge of the vertebra. On the other hand, when x is a negative value (the upper part of equation (11), when the specified condition is met), a value subjected to correction processing according to the distance from the outer edge of the vertebra is stored as the corrected difference value x''. That is, the difference value is corrected to "0" at the outer edge of the vertebra. Then, the degree of correction weakens as the distance from the outer edge of the vertebra increases, and the initial difference value x is applied as is at the boundary between the vertebral peripheral region and the background region.

[0093] In this way, for areas that are not clearly defined anatomically, such as the area around the vertebrae, the correction amount of the difference value is gradually changed from a reference position such as the outer edge of the vertebrae to maintain the continuity of the difference value, thereby preventing unnatural depiction of the difference value.

[0094] Furthermore, if the first region consists of multiple regions, the difference image generating unit 104 may use a different correction method for the initial difference value for each region in step S4022. More specifically, if both the spinal canal and the vertebral surrounding region are defined as the first region, the difference image generating unit 104 sets the negative difference value of the spinal canal to "0." On the other hand, the difference image generating unit 104 adjusts the corrected difference value for the vertebral surrounding region so that the degree of attenuation gradually increases with the distance from the outer edge of the vertebra. In this way, by using different correction methods for the initial difference value depending on the region, it is possible to depict difference values ​​that are appropriately corrected depending on the region.

[0095] (Variation 4) In this embodiment, a region (organ) where a disease is primarily present, which is depicted as an increase in pixel values ​​on the image, is defined as the first region, and a subtraction image is generated by removing or attenuating negative difference values ​​only for that region. Alternatively, a region (organ) where a disease is primarily present, which is depicted as a decrease in pixel values ​​on the image, may be defined as the first region, and a subtraction image is generated by removing or attenuating positive difference values ​​only for that region. For example, a time-lapse subtraction image using this processing may be generated for a mass depicted in a breast ultrasound image. Because a breast mass on an ultrasound image is primarily depicted as a darker region than normal tissue, removing or attenuating positive difference values ​​makes it possible to highlight the region of interest on the time-lapse subtraction image. Note that a region where negative differences are removed or attenuated and a region where positive differences are removed or attenuated (and a region where differences are not removed or attenuated) may coexist.

[0096] (Variation 5) In this embodiment, the difference image generation unit 104 calculates the initial difference values ​​for the first and second regions using different processes in the process of step S1040. However, the initial difference values ​​may be calculated using a common process that is independent of the regions. For example, a simple difference may be used as the initial difference value for both regions. In this case, the difference image generation unit 104 may perform the same process as step S4021 in step S4031. In this case, the difference image generation unit 104 may generate an initial difference image including the initial difference values ​​immediately after step S4005, and obtain the initial difference values ​​from this initial difference image in the processes of steps S4022 and S4032. Note that the initial difference values ​​may be calculated using other methods. For example, the initial difference values ​​may be calculated using a voxel matching method for both regions. Note that the method for calculating the initial difference values ​​is not limited to the simple difference or voxel matching method exemplified in the embodiment, and any known difference value calculation method may be used.

[0097] (Variation 6) In this embodiment, in step S1050, the display control unit 105 controls the display of the generated difference image on the display unit 150. However, the process of displaying the difference image is not necessarily required. Instead of displaying the difference image, the difference image may be associated with the first medical image and output to an external device (for example, the data server 130). Furthermore, the system may further include an image recognition unit (not shown) that performs image recognition processing using the generated difference image as input. For example, the difference image may be input to an inference model (not shown) to determine whether or not there is an abnormality, and the results may be displayed or saved.

[0098] (Variation 7) In the present embodiment, in the process of step S1040, the difference image generation unit 104 generates a difference image (integrated difference image) that integrates and represents difference values ​​for multiple regions, i.e., a first region and a second region, into a single image. However, a configuration may be adopted in which a difference image is generated for each region without generating an integrated difference image. For example, a first difference image that stores only difference values ​​for the first region and a second difference image that stores only difference values ​​for the second region may be separately generated. In this case, in the process of step S1050, the display control unit 105 may display the two difference images side by side, or may switch the displayed difference image in response to a user instruction. Furthermore, when displaying, the two difference images may be integrated and displayed. Furthermore, a configuration may be adopted in which both an integrated difference image and a difference image for each region are generated.

[0099] Furthermore, in the process of step S1040, the difference image generating unit 104 may be configured to generate an initial difference image obtained by visualizing an initial difference value, in addition to the difference image (corrected difference image) generated in the above embodiment. In this case, in the process of step S1050, the display control unit 105 may display the initial difference image and the corrected difference image side by side, or may switch between them in response to a user instruction. For example, the display control unit 105 may be configured to normally display the corrected difference image and only display the initial difference image while the user is pressing a specific key. Alternatively, the display control unit 105 may be configured to determine whether the user's cursor is pointing at a first region on the first medical image or the difference image based on the region identification information, and to display the corrected difference image only while the cursor is pointing at the first region, and to display the initial difference image otherwise. Furthermore, in a configuration in which the pixel values ​​of the coordinates indicated by the user's cursor are displayed as text information at a predetermined display position (for example, in the four corners of the image display area, in an information display area adjacent to the image display area, or diagonally below the cursor), similar to a general image viewer, even if a corrected difference image is displayed as an image, the pixel values ​​of the initial difference image may be used to display the text information. Also, a configuration in which pixel values ​​before and after correction are displayed side by side may be used. In this case, the pixel values ​​before and after correction may be displayed side by side only if they are different, and not displayed side by side if the pixel values ​​are the same. Furthermore, a configuration in which the manner in which the text information is displayed (such as by changing the color, font, or size) may be changed depending on whether correction has been made or not, to notify the user that correction has been made.

[0100] Furthermore, in the processing of step S1050, the difference image generating unit 104 may be configured to display only the initial difference image, and if a negative difference value exists in the first region on the displayed difference image, to notify the user by using an icon, text, or the like that there is a negative difference value that should be ignored. In this case, generation of the corrected difference image may be omitted. Furthermore, when displaying the initial difference image, the notification that there is a difference value that should be ignored may be provided only while the user's cursor is pointing at the first region on the first medical image or the initial difference image. Alternatively, the notification that there is a difference value that should be ignored may be provided only while the cursor is pointing at a pixel in the first region that satisfies a predetermined condition (i.e., is to be corrected). Furthermore, when the pixel value of the coordinates indicated by the cursor is displayed as text information at a predetermined display position, the pixel value of the corrected difference image may be displayed as the text information.

[0101] (Variation 8) Furthermore, in the process of step S1050, the display control unit 105 may further superimpose identification information of (at least one of) the first and second regions on the generated difference image to indicate which region each pixel in the difference image represents. The identification information may be superimposed in a predetermined, highly transparent color to distinguish it from the difference image (expressed in grayscale), or outline information of the region may be superimposed. Alternatively, the display control unit 105 may be configured to superimpose region information of the corresponding region on the first medical image or the difference image only when the user overlays the cursor on the displayed difference image. The user may be notified of which region the coordinates pointed by the cursor correspond to by switching the cursor design (color or type). Alternatively, an icon indicating the region may be displayed near the cursor or at a predetermined position on the displayed image.

[0102] The display control unit 105 may also display the image so that the area where the correction process has been performed can be identified. Furthermore, the display control unit 105 may display the image so that the type of correction process can be identified. That is, the display control unit 105 may display the difference image so that the area in the difference image that has a correction difference value as a pixel value and the type of correction process performed when the correction difference value was acquired can be identified. For example, the display control unit 105 may display an icon indicating the presence or absence of correction process and the type of correction process. For example, when the cursor is pointing at an area where negative values ​​are being removed or attenuated, a "+" icon may be displayed near the cursor. Similarly, when the cursor is pointing at an area where positive values ​​are being removed or attenuated, a "-" icon may be displayed near the cursor. Alternatively, the icon indicating the presence or absence of correction process and the type of correction process may be displayed only when the cursor is pointing at a pixel where correction process has been performed.

[0103] (Variation 9) In step S1040 of this embodiment, when the difference image generation unit 104 determines that the pixel of interest belongs to the first region, the difference image is generated by first calculating an initial difference value (step S4021) and then correcting the difference value (step S2022). However, the method of generating the difference image is not necessarily limited to this. When calculating the difference value for the first region, instead of calculating the initial difference value and determining its polarity, the pixel value of the first coordinate and the pixel value of the second coordinate used to calculate the difference value may be directly compared. More specifically, if the pixel value of the second coordinate is greater than the pixel value of the first coordinate, the difference value between them may be set to 0 or may be weakened. This eliminates the need to calculate and store the initial difference value.

[0104] That is, the difference image generating unit 104 determines whether the pixel values ​​of the first coordinates of the first medical image corresponding to the pixel of interest and the second coordinates of the second medical image corresponding to the first coordinates satisfy a predetermined condition, and performs a correction process on the initial difference value according to the determination result to obtain a corrected difference value. At this time, the determination of whether the pixel values ​​of the first coordinates and the second coordinates satisfy the predetermined condition may be made based on the initial difference value, which is the difference between these pixel values, or may be made by directly using these pixel values.

[0105] <Second embodiment> The image processing device according to this embodiment is a device that receives an existing difference image and applies correction according to the region to generate a corrected difference image. Only the parts that are different from the first embodiment will be described below.

[0106] 5 is a diagram showing the configuration of an image processing system 30 according to the second embodiment. The image processing system 30 includes an image processing device 500 and a data server 130. Note that, like the image processing system 10 of FIG. 1, the image processing system 30 may include a display unit 150.

[0107] The image processing device 500 is a device that generates a corrected differential image by applying correction to a differential image, and includes at least a differential image acquisition unit 501, an identification information acquisition unit 502, and a differential image correction unit 504. The differential image acquisition unit 501 acquires information on the differential image input to the image processing device 500. The identification information acquisition unit 502 acquires identification information of the area depicted in the differential image. The differential image correction unit 504 generates a corrected differential image by applying correction to the differential image based on the identification information. These components are realized by a processing circuit included in the image processing device 500 executing a program. The differential image acquisition unit 501 is an example of differential image acquisition means. The identification information acquisition unit 502 is an example of identification information acquisition means. The differential image correction unit 504 is an example of differential image correction means.

[0108] (Processing procedure of image processing device 500) FIG. 6 is a flowchart showing the overall processing procedure performed by the image processing device 500.

[0109] (S6010: Obtaining differential images) In step S6010, the difference image acquisition unit 501 acquires a difference image specified by the user via the instruction unit 140 from the data server 130. Then, the acquired difference image is output to the difference image correction unit 504. The difference image in this embodiment is a difference image generated in advance from a first medical image and a second medical image captured at different times of the subject, and is stored in association with the first medical image.

[0110] (S6020: Acquisition of identification information) In step S6020, the identification information acquisition unit 502 acquires identification information of the region associated with the difference image from the data server 130. Then, the acquired identification information is output to the difference image correction unit 504. The difference image in this embodiment is a labeled image of the region generated in advance for the first medical image, and is stored in association with the first medical image.

[0111] (S6040: Generate corrected differential image) In step S6040, the difference image corrector 504 performs correction processing on the difference image based on the part identification information to generate a corrected difference image, and then stores the generated corrected difference image in a storage unit (not shown).

[0112] The process performed by the differential image correction unit 504 in this step is generally similar to the process performed by the differential image generation unit 104 in step S1040 of the first embodiment, except for the method of acquiring the initial differential value. Specifically, the process of calculating the initial differential value of the pixel of interest based on the difference between images in step S4021 of the first embodiment is changed to a process of reading the initial differential value from the differential image acquired by the differential image acquisition unit 501 in step S4021 of this embodiment. As a result, when the identification information of the pixel of interest in the differential image is identified as the first region, it is determined whether the pixel value of the pixel of interest satisfies a predetermined condition, and a predetermined correction process corresponding to the determination result is applied to the pixel value to generate a corrected differential image. The same process is also performed when the identification information of the pixel of interest is identified as the second region.

[0113] In this way, the processing of the image processing device 500 is carried out.

[0114] According to this embodiment, unnecessary information contained in the difference image can be suppressed by performing correction according to the region.

[0115] <Third embodiment> The image processing device 300 of this embodiment differs from the image processing device 100 of the first embodiment in that it has an image adjustment unit 113 instead of the alignment information acquisition unit 103, and a differential image generation unit 114 and a differential image correction unit 116 instead of the differential image generation unit 104.

[0116] 7 , the image processing device 300 includes an image acquisition unit 101, an image adjustment unit 113, a difference image generation unit 114, an identification information acquisition unit 102, and a difference image correction unit 116. The image processing device further includes a display control unit 105 that performs display control on the difference image corrected by the difference image correction process performed by the difference image correction unit 116 and displays the corrected image on the display unit 150.

[0117] 8, image processing method 3000 according to this embodiment includes step S1010, which is an image processing step, step S1130, which is an image adjustment step, step S1140, which is a difference image generation step, step S1120, which is an identification information acquisition step, and step S1160, which is a difference image correction step. Image processing method 3000 further includes step S1050, which is a display step, in which display control is performed on the difference image corrected by the difference image correction processing in step S1160, which is the difference image correction step, and the display is displayed on display unit 150.

[0118] (Image adjustment unit 113) The image adjustment unit 113 adjusts at least one of the first image and the second image acquired by the image acquisition unit 101, which are of different time phases, so as to correspond one of them to the other. The image adjustment performed by the image adjustment unit 113 includes at least some of deformation processing, rotation processing, alignment processing, resolution conversion processing, tone adjustment processing, masking processing, trimming processing, angle of view expansion processing, and position reference acquisition processing that acquires information regarding a predetermined position reference. In some embodiments, the image adjustment unit 113 further performs alignment processing that performs alignment based on a predetermined position reference. The predetermined position reference includes anatomical features and landmarks assigned on the imaging.

[0119] In other words, image processing device 300 includes image acquisition unit 101 that acquires a first image and a second image captured at different times of the same subject, and image adjustment unit 113 that adjusts at least one of the first image and the second image so as to correspond to the other. The image adjustment operation performed by image adjustment unit 113 is run on a computer by a program that executes step S1130, which is an image adjustment process.

[0120] (Difference image generation unit) The difference image generating unit 114 is configured to calculate a difference value between the other of the first and second images and one of the adjusted images, the image captured later in time, relative to the earlier in time, to generate a difference image. In other words, the difference image generating unit 114 is a means for calculating a difference value between two different phase images, at least one of which has been subjected to registration processing by the image adjusting unit 113, and generating a difference image (third image).

[0121] The image processing system 30 to which the image processing device 300 of this embodiment is applied includes an image adjustment unit 113 that performs deformation and alignment, and a difference image generation unit 114 that generates a difference image between two images that have been deformation-aligned and associated. With these elements, the image processing system 30 to which the image processing device 300 is applied supports the identification of a changed area in a current image relative to a previous image, which is of greater interest in identifying the location of a diseased area, among two images of the same subject taken at different time phases.

[0122] The difference image generating unit 114 according to the present embodiment generates a difference image using two images of different time phases that have been adjusted by the image adjusting unit 113 but have not been corrected by the difference image correcting unit 116 (described later). In other words, the difference image (third image) generated by the difference image generating unit 114 is an initial difference image having the initial difference value described in the difference image generating unit 104 according to the first embodiment.

[0123] (Identification information acquisition unit) As in the first embodiment, the identification information acquisition unit 102 extracts an anatomical region from at least one of the chronologically subsequent images and the subtraction image (third image) and acquires identification information for the region. In other words, the identification information acquisition unit 102 identifies the anatomical region so that the first region corresponds to a region other than the bone. Furthermore, the identification information acquisition unit 102 identifies the anatomical region so that the first region corresponds to a spinal canal region or an extra-osseous region.

[0124] The identification information acquisition unit 102 of this embodiment is configured to extract an anatomical region from at least one of a chronologically subsequent image and a subtraction image corresponding to the third image. It can be seen from FIG. 7 that the identification information acquisition unit 102 is configured to extract an anatomical region based on a subtraction image corresponding to the third image from the subtraction image generation unit 114. The identification information acquisition unit 102 is configured to extract an anatomical region based on a chronologically subsequent image (current image) that is not a temporal subtraction image, but the signal transmission path from the image adjustment unit 113 to the identification information acquisition unit 102 is omitted in the illustration. The operation of acquiring identification information performed by the identification information acquisition unit 102 is run on a computer by a program that executes step S1020, which is an identification information acquisition process.

[0125] (Differential image correction section) The difference image correcting unit 116 performs correction on the difference image (third image) generated by the difference image generating unit 114 depending on whether the identification information of the pixel of interest in the difference image is the first part or not and whether the pixel value of the difference image is smaller than a predetermined threshold or not. The time-lapse difference image after the correction, which is adaptively corrected by the difference image correcting unit 116 depending on the identification information and the pixel value of the difference image, may be referred to in this specification as a fourth difference image.

[0126] 7, the differential image correction unit 116 acquires the identification information of the pixel of interest from the identification information acquisition unit 102. The differential image correction unit 116 may handle the pixel of interest by replacing it with a region of interest made up of multiple pixels that share anatomical characteristics. In other words, the differential image correction unit 116 is a correction means that performs correction processing depending on whether the pixel of interest falls within the first region, i.e., the "spinal canal."

[0127] 7, the differential image correction unit 116 is a correction unit that adaptively performs correction processing depending on whether the pixel value of the differential image (third image), which is the criterion for determining the pixel value of the differential image, is smaller than a predetermined threshold value. The differential image correction unit 116 can acquire the predetermined threshold value Pth via an instruction unit 140 that receives instructions from a user. The predetermined threshold value Pth may be selected by the user from a menu list, or may be determined based on information such as PACS, electronic medical records, and examination orders.

[0128] 9A to 9D, when the pixel of interest is identified as the spinal canal, which is the first region, the differential image correcting unit 116 corrects at least one of the pixel values ​​of the pixel of interest POI whose pixel value is smaller than a predetermined threshold Pth and the pixel values ​​of the pixel of interest POI whose pixel value is larger than the predetermined threshold Pth. At this time, the differential image correcting unit 116 corrects the differential image so that the pixels of the pixel of interest POI whose pixel value is smaller than the predetermined threshold Pth have lower visibility relative to the pixels of the pixel of interest POI whose pixel value is larger than the predetermined threshold Pth.

[0129] The differential image correction unit 116 and the differential image correction process, step S1160, correct the bone dissolution signals on the time-lapse differential image, in which the pixel values ​​of the differential image are lower than a predetermined threshold and the CT value decreases over time, to attenuate the components corresponding to the inside of the spinal canal, which is a non-bone region, as shown in Figures 9A to 9D.

[0130] The predetermined threshold Pth is appropriately specified based on the background noise level of the X-ray imaging system, the quantization noise level resulting from the image reconstruction process, the minimum gradation width of the brightness contour on the display including the zero brightness level of the difference image, and other factors relative to the difference value, which is the pixel value of the temporal difference image. To ensure reproducibility of the original image and reduce the impact on the interpretation accuracy of the radiologist, it is desirable for the predetermined threshold Pth to be a negative value within the minimum unit of noise level or gradation near the zero brightness. Figures 9A and 9C illustrate the correction processing operation, including the attenuation correction processing, of the difference image corrector 116 in the third embodiment, in which the predetermined threshold Pth is set to a negative value (Pth2<0). Meanwhile, Figure 9B illustrates the correction processing operation, including the attenuation correction processing, of the difference image corrector 116 in the third embodiment, in which the predetermined threshold Pth is set to 0. FIG. 9D shows an aspect of the correction processing operation including the attenuation correction processing of the differential image corrector 116 in the third embodiment, in which the predetermined threshold value Pth is set to a positive value (Pth3>0).

[0131] The differential image correction unit 116, in the same manner as the differential image correction unit 106 of the first embodiment as shown in Figures 9B to 9D, performs differential attenuation processing in which, when the pixel of interest POI is identified as the first region, the inside of the spinal canal, and the pixel value of the pixel of interest POI is smaller than a predetermined threshold value Pth, the pixel value of the pixel of interest POI is multiplied by a predetermined coefficient a greater than or equal to 0 and less than 1.

[0132] The differential image correction unit 116 does not perform differential attenuation processing on the pixel value of the pixel of interest POI if the pixel of interest POI is identified as not being in the first region (inside the spinal canal) or if the pixel value of the pixel of interest POI is greater than or equal to a predetermined threshold value Pth. The differential image correction processing operation performed by the differential image correction unit 116 is run on a computer by a program that executes step S1160, which is a differential image correction step.

[0133] <Fourth embodiment> An image processing device 400 according to this embodiment differs from the image processing device 300 according to the third embodiment in that it includes a differential image correction unit 126 instead of the differential image correction unit 116, as shown in FIG.

[0134] The image processing method (not shown) of this embodiment differs from the image processing method 3000 of the third embodiment in that, instead of step S1160 which is a differential image correction process, it includes a differential image correction process (not shown) which corrects the display mode other than weakening or strengthening of pixel values.

[0135] <Difference image correction section> As shown in FIGS. 11A and 11B, the difference image correction unit 126 corrects the difference image so that pixels of interest POI identified as the inside of the spinal canal, which is the first region, whose pixel value is smaller than a predetermined threshold Pth are displayed in a different manner from pixels identified as not belonging to the first region whose pixel value is smaller than the predetermined threshold Pth. For example, if the pixel value is larger than the predetermined threshold Pth or if the region is not identified as the first region, the difference image correction unit 126 corrects the difference image according to the display contour shown on the left side of FIG. 11B. On the other hand, if the pixel value is smaller than the predetermined threshold Pth and the region is identified as the first region, the difference image correction unit 126 corrects the difference image according to the display contour shown on the right side of FIG. 11B. Note that FIG. 11B illustrates a negative value (Pth4<0) as the predetermined threshold Pth. In this embodiment, the differential image correction unit 126 makes the grayscale contours (contour lines serving as legends) for the brightness values ​​on the differential image different between display mode 1 and display mode 2, but other display modes such as a color palette, lighting-flashing pattern, saturation (chromatic colors, achromatic colors), color temperature, and chromaticity (cool colors to warm colors) can be adopted as appropriate.

[0136] By doing this, the image processing device 400 of this embodiment, like the image processing devices of the first to third embodiments, displays osteolytic signals generated in non-bone regions, such as inside the spinal canal, separately from osteolytic signals generated in bone regions to assist users, including radiologists, in interpretation and differentiation.

[0137] The term "processor" used in the above description refers to circuits such as a CPU, a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). Instead of storing a program in a memory unit, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor performs its functions by reading and executing the program embedded in the circuitry.

[0138] The components of each device according to the above-described embodiments are conceptual and functionally independent, and are not necessarily physically configured as shown in the drawings. In other words, the specific form of distribution and integration of each device is not limited to that shown in the drawings, and all or part of each device can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0139] The image processing method described in the above-described embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. This program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by being read from the recording medium by a computer.

[0140] According to at least one of the embodiments described above, unnecessary information contained in the difference image can be suppressed.

[0141] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0142] 10: Image processing system 100: Image processing device 101: Image acquisition unit 102: Identification information acquisition unit 103: Alignment information acquisition unit 104: Differential image generation unit 105: Display control unit 113: Image adjustment section 114: Differential image generation unit 116: Differential image correction unit 120: Communication Network 130: Data Server 140: Instruction part 150: Display section 30: Image processing system 500: Image processing device 501: Differential image acquisition unit 502: Identification information acquisition unit 504: Differential image correction unit

Claims

1. an image acquisition means for acquiring a first image and a second image of a subject at different times; an identification information acquisition means for acquiring identification information of a part at a plurality of coordinates of the first image; a difference image generating means for generating a difference image between the first image and the second image when a pixel of interest is identified as a first portion by the identification information, the difference image generating means determining whether pixel values ​​at first coordinates in the first image corresponding to the pixel of interest and second coordinates in the second image corresponding to the first coordinates satisfy a predetermined condition, the difference image being generated by performing a predetermined correction process on an initial difference value representing a difference between pixel values ​​at the first coordinates and the second coordinates, the corrected difference value being set as the pixel value of the pixel of interest; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein said difference image generating means determines whether or not pixel values ​​at said first coordinates and said second coordinates satisfy said predetermined condition based on said initial difference value.

3. The image processing device according to claim 2 , wherein the predetermined condition is whether the initial difference value is equal to or less than a first threshold value of 0 or near 0, or whether the initial difference value is less than a first threshold value of 0 or near 0.

4. The image processing device according to claim 1 , wherein the predetermined correction process is a process in which, when pixel values ​​at the first coordinates and the second coordinates satisfy the predetermined condition, a value of 0 or a value close to 0 is set as the correction difference value.

5. The image processing device according to claim 1 , wherein the predetermined correction process is a process in which, when pixel values ​​of the first coordinates and the second coordinates satisfy the predetermined condition, a value obtained by attenuating the initial difference value is set as the corrected difference value.

6. 2. The image processing device according to claim 1, wherein the predetermined correction process is a process in which, when pixel values ​​between the first coordinates and the second coordinates do not satisfy the predetermined condition, an enhanced value of the initial difference value is set as the corrected difference value.

7. The image processing device according to claim 1 , wherein the first region is at least one of a spinal canal and a region surrounding a vertebra.

8. 2. The image processing device according to claim 1, wherein said differential image generating means does not perform said predetermined correction processing when said target pixel is identified as a second portion based on said identification information.

9. The image processing device according to claim 8 , wherein the second region is a bone region.

10. 2. The image processing device according to claim 1, wherein said difference image generating means does not perform said predetermined correction processing when said target pixel is identified as being in a background region based on said identification information.

11. further comprising a display control means for displaying the difference image on a display unit; 2. The image processing apparatus according to claim 1, wherein said display control means displays said difference image so that an area of ​​said difference image having said corrected difference value as a pixel value can be identified.

12. The image processing device according to claim 11 , wherein the display control means further displays the difference image in such a way that the type of correction processing executed when the correction difference value was obtained can be identified.

13. 4. The image processing device according to claim 3, wherein the difference image generating means extracts a region of interest by thresholding the initial difference value or the corrected difference value with a second threshold value greater than the first threshold value for a plurality of pixels of the difference image identified as the first part by the identification information.

14. a difference image acquisition means for acquiring a difference image between a first image and a second image of the subject captured at different times; an identification information acquisition means for acquiring identification information of a region in a pixel of interest of the difference image; a difference image correcting means for determining whether a pixel value of the target pixel of the difference image satisfies a predetermined condition when the target pixel of the difference image is identified as a first portion based on the identification information, and generating a corrected difference image by performing a predetermined correction process on the pixel value in accordance with the determination result; An image processing device comprising:

15. acquiring a first image and a second image of the subject at different times; acquiring identification information of parts at a plurality of coordinates of the first image; When a pixel of interest is identified as a first portion by the identification information, a difference image between the first image and the second image is generated so that a pixel value of the pixel of interest is set to a corrected difference value obtained by performing a predetermined correction process on an initial difference value representing a difference between pixel values ​​of the first coordinates and the second coordinates, the correction process being performed according to a result of a determination as to whether or not pixel values ​​of a first coordinate in the first image corresponding to the pixel of interest and a second coordinate in the second image corresponding to the first coordinates satisfy a predetermined condition. An image processing method comprising:

16. obtaining a difference image between a first image and a second image obtained by capturing the subject at different times; acquiring identification information of a part at a pixel of interest in the difference image; When the target pixel of the difference image is identified as a first portion based on the identification information, it is determined whether or not the pixel value of the target pixel satisfies a predetermined condition, and a corrected difference image is generated by performing a predetermined correction process on the pixel value according to the determination result. An image processing method comprising:

17. A program for causing a computer to execute the image processing method according to claim 15 or 16.

18. an image acquisition unit that acquires a first image and a second image obtained by imaging the same subject at different times; an image adjustment unit that adjusts at least one of the first image and the second image so as to correspond to the other of the first image and the second image; a difference image generating unit that calculates a difference value between the other image and one of the adjusted images, the image captured later in time relative to the image captured earlier in time, to generate a difference image; an identification information acquisition unit that extracts an anatomical region from at least one of the later image and the subtraction image and acquires identification information of the region; a differential image correction unit that corrects the differential image based on a correction method corresponding to the identification information, The differential image correction unit is an image processing device that performs correction depending on whether the identification information of a pixel of interest in the differential image is a first portion or not, and whether the pixel value of the differential image is smaller than a predetermined threshold or not.

19. The differential image correction unit When the pixel of interest is identified as the first portion, 19. The image processing device according to claim 18, wherein at least one of the pixel values ​​of pixels whose pixel values ​​are smaller than the predetermined threshold and the pixel values ​​of pixels whose pixel values ​​are larger than the predetermined threshold are corrected so that the pixels whose pixel values ​​are smaller than the predetermined threshold are relatively less visible than the pixels whose pixel values ​​are larger than the predetermined threshold.

20. The differential image correction unit An image processing device as described in claim 18 or 19, wherein when the target pixel is identified as the first part and the pixel value is smaller than the predetermined threshold, a differential attenuation process is performed in which the pixel value is multiplied by a predetermined coefficient greater than or equal to 0 and less than 1.

21. The differential image correction unit The image processing device according to claim 20 , wherein the differential attenuation process is not performed on the pixel value when the pixel of interest is identified as not being the first portion or when the pixel value is greater than or equal to the predetermined threshold value.

22. The differential image correction unit The pixel whose pixel value is smaller than the predetermined threshold value and which is identified as the first portion is The image processing device according to claim 18 , wherein the differential image is corrected so that the pixel values ​​identified as not being the first portion are displayed in a different manner from pixels whose pixel values ​​are smaller than the predetermined threshold value.

23. 20. The image processing device according to claim 18, wherein the predetermined threshold value is 0.

24. The image processing device according to claim 18 or 19, wherein the identification information acquisition unit identifies the anatomical region so that the first region corresponds to a region other than bone.

25. The image processing device according to claim 18 or 19, wherein the identification information acquisition unit identifies the anatomical region so that the first region corresponds to a spinal canal region or an extraosseous region.

26. The image processing device according to claim 18 , wherein the image adjustment unit performs adjustment including at least one of transformation processing, rotation processing, alignment processing, resolution conversion processing, tone adjustment processing, masking processing, trimming processing, and angle of view widening processing.

27. an image acquisition step of acquiring a first image and a second image of the same subject at different times; an image adjusting step of adjusting at least one of the first image and the second image so as to correspond to the other of the first image and the second image; a difference image generating step of calculating a difference value of an image captured later in time between the other image and one of the adjusted images and generating a difference image from an image captured earlier in time; an identification information acquisition step of extracting an anatomical site from at least one of the later image and the subtraction image and acquiring identification information of the site; a differential image correcting step of correcting the differential image based on a correction method according to the identification information, The differential image correction process is an image processing method that performs correction depending on whether the identification information of the pixel of interest in the differential image is a first portion or not, and whether the pixel value of the differential image is smaller than a predetermined threshold or not.

28. A program that causes a computer to execute each step of the image processing method according to claim 27.

Images