Image processing device, image processing method and program

Abstract

To provide an image processing device, etc. capable of superimposing anatomic information and the like on the segment of the lung as an object site on a dynamic image.SOLUTION: An analyzer 3 includes: a first acquisition unit for acquiring a dynamic image consisting of a plurality of frame images by dynamically imaging an object site of a subject by a radiographic apparatus; a second acquisition unit for acquiring a functional information image including volume data consisting of a plurality of voxels by imaging the object site by another modality; a third acquisition unit for acquiring anatomic information, etc. on the object site of the functional information image; a generation unit for generating a DRR image on the basis of the functional information image acquired by the second acquisition unit and the anatomic information, etc. acquired by the third acquisition unit; an integration unit for integrating the plurality of frames of the dynamic image acquired by the first acquisition unit and the DRR image generated by the generation unit; and an output unit for outputting an integrated image that has been integrated. The generation unit adds depth information indicating a distance from a reference position of the anatomic information, etc. in the volume data to the DRR image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Radiography devices acquire dynamic images of a target area of ​​a subject by dynamic imaging, which involves continuously irradiating the target area with pulsed X-rays. Dynamic images allow observation of, for example, lung field movement due to breathing and heartbeat, as well as changes in brightness on the image. However, dynamic images make it difficult to observe anatomical information about the lung field, such as bronchi, blood vessels, and pulmonary lobes.

[0003] In recent years, dynamic imaging has been used to estimate the presence and extent of pleural adhesions in lung resection surgery, allowing for advance prediction of the difficulty and duration of the surgery. However, dynamic imaging cannot detect lung lobulation insufficiency and does not provide sufficient information necessary for lung resection surgery, making it difficult to accurately predict the difficulty of the surgery. CT and MRI devices can visualize anatomical information such as lung lobes, bronchi, and blood vessels. Therefore, when planning lung resections, it is currently necessary to separately check dynamic images taken by a radiographic device and the analysis results of images obtained by a CT device.

[0004] Patent Document 1 proposes an image processing device that allows simultaneous viewing of dynamic images captured by a radiographic device and images acquired by other modalities. The image processing device integrates the lung field region of the dynamic image captured by the radiographic device with the lung field region of the functional information image acquired by a SPECT device, and displays the integrated image on a display unit. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-183493 Summary of the Invention [Problem to be solved by the invention]

[0006] Although conventional technology allows for the superimposition of DRR images generated by a CT scanner onto dynamic images, conventional image processing devices have the problem that they are unable to superimpose anatomical information such as lobulation of the target area, or even the depth of the anatomical information, onto each frame of the dynamic image.

[0007] Therefore, in order to solve the above problem, the present invention aims to provide an image processing device, an image processing method, and a program that are capable of superimposing anatomical information such as the lobes of the lungs of the target area onto dynamic images. [Means for solving the problem]

[0008] The image processing device according to the present invention comprises: a first acquisition unit that acquires a dynamic image consisting of a plurality of frame images by performing dynamic imaging of a target region of a subject using a radiation imaging device; a second acquisition unit that acquires a functional information image including volume data consisting of a plurality of voxels by imaging the target region using a modality different from the radiation imaging device; a third acquisition unit that acquires anatomical information or lesion information regarding the target region of the functional information image; a generation unit that generates a DRR image based on the functional information image acquired by the second acquisition unit and the anatomical information or the lesion information acquired by the third acquisition unit; an integration unit that integrates a plurality of frames of the dynamic image acquired by the first acquisition unit and the DRR image generated by the generation unit; an output unit that outputs an integrated image integrated by the integration unit, The generating unit adds depth information indicating a distance from a reference position of the anatomical information or the lesion information in the volume data to the DRR image.

[0009] The image processing method according to the present invention comprises: a first acquisition step of acquiring a dynamic image consisting of a plurality of frame images by performing dynamic imaging of a target region of a subject using a radiation imaging device; a second acquisition step of acquiring a functional information image including volume data consisting of a plurality of voxels by imaging the target region using a modality different from the radiation imaging device; a third acquisition step of acquiring anatomical information or lesion information regarding the target region of the functional information image; a generating step of generating a DRR image based on the acquired functional information image and the anatomical information or the lesion information; an integration step of integrating the plurality of frames of the acquired dynamic image with the DRR image generated by the generation unit; and outputting the pre-integrated integrated image, In the generating step, depth information indicating a distance from a reference position of the anatomical information or the lesion information in the volume data is added to the DRR image.

[0010] The program according to the present invention comprises: Computer, a first acquisition unit that acquires a dynamic image consisting of a plurality of frame images by performing dynamic imaging of a target region of a subject using a radiation imaging device; a second acquisition unit that acquires a functional information image including volume data consisting of a plurality of voxels by imaging the target region using a modality different from the radiation imaging device; a third acquisition unit that acquires anatomical information or lesion information regarding the target region of the functional information image; a generation unit that generates a DRR image based on the functional information image acquired by the second acquisition unit and the anatomical information or the lesion information acquired by the third acquisition unit; an integration unit that integrates the plurality of frames of the dynamic image acquired by the first acquisition unit and the DRR image generated by the generation unit; an output unit that outputs an integrated image integrated by the integration unit; The generating unit adds depth information indicating a distance from a reference position of the anatomical information or the lesion information in the volume data to the DRR image. [Effects of the Invention]

[0011] According to the present invention, dynamic images are combined with DRR images to which depth information of anatomical information or lesion information has been added, so that depth information such as anatomical information that cannot be seen with dynamic imaging alone can be confirmed at the time of diagnosis. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram showing an example of the configuration of an image display system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of a block diagram of an analysis device according to an embodiment of the present invention. [Figure 3] 10 is a flowchart showing an example of the operation of the analysis device when performing an analysis process for combining a dynamic image and a DRR image according to the present embodiment. [Figure 4] FIG. 10 is a diagram showing an example of the configuration of volume data reconstructed in a coronal section according to this embodiment. [Figure 5] FIG. 10 is a diagram showing an example of the configuration of the tenth coronal slice of the volume data according to this embodiment. [Figure 6] FIG. 10 is a diagram showing a concept of generating a two-dimensional first DRR image and the like from a CT image by a ray casting algorithm according to the present embodiment. [Figure 7] FIG. 2 is a diagram showing an example of the configuration of a first DRR image according to the present embodiment. [Figure 8] FIG. 10 is a diagram showing an example of the configuration of a second DRR image according to the present embodiment. [Figure 9A] FIG. 10 is a diagram showing depth information of the boundaries of lung lobes as viewed from the y direction according to the present embodiment. [Figure 9B] FIG. 10 is a diagram showing depth information of the boundaries of lung lobes as viewed in the z direction according to the present embodiment. [Figure 10]FIG. 10 is a diagram showing an example of the configuration of a second DRR image in which the boundaries of lung lobes are color-coded according to distance according to the present embodiment. [Figure 11] FIG. 10 is a diagram showing an example of the configuration of a third DRR image according to the present embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of the configuration of an integrated image according to the present embodiment. [Figure 13] FIG. 10 is a diagram showing another display example 1 of the third DRR image in the case where only the boundary between the right upper lobe and the right middle lobe is visualized. [Figure 14] FIG. 10 is a diagram showing another display example 2 of the third DRR image in the case where only the boundary between the right middle lobe and the right lower lobe is visualized. [Figure 15] FIG. 10 is a diagram showing another display example 3 of the third DRR image in the case where only the boundary between the left upper lobe and the left lower lobe is visualized. [Figure 16] FIG. 4 is a diagram showing another display example 4 of the third DRR image in which the boundary between the right upper lobe and the right middle lobe, the boundary between the right middle lobe and the right lower lobe, and the boundary between the left upper lobe and the left lower lobe are visualized in the middle six sections out of 16 coronal sections. [Figure 17] FIG. 10 is a diagram showing another display example 5 of the third DRR image in which only the boundary between the right upper lobe and the right middle lobe is visualized in the middle six cross sections among the 16 coronal cross sections. [Figure 18] FIG. 10 is a diagram showing another display example 6 of the third DRR image in which only the boundary between the right middle lobe and the right lower lobe is visualized in the middle six cross sections among the 16 coronal cross sections. [Figure 19] FIG. 10 is a diagram showing another display example 7 of the third DRR image in which only the boundary between the left upper lobe and the left lower lobe is visualized in the middle six cross sections among the 16 coronal cross sections. DETAILED DESCRIPTION OF THE INVENTION

[0013] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0014] [Configuration example of image processing system 100] 1 shows an example of the configuration of an image processing system 100 according to this embodiment. The image processing system 100 includes a dynamic imaging device 1, a modality 2, an analysis device 3, and a PACS 4. The dynamic imaging device 1, the modality 2, the analysis device 3, and the PACS 4 are communicably connected to one another via a network N. Examples of the network N include a LAN, a WAN, and the Internet. LAN is an abbreviation for Local Area Network. WAN is an abbreviation for Wide Area Network.

[0015] The dynamic imaging device 1 is an example of a radiographic device and includes, for example, a radiation source, an exposure switch, an X-ray detector, etc. The dynamic imaging device 1 performs dynamic imaging by pressing the exposure switch to irradiate the imaging area of ​​the subject with X-rays from the radiation source. The X-ray detector is, for example, an FPD (Flat Panel Detector), and detects X-rays and the like that have passed through the subject. The dynamic imaging device 1 generates dynamic images of the subject's lungs, etc., based on the output from the X-ray detector.

[0016] In this embodiment, dynamic imaging refers to obtaining a series of images of a subject by repeatedly irradiating the subject with pulsed radiation, such as X-rays, at predetermined time intervals in response to a single imaging operation. Repeated irradiation of pulsed radiation at predetermined time intervals is called pulse irradiation. Alternatively, dynamic imaging refers to obtaining a series of images of a subject by continuously irradiating the subject with a low dose rate without interruption in response to a single imaging operation. Continuous irradiation without interruption is called continuous irradiation. A series of images obtained by dynamic imaging is called a dynamic image. Furthermore, each of all images that make up a dynamic image is called a frame image. Here, dynamic imaging includes video recording, but does not include capturing still images while displaying the video. Furthermore, dynamic images include video, but do not include images obtained by capturing still images while displaying the video.

[0017] The modality 2 is, for example, a CT device, an MRI device, or an ultrasound diagnostic device. In this embodiment, a case where a CT device is used as the modality 2 will be described, and will be referred to as the CT device 2. The CT device 2 irradiates X-rays from an X-ray tube onto a subject and detects the irradiated X-rays with an X-ray detector. The CT device 2 generates a CT image of the imaging region of the subject based on the output from the X-ray detector. The CT image is an example of a functional information image, and is composed of, for example, volume data VD (see FIG. 4). The volume data VD is three-dimensional shape data and includes a plurality of voxels obtained by dividing the interior of a rectangle into a plurality of unit areas in a grid pattern.

[0018] The analysis device 3 is an example of an image processing device, and acquires dynamic images of the imaging region of the subject imaged by the dynamic imaging device 1 and CT images of the imaging region of the subject imaged by the CT device 2. The dynamic images and CT images may be acquired from a PACS 4. The analysis device 3 acquires depth information of anatomical information and lesion information of the imaging region from the CT images, and adds the acquired depth information to a DRR image based on the CT images. The analysis device 3 integrates the DRR image with each frame image of the dynamic image, and outputs the integrated image to a PACS 4 or the like. DRR is an abbreviation for Digitally Reconstructed Radiograph.

[0019] The PACS (Picture Archiving and Communication System) 4 is a medical image management system. The PACS 4 stores and manages dynamic images captured by the dynamic imaging device 1, CT images captured by the CT device 2, and integrated images that have undergone predetermined processing by the analysis device 3. Based on instructions from users such as doctors and radiologists, the PACS 4 displays the dynamic images, CT images, integrated images, etc. on a display unit and outputs them to an information terminal, viewer, etc.

[0020] [Configuration example of analysis device 3] Next, the configuration of the analysis device 3 according to this embodiment will be described. Fig. 2 shows an example of a block diagram of the analysis device 3 according to this embodiment.

[0021] The analysis device 3 includes a control unit 30, a memory unit 31, an operation unit 32, a display unit 33, and a communication unit 34. The control unit 30, the memory unit 31, the operation unit 32, the display unit 33, and the communication unit 34 are connected to each other via wiring such as a bus 35.

[0022] The control unit 30 includes, for example, a processor such as a CPU that performs calculations and control, and a memory. CPU is an abbreviation for Central Processing Unit. The control unit 30 executes a program 31a (described later) stored in, for example, a memory such as a RAM, the storage unit 31, or the like, to perform processing such as adding depth information and the like of the anatomical information of the CT image to the DRR image of the CT image. The control unit 30 may include electronic circuits such as an ASIC and an FPGA. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0023] In this embodiment, the control unit 30 functions as a first acquisition unit, a second acquisition unit, a third acquisition unit, a generation unit, an integration unit, and an output unit. The processor of the control unit 30 executes a program stored in the storage unit 31, etc., thereby realizing the functions of the first acquisition unit, the second acquisition unit, the third acquisition unit, the generation unit, the integration unit, the output unit, etc.

[0024] The first acquisition unit acquires a dynamic image consisting of multiple frame images by dynamically imaging the imaging region of the subject using a dynamic imaging device 1. The second acquisition unit acquires a CT image including volume data VD (see Figure 4) consisting of multiple voxels by imaging the imaging region using a CT device 2 different from the dynamic imaging device 1. The third acquisition unit acquires anatomical information or lesion information regarding the imaging region of the CT image. Examples of anatomical information include lung lobes, blood vessels, bronchi, etc. Examples of lesion information include nodules, tumors, vascular stenosis, calcification, aortic aneurysms, etc.

[0025] The generator generates a third DRR image (see FIG. 11 ) based on the CT image acquired by the second acquirer and the anatomical information or lesion information acquired by the third acquirer. In this case, the generator adds depth information indicating the distance from a reference position of the anatomical information or lesion information in the volume data VD to the DRR image. The integrator integrates multiple frames of the dynamic image acquired by the first acquirer with the third DRR image generated by the generator. The outputter outputs the integrated image integrated by the integrator to the display unit 33 or the like.

[0026] The storage unit 31 includes any storage module, such as an HDD, SSD, ROM, and RAM. HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive. ROM is an abbreviation for Read Only Memory. The storage unit 31 stores, for example, system programs, application programs, and various data. Specifically, the storage unit 31 stores a program 31a for executing a process of adding depth information of anatomical information of a CT image to a DRR image of the CT image.

[0027] The operation unit 32 includes, for example, a mouse, a keyboard, switches, buttons, etc. The operation unit 32 may be, for example, a touch panel integrally combined with a display, or an interface that accepts voice input. The operation unit 32 accepts instructions corresponding to various input operations from the user, converts the accepted instructions into operation signals, and outputs the signals to the control unit 30. Specifically, the operation unit 32 accepts instructions such as the selection of a dynamic image or DRR image to be used when combining a DRR image with a dynamic image.

[0028] The display unit 33 is, for example, a display such as a liquid crystal display or an organic EL display. EL is an abbreviation for Electro Luminescence. The display unit 33 displays an image that has undergone predetermined analysis processing, a GUI for accepting various input operations from the user, etc. GUI is an abbreviation for Graphical User Interface. Specifically, the display unit 33 displays a combined image in which a DRR image to which depth information and other anatomical information of a CT image has been added is combined with each frame image of a dynamic image, etc.

[0029] The communication unit 34 includes, for example, a communication module including a NIC, a receiver, and a transmitter. NIC is an abbreviation for Network Interface Card. The communication unit 34 communicates various information and image data with the dynamic imaging apparatus 1, the modality 2, and the PACS 4 via the network N.

[0030] [Example of operation of analyzer 3] Next, a flow of executing the analysis method according to this embodiment will be described. Fig. 3 is a flowchart showing an example of the operation of the analysis device 3 when executing the analysis process of combining a dynamic image and a DRR image according to this embodiment. The following describes the case where the imaging site is the lung.

[0031] The control unit 30 acquires a dynamic image including multiple frames of the lungs to be processed (step S1). Step S1 corresponds to a first acquisition step. For example, the control unit 30 may display an image search screen or the like on the display unit 33 and acquire a predetermined dynamic image from the image search screen or the like through an input operation on the operation unit 32. The dynamic image may be acquired from the PACS 4 or the memory unit 31.

[0032] The control unit 30 identifies a lung field region from each frame image of the acquired dynamic image (step S2). The method for identifying the lung field region is not particularly limited, and known methods can be applied. For example, the identification method disclosed in Japanese Patent No. 2987633 can be applied. Specifically, the lung field region is identified by utilizing the fact that the image density of the left and right lung portions of the lung field region in an X-ray image is higher than that of the surrounding area. The control unit 30 creates a density histogram of an arbitrary frame image. Next, the control unit 30 determines an image portion of a high-density region corresponding to the lung field region based on the shape and area of ​​the created density histogram, and identifies the image portion as the lung field. Alternatively, the identification method disclosed in Japanese Patent Laid-Open No. 2003-6661 can be applied. Specifically, the control unit 30 identifies the lung field region by performing template matching on an arbitrary frame image using a template that defines the outline of a standard lung field region.

[0033] The control unit 30 acquires a CT image including volume data VD of the lungs to be processed (step S3). Step S2 corresponds to a second acquisition step. For example, the control unit 30 may display an image search screen or the like on the display unit 33, and acquire a predetermined CT image from the image search screen or the like through an input operation on the operation unit 32. The CT image may be acquired from the PACS 4 or the storage unit 31. The control unit 30 extracts a plurality of coronal (coronal) slices I from the volume data VD of the acquired CT image. n As a reconstruction method, a known technique can be applied.

[0034] FIG. 4 is a coronal section I according to this embodiment. n (n is an integer) is used to represent an example of the configuration of the volume data VD reconstructed from coronal slice I. n The plane directions of the x and z directions are defined as the x and z directions. n The direction in which the volumes VD are stacked is the y direction. The front side of the volume data VD in the y direction is the patient's ventral side (A side), and the back side of the volume data VD in the y direction is the patient's dorsal side (P side). In this embodiment, the coronal slice I reconstructed from the volume data VD is nThe coronal section I is composed of 160 images. Furthermore, the 160 coronal sections I are divided into 16 sections, and the first coronal section of the 10 sections in each section is treated as the representative for that section. Therefore, the volume data VD in FIG. 4 is composed of coronal sections n=1, 11, 21, 31, 41, 51, 61, 71, 81, 91, 101, 111, 121, 131, 141, and 151. In addition to using one of the 10 coronal sections in each section as the representative, a coronal section obtained by stacking 10 sections and calculating the average of each pixel across the 10 sections may also be used as the representative. The volume data VD includes voxels of the lungs, etc., divided into five lobes.

[0035] The control unit 30 acquires the boundaries of the lung lobes as anatomical information from the volume data VD of the acquired lung CT image (step S4). Step S4 corresponds to the third acquisition step. As a method for acquiring the boundaries of the lung lobes, for example, the technology disclosed in Japanese Patent Application Laid-Open No. 2008-142481 can be applied. The control unit 30 emphasizes the interlobar fissures running between the lung lobes from the voxels of the volume data VD, which is the CT image, and extracts the emphasized interlobar fissures from the CT image. Next, the control unit 30 divides the lung into lobe units using the extracted interlobar fissures as boundaries. Figure 5 shows the tenth coronal section I of the volume data VD according to this embodiment. 10 The right lung has a boundary BR1. n and Border BR2 n The left lung is divided into three lobes: the right upper lobe (RU), the right middle lobe (RM), and the right lower lobe (RL). n The control unit 30 divides the right lung into two lobes, the left upper lobe LU and the left lower lobe LL, by the boundary BR1. n and Border BR2 n The anatomical information of the left lung is obtained by dividing the boundary BL n The divided lung lobes may be further divided into lung segments.

[0036] Furthermore, when blood vessels are acquired as anatomical information, the technology described in "Yamamoto h, et al. Technical Report of IEICE, JAMIT pp.169-pp.175 (2005)" may be applied. When bronchi are acquired as anatomical information, the technology described in "Study on bronchial extraction from chest CT images using machine learning and graph cuts, Institute of Electrical, Information and Communication Engineers, IEICE Techniques MI2012-98 (2013-01)" may be applied. When acquiring nodules or tumors as lesion information, the technology described in the following document may be applied: "Detection of Imaging Findings of Pulmonary Nodules in Chest CT Images Using Deep Learning and Distinguishing Benign from Malignant, MEDICAL IMAGING TECHNOLOGY Vol. 37 No. 5 November 2019." When acquiring vascular stenosis or aortic aneurysms as lesion information, a method for determining stenosis or aneurysms using the vascular diameter in a cross section perpendicular to the center line of the extracted blood vessel may be applied. When acquiring calcification as lesion information, a method for determining calcification based on the CT value in the vascular region may be applied.

[0037] The control unit 30 generates a first DRR image Ia of the volume data VD based on the DRR generation conditions, and also acquires depth information at the acquired boundaries of the lung lobes (step S5). First, the control unit 30 generates a first DRR image Ia of the volume data VD of the CT image based on the DRR generation conditions. Step S5 corresponds to a generation step. Examples of the DRR generation conditions include the position of the X-ray source in the coordinate space of the volume data VD, six-axis rotation conditions for the volume data VD, adjustment of X-ray absorption and scattering modeling, and contrast adjustment of the DRR image.

[0038] FIG. 6 is a conceptual diagram illustrating the generation of a two-dimensional first DRR image Ia or the like from a CT image using a ray casting algorithm according to this embodiment. FIG. 7 is a diagram illustrating an example of the configuration of the first DRR image Ia. The control unit 30 generates the DRR image to be processed by, for example, executing a ray casting algorithm. In ray casting, assuming that the starting point is the X-ray source S, a radial ray R passing through the X-ray source S and the volume data VD is assumed. The control unit 30 sums (integrates) the brightness values ​​of the voxels at the sampling points on the assumed ray R and reflects the final total brightness value in the pixel value P to generate the first DRR image Ia.

[0039] Next, the control unit 30 calculates each coronal cross section I from the volume data VD using a ray casting algorithm. n Specifically, the control unit 30 generates a second DRR image Ib that visualizes the boundaries of the lung lobes in two dimensions. n In this example, voxels located on the boundary of the lung lobes on the ray R from the X-ray source S are detected. FIG. 8 is a diagram showing an example of the configuration of the second DRR image Ib according to this embodiment. The second DRR image Ib includes voxels located on the boundary of the lung lobes on the ray R from the X-ray source S. n The boundary BL between the left upper lobe LU and the left lower lobe LL of the left lung n The voxels of each coronal section I are expressed as pixel values. n The boundary BR1 between the right upper lobe RU and the right middle lobe RM of the right lung n and the boundary BR2 between the right middle lobe RM and the right lower lobe RL of the right lung. n At this stage, no depth information is added to the second DRR image Ib.

[0040] The control unit 30 controls the coronal section I n Based on the coronal section I1, which is the lung field position closest to the X-ray source, each coronal section I1 containing the voxels at the boundary of the lung lobes n Distance D to nHereinafter, the reference coronal section I1 will be referred to as the reference coronal section I1. As an example, the reference coronal section I1 and the 10th coronal section I 10 Distance D between 10 9A is a diagram showing depth information of the boundaries of lung lobes as viewed from the y direction, and FIG. 9B is a diagram showing depth information of the boundaries of lung lobes as viewed from the z direction. As shown in FIGS. 9A and 9B, the control unit 30 calculates the depth information of the boundaries of the tenth coronal section I1 based on the reference coronal section I1. 10 Distance D in the y direction between 10 The distance D can be calculated, for example, by using the coronal section I n The number of images may be used, or the number of images may be added to the coronal section I n The distance obtained by multiplying the slice thickness (mm / image) of the reference coronal section I may be used. n is not limited to the coronal section I1, but also to other coronal sections I n etc. may also be used.

[0041] Next, the control unit 30 calculates the calculated depth information, which is the distance D n Based on each coronal section I n Specifically, the control unit 30 generates a second DRR image Ib that visualizes the boundaries of the lung lobes in each of the acquired coronal slices I. n The distance D between the reference coronal section I1 and n For example, the control unit 30 normalizes each of the acquired coronal slices I n The distance D between the reference coronal section I1 and n is scaled between the minimum value "0" and the maximum value "1". The control unit 30 scales each coronal slice I n Distance D n The normalized value D for each nn Get.

[0042] In this embodiment, the normalized value D nn The minimum value "0" is white and the maximum value "1" is black. nFor example, the control unit 30 visualizes the depth of the boundary between the lobes of the right lung by using a predetermined boundary BR1 between the right upper lobe RU and the right middle lobe RM of the right lung. n The boundary color (R BR1 ,G BR1 ,B BR1 ) and each coronal section I n Distance D n The normalized value D for each nn and based on boundary BR1 n Each coronal section in I n Distance D n Specifically, the color of each boundary BR1 is calculated using the following formula: n Each coronal section in I n Distance D n You can find the color of each. (R BR1 ,G BR1 ,B BR1 )=(R BR1 ×(1-D nn ),G BR1 ×(1-D nn ),B BR1 ×(1-D nn ))

[0043] Similarly, the boundary BR2 between the right middle lobe RM and the right lower lobe RL of the right lung n So, the preset boundary BR2 n The boundary color (R BR2 ,G BR2 ,B BR2 ) and each coronal section I n Distance D n The normalized value D for each nn and based on the boundary BR2 n Each coronal section in I n Distance D n The boundary BL between the left upper lobe LU and the left lower lobe LL of the left lung n So, the preset boundary BL n The boundary color (R BL ,G BL ,B BL ) and each coronal section I n Distance D n The normalized value D for each nnand based on the boundary BL n Each coronal section in I n Distance D n In this way, the color of each lobe of the lung in the second DRR image Ib can be calculated by dividing the lobes of the lung into the color of each coronal section I. n Distance D n Can be color coded accordingly.

[0044] FIG. 10 shows the distance D n 10 is a diagram showing an example of the configuration of a second DRR image Ib in which the boundaries of the lung lobes are colored according to the lobes of the right lung. For example, the boundary BR1 between the right upper lobe RU and the right middle lobe RM of the right lung is shown. n is set to the blue system and the normalized value D nn The boundary BR2 between the right middle lobe RM and the right lower lobe RL is expressed in a range of white, blue, or black depending on the patient's condition. n is set in yellow and normalized value D nn The boundary BL between the left upper lobe LU and the left lower lobe LL of the left lung is expressed in a range of white to yellow to black depending on the location. n is set in pink and normalized value D nn In this embodiment, the patient's ventral side (A side) is expressed in a lighter color, and the patient's dorsal side (P side) is expressed in a darker color. In addition, in the following Figure 10 etc., the boundaries of the lobes of the right lung and the lobes of the left lung are classified by the direction of hashing, and the distance D of the lobe boundary is determined by the interval of hatching. n The closer the hatching is, the further back the patient is positioned.

[0045] In addition, each coronal section I n The depth of the boundaries of the lung lobes in the second DRR image Ib may be expressed only by shading. n Distance D n In the example above, each coronal section I n Distance D n For example, the depth direction (AP direction) of the volume data VD may be divided into multiple sections, and the color and shade may be changed for each section.

[0046] The control unit 30 generates a third DRR image Ic by superimposing the generated second DRR image Ib shown in FIG. 8 on the generated first DRR image Ia shown in FIG. 7. FIG. 11 shows an example of the configuration of the third DRR image Ic according to this embodiment. According to the third DRR image Ic, each coronal slice I n The depth direction of the boundaries of the lung lobes can be expressed in two dimensions.

[0047] The control unit 30 identifies the lung field region from the third DRR image Ic (step S7). For example, the control unit 30 may binarize the third DRR image Ic using a predetermined threshold value and identify the region where the binarized value is equal to or greater than the threshold value as the lung field region. Alternatively, the lung field region may be identified from the third DRR image Ic by employing another known method. Note that the lung field region may be identified from the first DRR image Ia at a timing before the second DRR image Ib is superimposed on the first DRR image Ia.

[0048] The control unit 30 aligns the lung field region identified from the third DRR image Ic etc. with the lung field region of each frame of the dynamic image (step S8). For example, the control unit 30 non-rigidly aligns the lung field region of the third DRR image Ic etc. with the lung field region of each frame of the dynamic image based on landmarks such as the lung field, apex, and base of the lung.

[0049] After aligning the lung field region of the third DRR image Ic etc. with each frame of the dynamic image, the control unit 30 integrates (combines) each boundary of the lung lobes of the second DRR image Ib with each frame image of the dynamic image to generate an integrated image G (step S9). Step S9 corresponds to the integration step. For example, the control unit 30 generates the integrated image by performing alpha blending for each corresponding pixel of each frame image and the third DRR image Ic. Figure 12 shows an example of the configuration of the integrated image G according to this embodiment. In the integrated image G, each frame of the dynamic image is combined with each coronal section I n Distance D n The boundaries of the lung lobes are superimposed and color-coded according to the lung size.

[0050] The control unit 30 outputs the generated integrated image to the display unit 33, thereby displaying the integrated image on the screen (step S10). Step S10 corresponds to an output step. The integrated image may be displayed as a moving image, or may be displayed by arranging frame images side by side, or may be displayed by switching frame images sequentially in response to an operation of the operation unit 32. In this way, in this embodiment, a series of steps in the image integration process is executed.

[0051] As described above, according to this embodiment, a third DRR image Ic, in which the boundaries of lung lobes are visualized, is combined with each frame of a dynamic image. This allows, for example, during a lung resection diagnosis, not only pleural adhesions based on dynamic images but also the degree of lung lobular adhesion (lobulation failure) to be confirmed on the same screen. As a result, it is not necessary to separately confirm both the dynamic image from the dynamic imaging device 1 and the CT image from the CT device 2, thereby improving the efficiency of diagnosis for users such as physicians. While the above-described embodiment describes lung lobes as an example of anatomical information, similar effects can be achieved with other anatomical information, such as blood vessels or bronchi, and lesion information, such as nodules, tumors, vascular stenosis, calcification, or aortic aneurysms.

[0052] According to this embodiment, when the second DRR image Ib is generated from the CT image, each coronal slice I n The distance D from the reference position to each boundary of the lung lobes in n This allows the depth of each boundary of the lung lobes to be grasped in dynamic images, enabling accurate estimation of the degree of adhesion between lung lobes at the time of diagnosis.

[0053] According to this embodiment, the boundary BR1 between the right upper lobe RU and the right middle lobe RM during breathing n , the boundary BR2 between the right middle lobe RM and the right lower lobe RL n , the boundary BL between the left upper lobe LU and the left lower lobe LL nThe amount of movement of each lobe can be quantified, including in the depth direction. Quantification can be done, for example, by converting it into a numerical value. This makes it possible to predict the degree of lobulation insufficiency from the smoothness and magnitude of the movement of the lobe boundaries. Furthermore, by quantifying the difference in the movement of each lobe boundary between past and current examinations, changes in lung condition can be identified.

[0054] Furthermore, it is said that in cases of severe chronic obstructive pulmonary disease, the movement of the upper lung field tends to be small. The small movement of the upper lung field can be confirmed by analyzing dynamic images in the past. However, in the past analysis of dynamic images, it was limited to understanding the upper part of the entire lung field, and it was difficult to grasp the movement in detail by lobe. According to this embodiment, by quantifying the degree of movement between lobes, symptoms can be observed separately for each lobe, enabling accurate analysis of chronic obstructive pulmonary disease.

[0055] [Another example of the 3rd DRR image Ic] In the third DRR image Ic mentioned above, 16 coronal sections I n In the above example, the second DRR image Ib, which visualizes all of the boundaries of the lung lobes, is superimposed on the first DRR image Ia, which is a CT image. However, the present invention is not limited to this.

[0056] (Another display example of the 3rd DRR image Ic) Figure 13 shows the boundary BR1 between the right upper lobe RU and the right middle lobe RM. n 1 shows another display example 1 of the third DRR image Ic in which only the coronal slices I are visualized. n The boundary BR1 between the right upper lobe RU and the right middle lobe RM in n Next, the control unit 30 generates a third DRR image Ic by superimposing the generated second DRR image Ib on the first DRR image Ia of the CT image. According to another display example 1, the boundary BR1 between the right upper lobe RU and the right middle lobe RM is n Since only the lobes are partially displayed, it is possible to observe only the movement of the boundary of the lobes that are the object of observation.

[0057] (Another display example 2 of the 3rd DRR image Ic) Figure 14 shows the boundary BR2 between the right middle lobe RM and the right lower lobe RL. n 10 shows another display example 2 of the third DRR image Ic in which only the coronal slices I are visualized. n The boundary BR2 between the right middle lobe RM and the right lower lobe RL n Next, the control unit 30 generates a third DRR image Ic by superimposing the generated second DRR image Ib on the first DRR image Ia of the CT image. According to another display example 2, the boundary BR2 between the right middle lobe RM and the right lower lobe RL is n Since only the lobes are partially displayed, it is possible to observe only the movement of the boundary of the lobes that are the object of observation.

[0058] (Another display example 3 of the 3rd DRR image Ic) Figure 15 shows the boundary BL between the left upper lobe LU and the left lower lobe LL. n 10 shows another display example 3 of the third DRR image Ic in which only the coronal slices I are visualized. n The boundary BL between the left upper lobe LU and the left lower lobe LL in n Next, the control unit 30 generates a third DRR image Ic by superimposing the generated second DRR image Ib on the first DRR image Ia of the CT image. According to another display example 3, the boundary BL between the left upper lobe LU and the left lower lobe LL is n Since only the lobes are partially displayed, it is possible to observe only the movement of the boundary of the lobes that are the object of observation.

[0059] (Another display example 4 of the 3rd DRR image Ic) Figure 16 shows 16 coronal sections. n The boundary BR1 between the right upper lobe RU and the right middle lobe RM in the middle six slices n , the boundary BR2 between the right middle lobe RM and the right lower lobe RL n , the boundary BL between the left upper lobe LU and the left lower lobe LL n Another display example 4 of the third DRR image Ic when visualizing the above is shown below. In the following, as the six intermediate cross sections, for example, the sixth to eleventh coronal cross sections I 6~11The control unit 30 calculates the boundary BR1 between the right upper lobe RU and the right middle lobe RM in the six intermediate cross sections. n , the boundary BR2 between the right middle lobe RM and the right lower lobe RL n , the boundary BL between the left upper lobe LU and the left lower lobe LL n Then, the control unit 30 generates a second DRR image Ib that visualizes the first DRR image Ia of the CT image, thereby generating a third DRR image Ic. According to another display example 4, the 16 coronal slices I n Only the boundaries of each lobe in the middle six cross sections are displayed partially, allowing for more detailed observation of the movement of the boundaries of each lobe.

[0060] (Another display example 5 of the 3rd DRR image Ic) Figure 17 shows 16 coronal sections. n The boundary BR1 between the right upper lobe RU and the right middle lobe RM in the middle six slices n 10 shows another display example 5 of the third DRR image Ic in which only the right upper lobe RU and the right middle lobe RM are visualized. n Then, the control unit 30 generates a second DRR image Ib that visualizes the first DRR image Ia of the CT image, thereby generating a third DRR image Ic. According to another display example 5, the 16 coronal slices I n Only the boundaries of each lobe in the middle six cross sections are displayed, allowing for more detailed observation of the movement of the boundaries of each lobe.

[0061] (Another display example 6 of the 3rd DRR image Ic) Figure 18 shows 16 coronal sections. n The boundary BR2 between the right middle lobe RM and the right lower lobe RL in the middle 6 cross sections n 10 shows another display example 6 of the third DRR image Ic in which only the right middle lobe RM and the right lower lobe RL are visualized. nThen, the control unit 30 generates a second DRR image Ib that visualizes the first DRR image Ia of the CT image, thereby generating a third DRR image Ic. According to another display example 6, the 16 coronal slices I n Only the boundaries of each lobe in the middle six cross sections are displayed partially, allowing for more detailed observation of the movement of the boundaries of each lobe.

[0062] (Another display example 7 of the 3rd DRR image Ic) Figure 19 shows 16 coronal sections. n The boundary BL between the left upper lobe LU and the left lower lobe LL in the middle six sections n 10 shows another display example 7 of the third DRR image Ic in which only the left upper lobe LU and the left lower lobe LL are visualized. n Then, the control unit 30 generates a second DRR image Ib that visualizes the first DRR image Ia of the CT image, thereby generating a third DRR image Ic. According to another display example 7, the 16 coronal slices I n Only the boundaries of each lobe in the middle six cross sections are displayed partially, allowing for more detailed observation of the movement of the boundaries of each lobe.

[0063] While the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. Furthermore, various modifications and improvements will naturally fall within the technical scope of the present disclosure, provided that they are within the scope of the technical ideas described in the claims of those skilled in the art. [Explanation of symbols]

[0064] 1. Dynamic imaging device (radiography device) 2 Modality, CT equipment 3. Analysis equipment (image processing equipment) 30 Control Unit BR1 n Border between the right upper lobe and the right middle lobe BR2 n Border between the right middle lobe and right lower lobe BLn Border between the left upper lobe and the left lower middle lobe D n distance Ia First DRR image Ib Second DRR image Ic 3rd DRR image (DRR image) I n Coronal section VD Volume Data

Claims

1. a first acquisition unit that acquires a dynamic image consisting of a plurality of frame images by performing dynamic imaging of a target region of a subject using a radiation imaging device; a second acquisition unit that acquires a functional information image including volume data consisting of a plurality of voxels by imaging the target region using a modality different from the radiation imaging device; a third acquisition unit that acquires anatomical information or lesion information related to the target region of the functional information image; a generation unit that generates a DRR image based on the functional information image acquired by the second acquisition unit and the anatomical information or the lesion information acquired by the third acquisition unit; an integration unit that integrates the plurality of frames of the dynamic image acquired by the first acquisition unit and the DRR image generated by the generation unit; an output unit that outputs an integrated image integrated by the integration unit, the generation unit adds depth information indicating a distance from a reference position of the anatomical information or the lesion information in the volume data to the DRR image. Image processing device.

2. The anatomical information is a lung lobe, a blood vessel, or a bronchus. The image processing device according to claim 1 .

3. The lesion information is a nodule, a tumor, a vascular stenosis, a calcification, or an aortic aneurysm. The image processing device according to claim 1 .

4. the generating unit generates a plurality of coronal sections from the volume data, and two-dimensionally visualizes the anatomical information or the lesion information for each of the plurality of coronal sections in the DRR image. The image processing device according to claim 1 .

5. the depth information is a distance between a reference coronal slice serving as the reference position and the coronal slice including the anatomical information or the lesion information; The image processing device according to claim 4 .

6. the generation unit changes a display mode of the anatomical information or the lesion information of each of the coronal cross sections visualized in the DRR image according to the distance of the anatomical information or the lesion information on each of the coronal cross sections. The image processing device according to claim 5 .

7. A color range is set in advance for each of the anatomical information, the generation unit colors the anatomical information or the lesion information based on the color and the distance; The image processing device according to claim 6 .

8. the generation unit generates a first DRR image of the functional information image and a second DRR image of the anatomical information or the lesion information of the functional information image, and then generates the DRR image by superimposing the second DRR image on the first DRR image. The image processing device according to claim 1 .

9. a first acquisition step of acquiring a dynamic image consisting of a plurality of frame images by performing dynamic imaging of a target region of a subject using a radiation imaging device; a second acquisition step of acquiring a functional information image including volume data consisting of a plurality of voxels by imaging the target region using a modality different from the radiation imaging device; a third acquisition step of acquiring anatomical information or lesion information regarding the target region of the functional information image; a generating step of generating a DRR image based on the acquired functional information image and the anatomical information or the lesion information; an integration step of integrating the plurality of frames of the acquired dynamic image with the generated DRR image; and outputting the pre-integrated integrated image, In the generating step, depth information indicating a distance from a reference position of the anatomical information or the lesion information in the volume data is added to the DRR image. Image processing methods.

10. Computer, a first acquisition unit that acquires a dynamic image consisting of a plurality of frame images by performing dynamic imaging of a target region of a subject using a radiation imaging device; a second acquisition unit that acquires a functional information image including volume data consisting of a plurality of voxels by imaging the target region using a modality different from the radiation imaging device; a third acquisition unit that acquires anatomical information or lesion information regarding the target region of the functional information image; a generation unit that generates a DRR image based on the functional information image acquired by the second acquisition unit and the anatomical information or the lesion information acquired by the third acquisition unit; an integration unit that integrates the plurality of frames of the dynamic image acquired by the first acquisition unit and the DRR image generated by the generation unit; an output unit that outputs an integrated image integrated by the integration unit; the generating unit adds depth information indicating a distance from a reference position of the anatomical information or the lesion information in the volume data to the DRR image. program.

Images