Image processing device, imaging system, image processing method, and program

Abstract

To provide an image processing device capable of displaying a radiation intensity image and a substance discrimination image acquired by using a photon counting technology so that they can be compared easily.SOLUTION: An image processing device includes: an acquisition unit for acquiring a radiation intensity image acquired by imaging a subject using a radiation and a substance discrimination image indicating a discriminated substance, which is an image acquired by imaging the subject by counting a photon of the radiation; and a display control unit for displaying the radiation intensity image and the substance discrimination image in a display unit by aligning, switching, or superimposing them.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, an imaging system, an image processing method, and a program. [Background technology]

[0002] Photon-counting type radiation detectors are known as radiation detectors used in radiography systems such as X-ray CT (Computed Tomography) devices. For example, photon-counting type X-ray detectors measure the intensity of X-rays by capturing each incident X-ray as a photon and counting the number of photons. In addition, when converting X-ray photons into electric charges, photon-counting type X-ray detectors generate an amount of electric charge according to the energy of the X-ray photon, so that the energy of each X-ray photon can be measured. For this reason, photon-counting type X-ray detectors can also measure the energy spectrum of X-rays.

[0003] Also, a material decomposition technique is known that utilizes the fact that radiation absorption characteristics differ for each material to discriminate materials contained in a subject using data corresponding to a plurality of energy bands (energy bins). By applying the material decomposition technique to the energy spectrum of X-rays measured using a photon counting type X-ray detector, a material decomposition image showing the discriminated materials of a radiographed subject can be obtained. Patent Document 1 discloses that an image showing the result of material decomposition obtained by photon counting CT is displayed on a display unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-52349 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in a material decomposition image using such a photon counting technique, only a specific material that has been decomposed is shown, so it may be difficult to grasp the tissue structure of other materials around the material, or some areas may not be identified. In addition, it may be desirable to confirm the decomposed material while taking into consideration its positional relationship with the area of ​​interest, or to confirm the material decomposition image while checking a CT image or the like, which is a conventional radiation intensity image that doctors and others are familiar with.

[0006] Therefore, in one embodiment of the present disclosure, an image processing device is provided that can display a radiation intensity image and a material decomposition image obtained using a photon counting technique in a manner that allows easy comparison between them. [Means for solving the problem]

[0007] An image processing device according to one embodiment of the present disclosure includes an acquisition unit that acquires a radiation intensity image obtained by photographing a subject using radiation and a material decomposition image, which is an image obtained by photographing the subject by counting photons of radiation and indicates a decomposed material, and a display control unit that displays the radiation intensity image and the material decomposition image side by side, switched between, or superimposed on a display unit. Effect of the Invention

[0008] According to one embodiment of the present disclosure, a radiation intensity image and a material decomposition image obtained using a photon counting technique can be displayed so as to be easily compared. [Brief description of the drawings]

[0009] [Figure 1] 1 illustrates an example of a configuration of a CT system according to a first embodiment. [Diagram 2] 4 shows an example of data for a plurality of energy bands according to the first embodiment. [Diagram 3] 2 shows a flowchart of a series of processes according to the first embodiment. [Figure 4] 4 shows examples of a radiation intensity image and a material decomposition image according to the first embodiment. [Diagram 5] 3 shows an example of a display screen according to the first embodiment. [Figure 6] 13 shows another example of the display screen according to the first embodiment. [Figure 7] 13 shows another example of the display screen according to the first embodiment. [Figure 8] 13 shows another example of the display screen according to the first embodiment. [Figure 9] 13 illustrates an example of the configuration of a CT system according to a second embodiment. [Figure 10] 13 illustrates an example of a machine learning model according to a second embodiment. [Figure 11] 13 shows a flowchart of a series of processes according to a second embodiment. [Figure 12] 13 shows an example of a display screen according to the second embodiment. [Figure 13] 13 shows an example of a machine learning model according to variant example 5. [Figure 14] 13 shows another example of a machine learning model according to variant example 5. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Exemplary embodiments of the present disclosure will be described in detail below with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments are arbitrary and can be changed according to the configuration of the device to which the present disclosure is applied or various conditions. In addition, the same reference numerals are used in the drawings to indicate elements that are the same or functionally similar.

[0011] In the following embodiment, an imaging system using X-rays as an example of radiation will be described, but the imaging system according to the present disclosure may use other radiation. Here, the term radiation may include, for example, electromagnetic radiation such as X-rays and gamma rays, and particle radiation such as alpha rays, beta rays, particle rays, proton rays, heavy ion rays, and meson rays. In the present application, photons such as X-rays and gamma rays and particles such as beta rays and alpha rays may be collectively referred to as radiation photons. In addition, in the following, an image in which a material is discriminated using data obtained by radiation imaging is referred to as a material-disaggregated image. In contrast, an image that is a fluoroscopic image such as a CT image or a radiation image obtained by a CT system or a radiation imaging device, and that has not been subjected to material decomposition, is referred to as a radiation intensity image. In the following embodiment, a still image will be described as an image to which the present disclosure is applied, but the image to which the present disclosure is applied may be a moving image.

[0012] In the following embodiments, an imaging system using CT will be described as an example of a radiation imaging system, but the imaging system according to the present disclosure is not limited to this. For example, a DR (Digital Radiography) imaging system using an FPD (Flat Panel Detector) or a PET (Positron Emission Tomography) imaging system may be used. A SPECT (Signal Photon Emission Computed Tomography) imaging system may also be used. The above-mentioned radiation imaging system may be used as a radiation diagnostic device.

[0013] Furthermore, in the following embodiments, an imaging system that images a human body as a subject in the medical field etc. will be described. However, the present disclosure can also be applied to an imaging system that images a product or the like as a subject for non-destructive inspection in the industrial field etc.

[0014] Example 1 Hereinafter, a CT system 1 using X-rays and an image processing method according to the first embodiment will be described with reference to Fig. 1 to Fig. 8(b). The CT system 1 according to the present embodiment is a system capable of performing photon-counting CT. In photon-counting CT, a photon-counting radiation detector capable of counting photons of radiation is used to count radiation that has passed through a subject, thereby making it possible to reconstruct a CT image with a high signal-to-noise ratio.

[0015] Fig. 1 is a schematic diagram showing an example of a configuration of a CT system 1 using X-rays according to Example 1. As shown in Fig. 1, the CT system 1 according to this embodiment includes a gantry device 10, a bed device 20, and an image processing device 30.

[0016] 1, the rotation axis of the rotating frame 13 in a non-tilted state or the longitudinal direction of the top board 23 of the bed device 20 is defined as the Z-axis direction. The axial direction that is perpendicular to the Z-axis direction and horizontal to the floor surface is defined as the X-axis direction. The axial direction that is perpendicular to the Z-axis direction and perpendicular to the floor surface is defined as the Y-axis direction. For the sake of explanation, FIG. 1 illustrates the gantry device 10 from multiple directions, and shows a case where the CT system 1 has one gantry device 10.

[0017] The gantry device 10 is provided with an X-ray tube 11, an X-ray detector 12, a rotating frame 13, an X-ray high voltage device 14, a control device 15, a wedge 16, a collimator 17, and a DAS (Data Acquisition System) 18.

[0018] The X-ray tube 11 is a vacuum tube having a cathode (filament) that generates thermoelectrons and an anode (target) that generates X-rays upon impact of the thermoelectrons. The X-ray tube 11 generates X-rays to be irradiated onto the subject S by irradiating thermoelectrons from the cathode to the anode upon application of high voltage from the X-ray high voltage device 14. For example, the X-ray tube 11 may be a rotating anode type X-ray tube that generates X-rays by irradiating a rotating anode with thermoelectrons.

[0019] The rotating frame 13 is an annular frame that supports the X-ray tube 11 and the X-ray detector 12 so that they face each other, and rotates the X-ray tube 11 and the X-ray detector 12 by the control device 15. For example, the rotating frame 13 may be a casting made of aluminum. Note that the rotating frame 13 can further support an X-ray high voltage device 14, a wedge 16, a collimator 17, a DAS 18, and the like, in addition to the X-ray tube 11 and the X-ray detector 12. The rotating frame 13 can further support various components not shown.

[0020] The wedge 16 is a filter for adjusting the amount of X-rays irradiated from the X-ray tube 11. Specifically, the wedge 16 is a filter that transmits and attenuates the X-rays irradiated from the X-ray tube 11 so that the X-rays irradiated from the X-ray tube 11 to the subject S have a predetermined distribution. For example, the wedge 16 is a wedge filter or a bow-tie filter, and may be a filter made of processed aluminum or the like so as to have a predetermined target angle and a predetermined thickness.

[0021] The collimator 17 is a lead plate or the like for narrowing down the irradiation range of the X-rays transmitted through the wedge 16, and a slit is formed by combining a plurality of lead plates or the like. The collimator 17 may also be called an X-ray aperture. Also, while FIG. 1 shows a case where the wedge 16 is disposed between the X-ray tube 11 and the collimator 17, the collimator 17 may be disposed between the X-ray tube 11 and the wedge 16. In this case, the wedge 16 transmits and attenuates the X-rays that are emitted from the X-ray tube 11 and whose irradiation range is limited by the collimator 17.

[0022] X-ray high voltage device 14 includes a high voltage generator having electric circuits such as a transformer and a rectifier, which generates a high voltage to be applied to X-ray tube 11, and an X-ray control device which controls the output voltage according to the X-rays generated by X-ray tube 11. The high voltage generator may be of a transformer type or an inverter type. X-ray high voltage device 14 may be provided on rotating frame 13, or on a fixed frame (not shown).

[0023] The control device 15 has a processing circuit having a CPU (Central Processing Unit) and the like, and a driving mechanism such as a motor and an actuator. The control device 15 controls the operation of the gantry 10 and the bed 20 in response to an input signal from the input unit 308. For example, the control device 15 controls the rotation of the rotating frame 13, the tilt of the gantry 10, the operation of the bed 20 and the tabletop 23, and the like. As an example, the control device 15 rotates the rotating frame 13 around an axis parallel to the X-axis direction according to input inclination angle (tilt angle) information as a control for tilting the gantry 10. The control device 15 may be provided in the gantry 10 or in the image processing device 30.

[0024] The X-ray detector 12 outputs a signal capable of measuring the energy value of an X-ray photon each time an X-ray photon is incident. The X-ray photon is, for example, an X-ray photon irradiated from the X-ray tube 11 and transmitted through the subject S. The X-ray detector 12 has a plurality of detection elements that output one pulse of an electric signal (analog signal) each time an X-ray photon is incident. Therefore, by counting the number of electric signals (pulses) output from each detection element, it is possible to count the number of X-ray photons incident on each detection element. In addition, by performing arithmetic processing on this signal, it is possible to measure the energy value of the X-ray photon that caused the output of the signal.

[0025] The above-mentioned detection element is, for example, a semiconductor detection element such as CdTe (cadmium telluride) or CdZnTe (cadmium zinc telluride) with electrodes arranged thereon. That is, the X-ray detector 12 is a direct conversion type detector that directly converts incident X-ray photons into an electric signal. Note that the X-ray detector 12 is not limited to a direct conversion type detector, and may be an indirect conversion type detector that converts X-ray photons into visible light using a scintillator or the like, and then converts the visible light into an electric signal using a photosensor or the like.

[0026] The X-ray detector 12 is provided with the above-mentioned detection elements and a plurality of ASICs (Application Specific Integrated Circuits) that are connected to the detection elements and count the X-ray photons detected by the detection elements. The ASIC counts the number of X-ray photons incident on the detection elements by discriminating the individual charges output by the detection elements. The ASIC also measures the energy of the counted X-ray photons by performing arithmetic processing based on the magnitude of each charge. Furthermore, the ASIC outputs the counting result of the X-ray photons to the DAS 18 as digital data.

[0027] The DAS 18 generates detection data based on the results of the counting process input from the X-ray detector 12. The detection data is, for example, a sinogram. The sinogram is data in which the results of the counting process of X-rays incident on each detection element at each position of the X-ray tube 11 are arranged. The sinogram is data in which the results of the counting process are arranged in a two-dimensional orthogonal coordinate system with the view direction and channel direction as axes. The DAS 18 generates a sinogram, for example, in units of rows in the slice direction of the X-ray detector 12. The DAS 18 transfers the generated detection data to the image processing device 30. The DAS 18 can be realized by, for example, a processor such as a CPU.

[0028] Here, the result of the counting process is data in which the number of X-ray photons is assigned to each energy bin (energy bins E1 to E4) as shown in Fig. 2. For example, the DAS 18 counts photons (X-ray photons) originating from X-rays irradiated from the X-ray tube 11 and transmitted through the subject S, and discriminates the energy of the counted X-ray photons to obtain the result of the counting process. Note that, although Fig. 2 shows an example of a plurality of energy bins, the number and width of the energy bins to be discriminated are not limited thereto, and may be set according to a desired configuration.

[0029] The data generated by the DAS 18 is transmitted by optical communication from a transmitter having a light emitting diode (LED) provided on the rotating frame 13 to a receiver having a photodiode provided on a non-rotating part of the gantry 10, and then transferred to the image processing device 30. Here, the non-rotating part may be, for example, a fixed frame (not shown) that rotatably supports the rotating frame 13. Note that the method of transmitting data from the rotating frame 13 to the non-rotating part of the gantry 10 is not limited to optical communication, and any non-contact type data transmission method or a contact type data transmission method may be adopted.

[0030] The bed device 20 is a device on which the subject S to be imaged is placed and moved, and is provided with a base 21, a bed driving device 22, a top plate 23, and a support frame 24. The base 21 is a housing that supports the support frame 24 so that it can move in the vertical direction. The bed driving device 22 is a drive mechanism that moves the top plate 23, on which the subject S is placed, in the longitudinal direction of the top plate 23, and includes a motor, an actuator, etc. The top plate 23, which is provided on the upper surface of the support frame 24, is a plate on which the subject S is placed. Note that the bed driving device 22 may move the support frame 24 in the longitudinal direction of the top plate 23 in addition to the top plate 23.

[0031] The image processing device 30 includes an acquisition unit 301, a generation unit 302, an analysis unit 303, a display control unit 304, an imaging control unit 305, and a storage unit 306. The image processing device 30 is communicably connected to a display unit 307, an input unit 308, the gantry device 10, and the bed device 20. In this embodiment, the image processing device 30 and the gantry device 10 are described as separate entities, but the gantry device 10 may include the image processing device 30 or some of the components of the image processing device 30.

[0032] The image processing device 30 may be configured by a computer provided with a processor and a memory. Each component of the image processing device 30 other than the storage unit 306 is functionally configured using, for example, one or more processors such as CPUs and a program read from the storage unit 306. The processor may be, for example, a micro processing unit (MPU), a graphical processing unit (GPU), or a field-programmable gate array (FPGA). Each component of the image processing device 30 other than the storage unit 306 may be configured by an integrated circuit performing a specific function such as an ASIC. The internal configuration of the image processing device 30 may also include a graphic control unit such as a GPU, a communication unit such as a network card, and an input / output control unit such as a keyboard, a display, or a touch panel.

[0033] The acquiring unit 301 can acquire data generated by the DAS 18, and various operations and patient information input by an operator via the input unit 308. The acquiring unit 301 may also acquire data obtained by imaging the subject S, a CT image of the subject S, a material decomposition image of the subject S, and patient information from an image processing device or storage device (not shown) connected to the image processing device 30 via an arbitrary network. The arbitrary network may include, for example, a LAN (Local Area Network), an intranet, the Internet, and the like.

[0034] The generation unit 302 performs preprocessing such as logarithmic conversion processing, offset correction processing, inter-channel sensitivity correction processing, beam hardening correction, etc. on the data output from the DAS 18 to generate projection data. The generation unit 302 also performs reconstruction processing using a filtered back projection method, an iterative reconstruction method, etc. on the generated projection data to generate a CT image. The generation unit 302 stores the reconstructed CT image in the storage unit 306.

[0035] Here, the projection data generated from the counting results obtained by photon counting CT contains information on the energy of X-rays attenuated by passing through the subject S. Therefore, the generating unit 302 can reconstruct a CT image in a specific energy band. The generating unit 302 can reconstruct CT images in each of a plurality of energy bands. Note that a CT image reconstructed without dividing by energy band (all energy bands) corresponds to a radiation intensity image.

[0036] The generating unit 302 can also generate a plurality of color-coded CT images by, for example, assigning a color tone according to the energy band to the CT image of each energy band. Furthermore, the generating unit 302 can also generate an image in which a plurality of CT images color-coded according to the energy band are superimposed.

[0037] Furthermore, the generating unit 302 can generate a material decomposition image that enables identification of a material by using, for example, a K-absorption edge specific to the material. Note that the method of generating a material decomposition image is not limited to the method of using the K-absorption edge, and any known method may be used. As for the material decomposition image, the generating unit 302 can generate a material decomposition image that is color-coded according to the material, or an image in which a plurality of color-coded material decomposition images are superimposed, similar to the CT image that is color-coded according to the above energy band. In addition, the generating unit 302 can generate, for example, a monochromatic X-ray image, a density image, an effective atomic number image, and the like.

[0038] To reconstruct a CT image, projection data for 360° around the subject S is required, and even in the half-scan method, projection data for 180° + fan angle is required. This embodiment can be applied to either reconstruction method. For simplicity of explanation, the following will use a reconstruction method (full-scan reconstruction) that uses projection data for 360° around the subject S for reconstruction.

[0039] Furthermore, the generation unit 302 can convert the generated CT image into a tomographic image of an arbitrary cross section, a three-dimensional image by rendering processing, or the like, by a known method based on an input from an operator via the input unit 308. The generation unit 302 stores the generated CT image, material decomposition image, etc., as well as the converted tomographic image, three-dimensional image, etc., in the storage unit 306.

[0040] The analysis unit 303 performs a desired analysis process using various images generated by the generation unit 302. For example, the analysis unit 303 performs image processing on the CT images, material decomposition images, and the like generated by the generation unit 302, and obtains analysis results such as the size of an abnormal area in the subject S and the density of materials contained in the tissue. The analysis unit 303 may perform the analysis process using projection data before it is imaged. The analysis unit 303 stores the generated analysis results in the storage unit 306.

[0041] The display control unit 304 causes the display unit 307 to display the patient information, various images, analysis results, information related to the various images, and the like stored in the storage unit 306. In particular, the display control unit 304 according to this embodiment causes the display unit 307 to display the CT image, which is a radiation intensity image, and the material decomposition image generated using the photon counting technique in a manner that makes it easy to compare them. For example, the display control unit 304 causes the CT image and the material decomposition image to be displayed side by side, switched between, or superimposed.

[0042] The imaging control unit 305 controls the CT scan performed in the gantry device 10. For example, the imaging control unit 305 controls the operations of the X-ray high voltage device 14, the X-ray detector 12, the control device 15, the DAS 18, and the bed driving device 22, thereby controlling the collection process of the counting results in the gantry device 10. For example, the imaging control unit 305 controls the collection process of the projection data in the imaging for collecting positioning images (scanograms) and in the main imaging (scan) for collecting images used for observation.

[0043] The storage unit 306 is realized by, for example, a random access memory (RAM), a semiconductor memory element such as a flash memory, a hard disk, an optical disk, etc. The storage unit 306 stores, for example, patient information, projection data, various images such as CT images and material decomposition images, analysis results, and information related to the various images. In addition, for example, the storage unit 306 can store a program for realizing the functions of each of the above-mentioned components. Note that the storage unit 306 may be realized by a group of servers (cloud) connected to the CT system 1 via a network.

[0044] The display unit 307 displays various information. For example, the display unit 307 displays various images generated by the generation unit 302, or displays a GUI (Graphical User Interface) for receiving various operations from an operator. The display unit 307 may be any display, such as a liquid crystal display, an organic EL display, or a CRT (Cathode Ray Tube) display. The display unit 307 may be a desktop type, or may be a tablet terminal capable of wireless communication with the image processing device 30 main body.

[0045] The input unit 308 accepts various input operations from the operator, converts the accepted input operations into electrical signals, and outputs the electrical signals to the image processing device 30. In addition, for example, the input unit 308 accepts input operations such as reconstruction conditions when reconstructing a CT image and image processing conditions when generating a post-processed image from the CT image from the operator.

[0046] For example, the input unit 308 is realized by a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad that performs an input operation by touching an operation surface, a touch screen in which a display screen and a touchpad are integrated, or the like. The input unit 308 may be realized by a non-contact input circuit using an optical sensor, a voice input circuit, or the like. The input unit 308 may be provided in the pedestal device 10. The input unit 308 may be configured by a tablet terminal or the like that can wirelessly communicate with the main body of the image processing device 30. The input unit 308 is not limited to only those that have physical operation parts such as a mouse and a keyboard. For example, an example of the input unit 308 includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the image processing device 30 and outputs the electrical signal to the image processing device 30.

[0047] Next, a series of processes including image processing according to this embodiment will be described with reference to Fig. 3 to Fig. 8(b). Fig. 3 is a flowchart of a series of processes according to this embodiment. When the process according to this embodiment is started in response to an instruction from an operator, the process proceeds to step S301.

[0048] In step S301, the acquiring unit 301 acquires data obtained by imaging the subject S using the gantry device 10 based on imaging conditions input by the operator. The acquired data includes counting results obtained by photon counting CT. The acquiring unit 301 may also acquire data obtained by imaging the subject S from an image processing device or storage device (not shown) via an arbitrary network.

[0049] Next, in step S302, the generating unit 302 generates a CT image based on the acquired data. The generating unit 302 also generates a material decomposition image based on the counting result included in the acquired data. As described above, the method of generating the material decomposition image may be a method using the K-absorption edge, or any other known method.

[0050] Here, Fig. 4(a) to Fig. 4(c) show examples of schematic CT images and material decomposition images of one cross section of a subject S. Fig. 4(a) shows a CT image 401. The CT image 401 is a radiation intensity image corresponding to a conventional CT image, and is an image in which pixel values ​​are CT values ​​obtained using data of all energy bands included in the projection data. Fig. 4(b) shows a material decomposition image 402 of iodine obtained using the projection data. Fig. 4(c) shows a material decomposition image 403 of gadolinium obtained using the projection data.

[0051] Note that the material decomposition images are not limited to those related to iodine and gadolinium, and the generating unit 302 may generate material decomposition images decomposed for other materials, such as calcium, bone, soft tissue, etc. In addition, in Fig. 4(b) and Fig. 4(c), hatching of a predetermined pattern is used to make it easy to understand the difference between the material decomposition images, but in reality, a value to which a color tone according to the material is assigned may be used as each pixel value.

[0052] The generating unit 302 can generate cross-sectional images of the three-dimensional CT images and material decomposition images generated using the projection data according to an instruction from an operator or a preset setting. In the following, for the sake of simplicity, the CT images and material decomposition images are described as cross-sectional images. As described above, the generating unit 302 may generate images for each energy band, monochromatic X-ray images, etc. The preset setting may include at least one of a setting predetermined for each imaging condition including an imaging site, etc., and a setting predetermined for each imaging mode corresponding to a disease, etc.

[0053] Next, in step S303, the analysis unit 303 performs an analysis process using the various images generated in step S302. The analysis process may include detection of an abnormal portion and calculation of the density of a predetermined substance. However, the analysis process is not limited to these, and any analysis process required in the medical field or industrial field may be performed according to a desired configuration. The analysis unit 303 may perform the analysis process using projection data before it is imaged. Furthermore, the analysis process may be omitted according to an instruction from an operator or a previous setting.

[0054] In step S304, the display control unit 304 causes the display unit 307 to display the CT image and the material decomposition image in a display mode that allows easy comparison between them. As described above, by using the photon counting technique, a material decomposition image in which a desired material is decomposed can be generated. However, in the material decomposition images 402 and 403 shown in FIG. 4(b) and FIG. 4(c), it is possible to grasp the tissues containing the material to be decomposed, but it is difficult to grasp the surrounding tissues that do not contain the material to be decomposed. Therefore, it may be difficult to grasp the relationship between the respective tissues. In addition, it may be desired to check the material decomposition image while checking a familiar conventional radiation intensity image such as a CT image.

[0055] For this reason, the display control unit 304 according to this embodiment displays the CT image and the material decomposition image side by side, in a switching manner, or in a superimposed manner so that the CT image and the material decomposition image can be easily compared with each other. Here, the display screen that the display control unit 304 according to this embodiment causes the display unit 307 to display will be described with reference to Figs. 5 to 8(b).

[0056] 5 shows a display screen 501 displaying a CT image 540 and a material decomposition image 550 side by side as an example of a display screen according to the present embodiment. The display screen 501 shows an example of a material decomposition image 550 in which iodine has been decomposed. The display screen 501 shows a patient ID 510, a patient's name 520, comments 530 about the patient and the image, a CT image 540, and a material decomposition image 550. In addition, around the CT image 540 and the material decomposition image 550, types of images 541, 551, dates of image capture 542, 552, and analysis results 543, 553 are displayed. In addition, around the material decomposition image 550, a display example 555 of a type of material 554 to be decomposed and a color tone corresponding to the material is displayed.

[0057] The display control unit 304 can read and display information stored in the storage unit 306 regarding the patient ID 510, the name 520, and the comment 530. The patient ID 510, the name 520, and the comment 530 may be stored in the storage unit 306 in association with the CT image 540 and the material decomposition image 550. As the comment 530, for example, the name of the disease, the examination site, the presence or absence of an imaging failure in the image, the reason for the imaging failure, etc. may be displayed.

[0058] Furthermore, the patient ID 510, the name 520, and the comment 530 may be additionally input by the operator via the input unit 308, and the display control unit 304 may store the input information in the storage unit 306 in association with the CT image 540 and the material decomposition image 550. Furthermore, regarding the presence or absence of imaging failure and the reason thereof, a separate selection button or input frame may be provided to accept input by the operator.

[0059] Image types 541 and 551 respectively indicate the types of images of the displayed CT image 540 and material decomposition image 550. In the example shown in Fig. 5, the image type 541 indicates a CT image, and the image type 551 indicates a PCCT image (material decomposition image based on photon counting).

[0060] The image capture dates 542, 552 indicate the dates on which the displayed CT image 540 and material decomposition image 550 were captured. Note that the CT image 540 and material decomposition image 550 may be displayed for comparison with the aim of making it easier to understand the relationship between tissues, for example, and the images do not have to be acquired by the same capture. Therefore, by displaying the image capture dates 542, 552, the operator can understand when each image was captured, and can also understand that differences in tissue structure in the images are due to changes over time, etc.

[0061] The analysis results 543 and 553 indicate the results of the analysis process performed using each image. The analysis results do not have to be displayed for each image, and may be displayed in a single display frame. The analysis results do not have to be values, and for example, an area of ​​an abnormal part may be detected and displayed as a result of the analysis. In this case, for example, in response to an operator's instruction such as turning on / off a button (not shown), the area corresponding to the detected abnormal part may be highlighted in the corresponding image so that the area can be easily grasped.

[0062] The material type 554 indicates a material to be decomposed in the material decomposition image 550, and iodine is shown as an example on the display screen 501. The display control unit 304 can display the material type 554 so that the operator can select a material to be decomposed, for example, can display options selectable by the operator. Furthermore, the display control unit 304 can display, as the material decomposition image 550, a material decomposition image corresponding to a material selected in response to an instruction from the operator.

[0063] The types of materials may include, for example, iodine, gadolinium, calcium, bone, soft tissue, and any metal in the medical field, and may include, for example, solder, silicon, and the like in the industrial field. The types of materials may also include other materials according to a desired configuration. The types of materials may include a plurality of materials, for example, iodine and gadolinium. In this case, an image in which the material decomposition image of iodine and the material decomposition image of gadolinium are superimposed on each other can be displayed as the material decomposition image 550.

[0064] The display example 555 of color tones etc. corresponding to a substance exemplifies color tones etc. corresponding to a substance displayed in the material decomposition image 550. The display example 555 may include color tones, display patterns etc. The display example 555 of color tones etc. corresponding to a substance can be useful for the operator to identify decomposed substances in the material decomposition image 550 or an image on which the material decomposition image 550 is superimposed.

[0065] Since the CT image 540 and the material decomposition image 550 are displayed side by side on the display screen 501, the operator can easily compare these images. Therefore, the operator can more easily grasp the relationship between tissues containing the material to be decomposed and tissues not containing the material to be decomposed, and can perform observation more efficiently, compared to the case where the material decomposition image 550 is observed alone.

[0066] In the example shown in Fig. 5, the CT image 540 and one material decomposition image 550 are displayed side by side. Alternatively, a CT image and a plurality of types of material decomposition images may be displayed side by side. Fig. 6 shows a display screen 601 that displays a CT image 540 and two material decomposition images 550, 660 side by side, as an example of a display screen according to this embodiment. In the example shown in Fig. 6, compared to the example shown in Fig. 5, a material decomposition image 660, a photographing date 662 of the image, an analysis result 663, a type of material 664, and a display example 665 of color tones and the like corresponding to the material are displayed.

[0067] A material decomposition image 660 is an example of a material decomposition image obtained by decomposing gadolinium. An image capture date 662, an analysis result 663, a type of material 664, and a display example 665 of color tones corresponding to the material correspond to the material decomposition image 660. Note that the image capture date 662, the analysis result 663, the type of material 664, and the display example 665 of color tones corresponding to the material may be similar to the image capture date 552, the analysis result 553, the type of material 554, and the display example 555 of color tones corresponding to the material.

[0068] In this display screen 601, the CT image 540 and the material decomposition images 550, 660 are displayed side by side, so the operator can easily compare the CT image and the material decomposition images. In addition, the material decomposition images 550, 660 relating to different types of materials are displayed side by side, so the operator can easily compare tissues containing different materials. This makes it easier for the operator to understand the relationship between tissues containing different materials, and allows for more efficient observation.

[0069] When the CT image 540 and the multiple material decomposition images 550, 660 are displayed side by side, the multiple material decomposition images 550, 660 may be arranged adjacent to each other above and below and / or to the left and right of the CT image 540. For example, the material decomposition image 550 may be displayed on the left side of the CT image 540, and the material decomposition image 660 may be displayed on the right side of the CT image 540. In this case, the operator can more easily compare the material decomposition images 550, 660 with the CT image 540, and can perform efficient observation.

[0070] 5 and 6, a material decomposition image obtained by decomposing one material is displayed as the material decomposition image. Alternatively, a material decomposition image in which multiple material decomposition images obtained by decomposing different types of materials are superimposed may be displayed side by side with a CT image.

[0071] 7(a) and 7(b) show display screens 701 and 702 that switch between displaying a CT image 540 and a material decomposition image 550, as an example of a display screen according to this embodiment. Note that the same reference numerals are used for configurations similar to those in Fig. 5, and detailed explanations are omitted. Also, in Fig. 7(a) and 7(b), for the sake of simplicity of explanation, the image capture dates are omitted, but the image capture dates may be indicated on each display screen.

[0072] In the display screen 701 shown in Fig. 7(a), only the CT image 540 is displayed as a display image, and a switching button 780 is also displayed. When the switching button 780 is operated in response to an instruction from the operator, the display screen 701 is switched to a display screen 702 shown in Fig. 7(b). In the display screen 702, only the material decomposition image 550 is displayed as a display image. Furthermore, when the switching button 780 is further operated in response to an instruction from the operator, the display screen 702 is switched to the display screen 701 shown in Fig. 7(a).

[0073] Therefore, the CT image 540 and the material decomposition image 550 are displayed alternately on the display screens 701 and 702, allowing the operator to easily compare these images. Therefore, the operator can more easily grasp the relationship between tissues containing the target substance and tissues not containing the target substance, and can perform observation more efficiently, compared to the case where the material decomposition image 550 is observed alone.

[0074] In addition, by matching the display positions, sizes, etc. of the CT image 540 and the material decomposition image 550 on the display screen 701 and the display screen 702, the operator can observe these images in a manner that makes it easier for the operator to compare these images. In addition, the material decomposition image displayed in switchover with the CT image is not limited to one, but may be a plurality of material decomposition images obtained by decomposing different types of materials. In this case, by operating the switching button 780, the plurality of material decomposition images can be displayed in a manner that switches in a preset order. In addition, a material decomposition image in which a plurality of material decomposition images obtained by decomposing different types of materials are superimposed on each other may be displayed as the material decomposition image.

[0075] In FIG. 7(a) and FIG. 7(b), the display screen 701 displaying the CT image 540 and the display screen 702 displaying the material decomposition image 550 are switched in response to the operation of the switching button 780. However, the display control unit 304 may, for example, provide a slider (not shown) and cause the display unit 307 to display an image in which the CT image 540 and the material decomposition image 550 are blended (superimposed) with each other at a blend ratio according to the operation of the slider. Here, the blend ratio may be a transparency ratio having a relationship in which increasing the transparency of one of the images to be superimposed with each other reduces the other. In this case, the display control unit 304 may set the blend ratio of the CT image 540 and the material decomposition image 550 in response to an instruction from the operator, and cause the display unit 307 to display the CT image 540 and the material decomposition image 550 superimposed with each other at the set blend ratio. Furthermore, the display control unit 304 may set the transparency of either image according to the operation of the slider, and superimpose the image on the other image to display it on the display unit 307. Even in these cases, the operator can easily compare the CT image 540 and the material decomposition image 550.

[0076] Fig. 7(c) shows a display screen 703 that displays a material decomposition image 550 superimposed on a CT image 540. Note that the same components as those in Fig. 5 are given the same reference numerals and detailed description will be omitted. Also, in Fig. 7(c), for the sake of simplicity, the image capture date is omitted, but the image capture date may be indicated on the display screen.

[0077] 7(c), a superimposed image 770 is displayed in which the material decomposition image 550 is superimposed on the CT image 540. Moreover, the image type 771, the analysis result 773, the material type 774, and the display example 775 of the color tone corresponding to the material correspond to the superimposed image 770. Note that the analysis result 773, the material type 774, and the display example 775 of the color tone corresponding to the material may be similar to the analysis result 553, the material type 554, and the display example 555 of the color tone corresponding to the material. Note that the image type 771 is set to CT+PCCT so as to indicate that the material decomposition image is superimposed on the CT image. Moreover, the analysis result 773 may correspond to the CT image 540 and / or the material decomposition image 550.

[0078] The superimposed image 770 may be a superimposed image in which the CT image 540 is superimposed on the material decomposition image 550. The material decomposition image is not limited to one type of material decomposition image, and may be a material decomposition image in which a plurality of material decomposition images each of which is decomposed for different types of materials are superimposed.

[0079] 7(a) or 7(b) is operated, the display control unit 304 may switch the display screen 701 or 702 to the display screen 703. In this case, the CT image 540 and the material decomposition image 550 are displayed in a superimposed state, so that the operator can easily compare these images. Also, the CT image 540 or the material decomposition image 550, which are not superimposed on each other, and the superimposed image 770 are displayed in a switched state, so that the operator can easily compare these images.

[0080] With regard to the superimposed image 770, a material decomposition image 550 having a transparency set in accordance with an operator's instructions or a predetermined setting may be superimposed on the CT image 540, or a CT image 540 having a similar transparency set may be superimposed on the material decomposition image 550.

[0081] Also, in the example shown in FIG. 5 and FIG. 6 in which a CT image and a material decomposition image are displayed side by side, a switching button 780 may be provided, and the respective images and the superimposed image may be switched for display in response to the operation of the switching button 780. In this case, the switching button 780 may be provided for each CT image and each material decomposition image. For example, when the switching button 780 corresponding to a CT image is operated, the display of the CT image can be switched to the display of a superimposed image in which the material decomposition image is superimposed on the CT image. Similarly, when the switching button 780 corresponding to a material decomposition image is operated, the display of the material decomposition image can be switched to the display of a superimposed image in which the CT image is superimposed on the material decomposition image. In this case as well, the operator can easily compare these images.

[0082] When a CT image and a plurality of material decomposition images are displayed side by side, an option may be provided so that the material decomposition image to be superimposed on the CT image can be selected in response to an instruction from the operator. When a CT image is superimposed on a plurality of material decomposition images, the plurality of material decomposition images can be switched to the superimposed image collectively in response to an operation of the switching button 780. A switching button for switching between the material decomposition image and the superimposed image may be provided for each of the plurality of material decomposition images.

[0083] 8(a) and 8(b) show display screens 801 and 802 for follow-up observation, which switch between displaying a plurality of CT images and a plurality of material decomposition images, as an example of a display screen according to this embodiment. Note that the same reference numerals are used for configurations similar to those in Fig. 5, and detailed explanations are omitted. Also, in Fig. 8(a) and 8(b), analysis results are omitted for the sake of simplicity of explanation, but the analysis results may be shown on each display screen.

[0084] CT images 8401, 8402, 8403, 8404, and 8405 acquired at different times are displayed on a display screen 801 shown in Fig. 8(a). Also displayed on the display screen 801 are image types 841, image capture dates 8421, 8422, 8423, 8424, and 8425 corresponding to the respective images, and a switching button 880. When the switching button 880 is operated in response to an instruction from an operator, the display screen 801 switches to a display screen 802 shown in Fig. 8(b).

[0085] The display screen 802 shows material decomposition images 8501, 8502, 8503, 8504, and 8505 corresponding to CT images 8401, 8402, 8403, 8404, and 8405 acquired at different times. The display screen 802 also shows an image type 851, image capture dates 8521, 8522, 8523, 8524, and 8525 corresponding to each image, a material type 854, a display example 855 of color tones, etc. corresponding to the material, and a switching button 880. The image types 841 and 851, the material type 854, and the display example 855 of color tones, etc. corresponding to the material may be similar to the image types 541 and 551, the material type 554, and the display example 555 of color tones, etc. corresponding to the material in FIG. 5. Furthermore, when the switching button 880 is further operated in response to an instruction from the operator, the display screen 802 is switched to the display screen 801 shown in FIG. 8(a).

[0086] Therefore, on the display screens 801 and 802, a plurality of CT images 8401, 8402, 8403, 8404, and 8405 and a plurality of material decomposition images 8501, 8502, 8503, 8504, and 8505 are displayed in a collectively switched manner. Therefore, the operator can easily compare these images. Note that the switching button 880 may be provided for each image.

[0087] In addition, the CT images 8401, 8402, 8403, 8404, and 8405 may be switched between superimposed images in which the corresponding material decomposition images 8501, 8502, 8503, 8504, and 8505 are superimposed. Similarly, the material decomposition images 8501, 8502, 8503, 8504, and 8505 may be switched between superimposed images in which the corresponding CT images 8401, 8402, 8403, 8404, and 8405 are superimposed. Even in these cases, the operator can easily compare these images.

[0088] Furthermore, as images for follow-up observation, a plurality of CT images acquired at different times and a plurality of corresponding material decomposition images may be displayed side by side. Also, a plurality of CT images or a plurality of material decomposition images acquired at different times and a plurality of superimposed images in which one of them is superimposed on the other may be displayed side by side. Even in these cases, the operator can easily compare these images. In this case, a plurality of types of images may be displayed in a staggered manner such that corresponding CT images and material decomposition images, or corresponding CT images or material decomposition images and their superimposed images, are adjacent to each other. In this case, the operator can observe the CT images and the material decomposition images in a manner that makes it easier to compare them.

[0089] In addition, even when displaying a list of thumbnails of each tomographic image included in a three-dimensional CT image or when displaying tomographic images at consecutive positions in sequence, which is called image flipping, the display control unit 304 may display the CT image and the material decomposition image side by side, by switching between them, or by superimposing them. In this case, the display control unit 304 may display a button or a slider for switching between and displaying the tomographic images successively on the display screen. For example, when displaying the CT image and the material decomposition image side by side, the display control unit 304 may switch and display the CT image and the material decomposition image collectively at an image position corresponding to the operation of the button or slider. In addition, even in the display of the time-dependent difference in which the difference between a plurality of images obtained at different times and a reference image are displayed side by side, the difference between the CT image, the reference image, the difference between the material decomposition image, and the reference image may be displayed side by side, or may be displayed by switching between them, or superimposed images of these may be displayed.

[0090] Also, when multiple CT images obtained at different times are successively switched and displayed, the corresponding material decomposition images may be displayed side by side, switched, or superimposed. In this case, when switching between a CT image and a material decomposition image, the CT image and the material decomposition image can be switched for subsequent consecutive images depending on the timing of the operation of the switching button.

[0091] When the display control process by the display control unit 304 ends in step S304, a series of processes according to this embodiment ends.

[0092] As described above, the image processing device 30 according to this embodiment includes an acquisition unit 301 and a display control unit 304. The acquisition unit 301 acquires a radiation intensity image obtained by photographing a subject using radiation, and a material decomposition image, which is an image obtained by photographing a subject by counting photons of radiation and indicates decomposed materials. The display control unit 304 causes the display unit 307 to display the radiation intensity image and the material decomposition image side by side, in a switched manner, or superimposed on each other.

[0093] With this configuration, the operator can easily compare the CT image and the material decomposition image, which allows the operator to more easily grasp the relationship between tissues that contain the target substance and tissues that do not contain the target substance, and allows for more efficient observation, compared to the case where the operator observes the material decomposition image alone.

[0094] Furthermore, the display control unit 304 can display options for selecting a material to be decomposed around the material decomposition image on the display unit 307. In this case, the operator can easily compare the radiation intensity image and the material decomposition image while appropriately switching between the material decomposition images according to the purpose of the observation, thereby enabling more efficient observation.

[0095] The radiation intensity image and the material decomposition image can be images generated using common data obtained by imaging a subject using radiation. In this embodiment, as described above, projection data is obtained from data obtained by imaging using the gantry device 10, and a CT image and a material decomposition image are generated based on the projection data. In this case, the radiation intensity image and the material decomposition image show tissues having a common shape, etc., so that the radiation intensity image and the material decomposition image can be more easily compared.

[0096] However, the radiation intensity image and the material decomposition image that the display control unit 304 causes to be displayed on the display unit 307 may be displayed for comparison with the aim of making it easier to understand the relationship between tissues, for example, and may not be images generated using common data. Therefore, the radiation intensity image and the material decomposition image that the display control unit 304 causes to be displayed on the display unit 307 may be generated using different data. For example, the radiation intensity image may be generated using data obtained from a CT system that does not use photon counting technology, and the material decomposition image may be generated using data obtained from a CT system that uses photon counting technology.

[0097] Furthermore, the material decomposition image displayed on the display unit 307 may include an image in which a plurality of material decomposition images obtained by decomposing different types of materials are superimposed on each other. In this case, the operator can easily grasp the relationship between tissues containing different types of materials and tissues not containing those materials, and can perform observation more efficiently.

[0098] Furthermore, the material decomposition image displayed on the display unit 307 may include a plurality of material decomposition images obtained by decomposing different types of materials. In this case, since a plurality of material decomposition images relating to a plurality of types of materials are displayed, the operator can easily compare tissues containing different materials, thereby enabling more efficient observation.

[0099] The display control unit 304 can display each of the multiple material decomposition images adjacent to the radiation intensity image on the display unit 307. In this case, the operator can more easily compare each material decomposition image with the radiation intensity image, and can perform efficient observation.

[0100] Furthermore, the display control unit 304 can collectively switch between a plurality of radiation intensity images and a plurality of material decomposition images and display them on the display unit 307. The display control unit 304 can collectively switch between a plurality of radiation intensity images or a plurality of material decomposition images and a plurality of superimposed images of a plurality of radiation intensity images and a plurality of material decomposition images and display them on the display unit 307. In these cases, the operator can observe the plurality of radiation intensity images and the plurality of material decomposition images in a manner that makes it easy to compare them with a simpler operation, and can perform observation more efficiently.

[0101] Furthermore, the display control unit 304 can set the transparency of one of the radiation intensity image and the material decomposition image in response to an instruction from the operator, and display the set of images on the display unit 307 by superimposing the set of images on the other of the radiation intensity image and the material decomposition image. In this case, the operator can check the superimposed image of the radiation intensity image and the material decomposition image superimposed at the desired transparency. Therefore, the operator can observe the superimposed image in a form that is easy for the operator to observe, and can observe more efficiently.

[0102] Furthermore, when switching between a radiation intensity image and a material decomposition image at a different position on the subject, the display control unit 304 can, in response to an instruction from the operator, switch between the radiation intensity image and the material decomposition image collectively and display them on the display unit 307. In this case, the operator can observe the radiation intensity image and the material decomposition image relating to the cross section at a desired position with a simple operation, thereby enabling more efficient observation.

[0103] The image processing device 30 may further include an analysis unit 303 that analyzes at least one of the radiation intensity image and the material decomposition image. In this case, the display control unit 304 may cause the display unit 307 to display the analysis results by the analysis unit 303 around the analyzed image or superimposed on one of the radiation intensity image and the material decomposition image. In this case, the operator can easily compare the analysis results obtained by analyzing the images, in addition to the radiation intensity image and the material decomposition image, and thus can perform observation more efficiently.

[0104] Furthermore, in response to an instruction from the operator, the display control unit 304 can display information regarding imaging defects in at least one of the radiation intensity image and the material decomposition image on the display unit 307. In this case, the operator can easily grasp the presence or absence of imaging defects and the reasons thereof when observing the image, thereby enabling more efficient observation.

[0105] (Variation 1) In the first embodiment, the display screen is described in which the CT image, which is a radiation intensity image, and the material decomposition image are displayed side by side, switched between, or superimposed on each other. In contrast, the display control unit 304 may select one of these display screens and display it on the display unit 307 in response to an instruction from an operator or a setting made in advance.

[0106] In this case, the display control unit 304 selects a display screen to be displayed in accordance with an instruction from the operator or a previous setting, and causes the display unit 307 to display the selected display screen. For example, when the operator instructs the selection of a display screen that displays a CT image and a material decomposition image side by side, the display control unit 304 causes the display unit 307 to display a display screen 501 shown in Fig. 5 in accordance with the instruction.

[0107] Furthermore, in response to an instruction from the operator, the display control unit 304 may switch the display screen to be displayed and display it on the display unit 307. For example, when the operator further instructs to select a display screen that switches between a CT image and a material decomposition image, the display control unit 304 may switch the display screen 501 shown in Fig. 5 to the display screen 701 shown in Fig. 7(a) and display it on the display unit 307 in response to the instruction.

[0108] (Variation 2) The display screen shown in the first embodiment corresponds to an analysis screen capable of displaying detailed analysis results and a display screen for performing image observation. Meanwhile, the image processing according to the first embodiment can be similarly applied to displaying a preview image such as a CT image on a confirmation screen for confirming an image captured immediately after capturing an image of the subject S. Note that the confirmation screen may display a CT image with a short processing time prior to a material decomposition image, and switch to and display the material decomposition image later. In this case, the operator can quickly confirm the CT image with a short processing time, and can quickly determine the success or failure of the capture.

[0109] The image and the analysis result of the image displayed on the confirmation screen may be simpler than the image and the analysis result of the image displayed on the above-mentioned analysis screen or display screen. That is, the display control unit 304 may display on the display unit 307, as the image and the analysis result on the confirmation screen displayed immediately after the photographing of the subject, an image and the analysis result that are simpler than the image and the analysis result on the analysis screen for analyzing the details of the subject. For example, an image based on data in which the amount of data is thinned by omitting certain data, or the analysis result of the image, etc. can be displayed on the confirmation screen. In this case, the processing time until the confirmation screen is displayed can be shortened, and the operator can quickly determine the success or failure of the photographing.

[0110] Furthermore, the image processing device 30 can calculate an evaluation value of the captured image when displaying the confirmation screen. When the calculated evaluation value is equal to or less than a threshold, the image processing device 30 may determine that re-imaging is necessary and display a display recommending re-imaging on the confirmation screen. Note that, for example, a Q index value can be used as an image evaluation index, but is not limited to this, and any known evaluation index such as an S / N ratio or a contrast value may be used. Note that any known method may be used as a method for calculating the Q index value. Also, the display control unit 304 may display the calculated evaluation value of the image on the confirmation screen. In these cases, the operator can more efficiently determine whether or not re-imaging is necessary.

[0111] (Variation 3) The display screen shown in the first embodiment is an example. Therefore, display items may be added or omitted according to a desired configuration, and the configuration and arrangement of the display screen may be changed according to a desired configuration. For example, the radiation intensity image and the material decomposition image may be displayed side by side. In addition, the type of image, the image capture date, the analysis result, the type of material, etc. may be displayed by superimposing on the corresponding image, or may be displayed in the vicinity of the top, bottom, left, right, etc. of the corresponding image. For example, the display control unit 304 can display information indicating the type of image around the radiation intensity image and the material decomposition image, or superimposing on the radiation intensity image and the material decomposition image. Furthermore, for example, imaging information including the dose of radiation used in imaging, the amount and type of contrast agent, and the imaging site, etc. may be displayed around the top, bottom, left, right, etc. of the image, or superimposed on the image.

[0112] As described above, the images displayed for comparison may be images acquired using different imaging devices, for example, the CT images may be acquired using a CT device that does not use photon counting technology. For this reason, the type and model of the device used to acquire the images may be displayed around the image or superimposed on the image.

[0113] In addition, the analysis results may be displayed superimposed on various images in response to the operation of a button (not shown). In addition, the analysis results of a CT image and a material decomposition image may be displayed superimposed on one of the images. For example, in response to the operation of the operator, the analysis results of the material decomposition image may be displayed superimposed on the CT image, or the analysis results of the CT image may be displayed superimposed on the material decomposition image.

[0114] (Variation 4) In the first embodiment, the order of transfer of captured data, the order of image generation, and the order of image display are not particularly limited. In contrast, these orders may be determined by an operator's instruction or a pre-setting. As described above, the pre-setting may include at least one of a setting pre-defined for each imaging condition including an imaging site, etc., and a setting pre-defined for each imaging mode corresponding to a disease, etc.

[0115] For example, the order of data transfer from the DAS 18 of the gantry device 10 to the image processing device 30 may be determined for each radiation energy band. Depending on the observation purpose, it may be more desirable to display an image based on data in a high energy band or an image based on data in a low energy band. For this reason, the order of data transmission from the DAS 18 to the image processing device 30 may be determined for each energy band depending on the prior settings or the observation purpose. In this case, the acquisition unit 301 can acquire data related to a specific energy band obtained by photographing a subject earlier than data related to other energy bands depending on the prior settings or the observation purpose. Therefore, the operator can check the desired image more efficiently.

[0116] In addition, the order of image generation may be determined so that images generated for display are preferentially generated for a cross-section image desired to be displayed and confirmed first. When observing a CT image or a material decomposition image, it may be desired to preferentially confirm a cross-section image at a predetermined position or a cross-section image of a predetermined tissue according to the purpose of observation. For this reason, the generating unit 302 may determine the order of image generation according to a prior setting or the purpose of observation. Specifically, the generating unit 302 can generate a radiation intensity image and a material decomposition image for a specific position of the subject before generating radiation intensity images and material decomposition images for other positions of the subject according to a prior setting or the purpose of observation. The cross-section for which an image should be preferentially generated may be determined according to a prior setting or the purpose of observation, or may be automatically determined using a result of material decomposition performed prior to image generation.

[0117] Regarding the display order of the images, the display control unit 304 may display a radiation intensity image such as a CT image and then display a material decomposition image on the display unit 307 in accordance with an instruction from an operator or a previous setting. For example, the display control unit 304 may display a radiation intensity image on the display unit 307 after a radiation intensity image requiring a relatively short processing time has been generated, and then display a material decomposition image on the display unit 307 after a material decomposition image requiring a relatively long processing time has been generated. In this case, the radiation intensity image and the material decomposition image may be images generated using common data obtained by photographing a subject using radiation.

[0118] However, regarding the display order, there may be cases where it is desired that the radiation intensity image and the material decomposition image are displayed simultaneously. Therefore, in response to an instruction from an operator, it may be possible to select to display the material decomposition image after the radiation intensity image, or to display the radiation intensity image and the material decomposition image simultaneously. In these cases, the purpose of the observation may be determined based on an instruction from the operator regarding imaging conditions such as the imaging site, or an instruction from the operator regarding the observation purpose. Moreover, the above-mentioned modified examples 1 to 4 are also applicable to the following embodiments.

[0119] Example 2 In a second embodiment of the present disclosure, an example of performing image quality improvement processing for improving the image quality of a radiation intensity image or a material decomposition image using a machine learning model that has been trained will be described. Hereinafter, a CT system 9 according to this embodiment will be described with reference to Figs. 9 to 12, focusing on the differences from the CT system 1 according to the first embodiment. Note that, among the configurations of the CT system 9 according to this embodiment, the same reference symbols are used for configurations similar to those of the CT system according to the first embodiment, and detailed descriptions thereof will be omitted.

[0120] In the following, the machine learning model refers to a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor, naive Bayes, decision trees, and support vector machines. In addition, deep learning, which uses a neural network to generate features and connection weighting coefficients for learning, can also be mentioned. In addition, examples of algorithms using decision trees include methods using gradient boosting such as LightGBM and XGBoost. Appropriately, any of the above algorithms that can be used can be used in the following embodiments and modified examples. In addition, teacher data refers to learning data, and is composed of a pair of input data and output data.

[0121] A trained model is a model that has been trained (learned) in advance using appropriate teacher data (learning data) for a machine learning model that follows any machine learning algorithm such as deep learning. However, although a trained model is obtained in advance using appropriate learning data, this does not mean that no further learning is performed, and additional learning can be performed. Additional learning can also be performed after the device is installed at the site of use.

[0122] 9 is a schematic diagram showing an example of the configuration of a CT system 9 according to this embodiment. An image processing device 930 according to the CT system 9 according to this embodiment includes an image quality improvement unit 907 in addition to the components of the image processing device 30 according to the first embodiment. The image quality improvement unit 907 uses the CT image or material decomposition image generated by the generation unit 302 as an input to the trained model to obtain a CT image or material decomposition image with improved image quality.

[0123] The analysis unit 303 according to this embodiment can perform analysis processing on the high-quality CT image, material decomposition image, etc. acquired by the image quality improvement unit 907. The display control unit 304 according to this embodiment can display the high-quality CT image, material decomposition image, etc. acquired by the image quality improvement unit 907 on the display unit 307.

[0124] (High resolution model) Hereinafter, a trained model (image quality improvement model) for improving the image quality of an image according to this embodiment will be described with reference to Fig. 10. Note that, in this embodiment, the image quality improvement model is stored in the storage unit 306, and is configured to be used in the image quality improvement process by the image quality improvement unit 907, but the image quality improvement model may be provided in an external device (not shown) connected to the image processing device 930.

[0125] The image quality improvement model according to this embodiment is a trained model obtained by performing training (learning) related to a machine learning algorithm. In this embodiment, training of the machine learning model related to the machine learning algorithm uses learning data consisting of a pair group of input data, which is a low-quality image having a specific imaging condition assumed to be processed, and output data, which is a high-quality image corresponding to the input data. The specific imaging conditions specifically include a predetermined imaging site, imaging method, X-ray tube voltage, image size, etc.

[0126] The high image quality model according to the present embodiment is configured as a module that outputs a high image quality image based on an input low image quality image. Here, in this specification, high image quality refers to generating an image with image quality more suitable for image inspection from an input image, and a high image quality image refers to an image with image quality more suitable for image inspection. In addition, a low image quality image is, for example, a two-dimensional image or three-dimensional image acquired by CT or the like, or a three-dimensional moving image of a CT image taken continuously without being set to have a particularly high image quality. Specifically, a low image quality image includes, for example, an image taken at a low dose by CT or the like.

[0127] In addition, when high-quality images with low noise and high contrast are used for various analysis processes and image analysis such as area segmentation processing of CT images, the analysis can often be performed more accurately than when low-quality images are used. Therefore, high-quality images output by the image quality improvement engine can be useful not only for image inspection but also for image analysis.

[0128] In addition, the content of image quality suitable for image inspection depends on what is to be inspected by various image inspections. Therefore, it is not possible to make a general statement, but for example, image quality suitable for image inspection includes image quality with little noise, high contrast, displaying the photographed subject in colors and gradations that are easy to observe, large image size, and high resolution. It can also include image quality in which non-existent objects and gradations that are drawn in the image generation process are removed from the image.

[0129] In the image quality improvement process executed by the image quality improvement unit 907 in this embodiment, a process using various machine learning algorithms such as deep learning is performed. Note that in the image quality improvement process, in addition to the process using the machine learning algorithm, any existing process such as various image filter processes, matching process using a database of high-quality images corresponding to similar images, and knowledge-based image processing may be performed.

[0130] Hereinafter, with reference to Fig. 10, a configuration example of a CNN (Convolutional Neural Network) related to an image quality improvement model according to this embodiment will be described. Fig. 10 shows an example of the configuration of an image quality improvement model. The configuration shown in Fig. 10 is composed of a plurality of layers that process input value groups and output the processed values. As shown in Fig. 10, the types of layers included in this configuration include a convolution layer, a downsampling layer, an upsampling layer, and a merging layer.

[0131] The convolution layer is a layer that performs convolution processing on a group of input values ​​according to parameters such as the set filter kernel size, the number of filters, the stride value, the dilation value, etc. The number of dimensions of the filter kernel size may also be changed according to the number of dimensions of the input image.

[0132] The downsampling layer is a layer that performs processing to reduce the number of output value groups to be less than the number of input value groups by thinning out or combining input value groups. Specifically, such processing includes, for example, Max Pooling processing.

[0133] The upsampling layer is a layer that performs a process to make the number of output values ​​larger than the number of input values ​​by duplicating the input values ​​or adding values ​​interpolated from the input values, for example, a linear interpolation process.

[0134] A synthesis layer is a layer that inputs a group of values, such as a group of output values ​​of a layer or a group of pixel values ​​that make up an image, from multiple sources and performs processing to synthesize them by concatenating or adding them.

[0135] In such a configuration, a group of pixel values ​​constituting an input image 1010 is output through a convolution processing block, and the group of pixel values ​​constituting the input image 1010 is synthesized in a synthesis layer. The synthesized group of pixel values ​​is then formed into a high-quality image 1020 in the final convolution layer.

[0136] Note that different parameter settings for the layers and nodes that make up the neural network may affect the degree to which the trends trained from the learning data can be reproduced during inference. In other words, in many cases, appropriate parameters differ depending on the implementation form, so they can be changed to preferred values ​​as necessary.

[0137] In addition to changing the parameters as described above, CNNs may be able to obtain better characteristics by changing their configurations, such as outputting radiological images with reduced noise with higher accuracy, shortening processing time, or shortening the time required to train a machine learning model.

[0138] The CNN used in this embodiment is a U-net type machine learning model having an encoder function consisting of multiple layers including multiple downsampling layers and a decoder function consisting of multiple layers including multiple upsampling layers. That is, the CNN configuration includes a U-shaped structure having an encoder function and a decoder function. In the U-net type machine learning model, the position information (spatial information) obscured in multiple layers configured as an encoder is configured so that it can be used in layers of the same dimension (layers corresponding to each other) in multiple layers configured as a decoder (for example, by using skip connections).

[0139] Although not shown in the figure, as an example of a modification of the CNN configuration, for example, a batch normalization layer or an activation layer using a rectifier linear unit may be incorporated after the convolution layer.

[0140] Here, the GPU can perform efficient calculations by processing more data in parallel. For this reason, when learning is performed multiple times using a machine learning algorithm such as deep learning, it is effective to perform processing with the GPU. Therefore, in this embodiment, a GPU is used in addition to a CPU for processing by the image quality improvement unit 907 that functions as an example of a learning unit. Specifically, when a learning program including a learning model is executed, the CPU and the GPU cooperate to perform calculations to perform learning. In addition, in the processing of the learning unit, calculations may be performed only by the CPU or the GPU. In addition, the image quality improvement process according to this embodiment may also be realized using a GPU like the learning unit. In addition, when the learned model is provided in an external device, the image quality improvement unit 907 does not need to function as a learning unit.

[0141] The learning unit may also include an error detection unit and an update unit, both not shown. The error detection unit obtains an error between correct data and output data output from the output layer of the neural network according to input data input to the input layer. The error detection unit may use a loss function to calculate an error between the output data from the neural network and correct data. The update unit updates the connection weighting coefficients between the nodes of the neural network, etc., based on the error obtained by the error detection unit, so as to reduce the error. The update unit updates the connection weighting coefficients, etc., using, for example, an error backpropagation method. The error backpropagation method is a method of adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the error.

[0142] It should be noted that when using some image processing methods, such as image processing using CNN, attention should be paid to the image size. Specifically, it should be noted that in order to address the problem of the peripheral areas of a high-quality image not being of sufficient quality, different image sizes may be required for the input low-quality image and the output high-quality image.

[0143] For the sake of clarity, this embodiment does not specify this, but when a high image quality model is adopted that requires different image sizes for the image input to the high image quality model and the image output, the image size is adjusted appropriately. Specifically, the image size is adjusted by padding the input image, such as the image used as learning data for training the machine learning model or the image input to the high image quality model, or by combining the shooting area around the input image. Note that the area to be padded can be filled with a certain pixel value, filled with a neighboring pixel value, or mirror padded according to the characteristics of the high image quality method so that the image quality can be effectively improved.

[0144] The image quality improvement process in the image quality improvement unit 907 may be performed using only one image processing method, or may be performed using a combination of two or more image processing methods. A plurality of image quality improvement methods may be performed in parallel to generate a plurality of high-quality image groups, and the image with the highest image quality may be selected as the final high-quality image. The selection of the image with the highest image quality may be performed automatically using an image quality assessment index, or may be performed according to an instruction from the examiner (operator) by displaying a plurality of high-quality image groups on a UI (User Interface) provided on the display unit 307 or the like.

[0145] In addition, since an image that has not been enhanced in image quality may be more suitable for image inspection, an image that has not been enhanced in image quality may be included in the final image selection. In addition, parameters may be input to the image enhancement model together with a low-image quality image. For example, a parameter specifying the degree of image enhancement or a parameter specifying an image filter size used in an image processing method may be input to the image enhancement model together with the input image.

[0146] (Learning data for high-resolution model) Next, the learning data of the high image quality model according to this embodiment will be described. The input data of the learning data according to this embodiment is a low image quality image acquired by the same model and the same settings as the gantry device 10. The output data of the learning data of the high image quality model is a high image quality image acquired by using image processing such as superposition processing. Specifically, the output data can be a high image quality image obtained by performing superposition processing such as averaging on a group of images acquired by taking images multiple times. The output data of the learning data can also be a high image quality image calculated from a high image quality image acquired by taking an image at a dose higher than the dose related to the input data.

[0147] By using the image quality improvement model that has been trained in this manner, the image quality improvement unit 907 can output a high-image-quality image in which noise has been reduced by overlay processing, etc. Therefore, the image quality improvement unit 907 can generate a high-image-quality image suitable for image inspection based on a low-image-quality image that is an input image.

[0148] Although an example of using a superimposed image as output data of the learning data has been described here, the output data of the learning data of the high image quality model is not limited to this. The output data of the learning data may be a high-image quality image corresponding to the input data, for example, an image that has been contrast-corrected to be suitable for inspection, an image with high resolution, etc. Also, an image obtained by subjecting a low-image quality image, which is the input data, to image processing using statistical processing such as maximum a posteriori probability estimation (MAP estimation) processing can be used as the output data of the learning data. Any known method may be used to generate a high-image quality image.

[0149] In addition, as the high image quality model, a plurality of high image quality models each performing various high image quality processes such as noise reduction, contrast adjustment, and further resolution enhancement may be prepared. Also, one high image quality model performing at least two high image quality processes may be prepared. In these cases, a high image quality image corresponding to the desired process may be used as the output data of the learning data. For example, for a high image quality model including individual processes such as noise reduction, a high image quality image subjected to the individual processes such as noise reduction may be used as the output data of the learning data. In addition, for a high image quality model performing multiple high image quality processes, a high image quality image subjected to, for example, noise reduction and contrast correction may be used as the output data of the learning data.

[0150] By using such training data, a trained model that generates high-quality images with improved image quality can be constructed. Note that, according to such a trained model, for example, a low-quality image captured at a low dose can be used as an input to obtain a high-quality image, so that it is expected that the amount of radiation used for imaging can be reduced and the occurrence of pile-ups in photon counting technology can be suppressed.

[0151] For example, a trained model that takes a low-quality CT image as an input and outputs a high-quality CT image can be acquired by performing training using training data configured as a pair of a low-quality CT image and a high-quality CT image. Similarly, a trained model that takes a low-quality material decomposition image as an input and outputs a high-quality material decomposition image can be acquired by performing training using training data configured as a pair of a low-quality material decomposition image and a high-quality material decomposition image.

[0152] Next, a series of processes including image processing according to this embodiment will be described with reference to Fig. 11 and Fig. 12. Fig. 11 shows a flowchart of a series of processes according to this embodiment. Note that, for the processes in Fig. 11, the same reference numerals as those in the series of processes according to the first embodiment shown in Fig. 3 are used and detailed descriptions thereof are omitted. When the process is started and steps S301 and S302 are performed, the process proceeds to step S1105. Note that in step S301, the acquisition unit 301 can acquire data obtained by imaging the subject S at a low dose, for example.

[0153] In step S1105, the image quality improvement unit 907 obtains a high-quality CT image or a high-quality material decomposition image by using the CT image or the material decomposition image generated by the generation unit 302 as an input of the image quality improvement model. Note that the image quality improvement model may be prepared for each type of image or each type of material, and the image quality improvement unit 907 can select and use the image quality improvement model used for the image quality improvement process according to the type of image to be improved or the type of material to be discriminated. Also, a plurality of image quality improvement models may be provided according to the imaging conditions such as the dose and the imaging site. In this case, the image quality improvement unit 907 can perform the image quality improvement process using the image quality improvement model according to the imaging conditions of the image used as the input. Note that the trained model according to the type of image, the type of material, and the imaging conditions can be obtained by learning using the learning data for each type of image, the learning data for each type of material, and the learning data for each imaging condition.

[0154] In step S303, the analysis unit 303 performs an analysis process on the high-image-quality CT image and material decomposition image acquired by the image quality improvement unit 907. The analysis process may be the same as the analysis process performed in step S303 according to the first embodiment. Note that the analysis unit 303 can also perform the analysis process on the CT image and material decomposition image before the image quality improvement, similar to the first embodiment.

[0155] In step S304, the display control unit 304 causes the display unit 307 to display the high-quality CT image and the high-quality material decomposition image acquired by the image quality improving unit 907 side by side, switched between them, or superimposed on each other. The display control unit 304 also causes the display unit 307 to display the analysis results using the high-quality images by the analysis unit 303. Note that the display screen displayed on the display unit 307 by the display control unit 304 may be the same as the display screen described in the first embodiment, and a high-quality CT image or a material decomposition image can be displayed as the displayed CT image or material decomposition image.

[0156] In addition, the display screen displayed by the display control unit 304 can be switched between an image before being enhanced in quality and an image after being enhanced in quality. Fig. 12 shows an example of a display screen 1201 according to this embodiment. The display screen 1201 is the same as the display screen 501 shown in Fig. 5, but the display screen 1201 further includes a high quality button 1280. When the high quality button 1280 is operated by the operator via the input unit 308, the display control unit 304 collectively switches the CT image 540 and the material decomposition image 550 to the CT image and the material decomposition image after being enhanced in quality. In addition, the high quality button 1280 may be provided for each image to be enhanced in quality.

[0157] This process can be similarly applied to the other display screens 601, 701, 702, 703, 801, 802, etc. described in the first embodiment. In addition, this process can be similarly applied to the screens of the thumbnail list and the temporal difference, etc., described in the first embodiment.

[0158] Furthermore, when switching between an image before being enhanced and an image after being enhanced, the display control unit 304 can switch the displayed analysis results to the analysis results corresponding to the image to be switched and displayed. For example, when the image quality improvement button 1280 is operated to switch the CT image 540 and the material decomposition image 550 to the image after being enhanced, the display control unit 304 can collectively switch the analysis results 543, 553 to the analysis results for the image after being enhanced. Note that, when the image quality improvement button is provided for each image, the display control unit 304 can switch the analysis results corresponding to the image to be switched between the analysis results of the image before being enhanced and the analysis results of the image after being enhanced. In step S304, when the display control process by the display control unit 304 ends, a series of processes according to this embodiment ends.

[0159] As described above, the display control unit 304 according to this embodiment switches to an image having higher image quality than at least one of the radiation intensity image and the material decomposition image acquired by using the at least one of the images as an input of the learned model, in response to an instruction from the operator, and causes the display unit 307 to display the at least one image. In this case, the operator can appropriately observe the radiation intensity image and the material decomposition image in a state in which the image quality is improved and the image can be easily compared, thereby enabling more efficient observation.

[0160] The trained model may include at least one of a plurality of trained models according to the type of image and a plurality of trained models according to the type of material to be discriminated. In this case, the trained model makes it possible to obtain an image that has been subjected to more appropriate image quality improvement processing according to the type of image or the type of material to be discriminated, and the operator can observe the image with higher image quality more efficiently.

[0161] In this embodiment, an example using a high image quality model for performing high image quality processing has been described. However, the processing using the trained model is not limited to the high image quality processing, and may be a segmentation processing or the like. In this case, as the training data of the trained model, a CT image or a material decomposition image may be used as input data, and a labeled image in which a doctor or the like labels each region of the input data may be used as output data. The output data may be a labeled image labeled by a known rule-based segmentation processing. The rule base refers to processing based on the regularity of tissues. The configuration of the machine learning model may be the same as that of the above-mentioned high image quality model.

[0162] In this case, the image processing device 930 can obtain an image in which each region of the at least one image used as the input is labeled by using at least one of the radiation intensity image and the material decomposition image generated by the generation unit 302 as an input to the trained model. The trained model may also be prepared for each type of image and each type of material. In this case, the image processing device 930 can select and use a trained model to be used for the segmentation process according to the type of image to be segmented and the type of material to be discriminated. In addition, a plurality of trained models may be provided according to the imaging conditions such as the dose and the imaging site. In this case, the image processing device 930 can perform the segmentation process using a trained model according to the imaging conditions of the image used as the input. The trained model according to the type of image, the type of material, and the imaging conditions can be obtained by learning using the learning data for each type of image, the learning data for each type of material, and the learning data for each imaging condition.

[0163] The display control unit 304 can display the acquired label image as the analysis result on the display screen. Note that, for the display of the analysis result using the trained model, a corresponding switching button may also be provided, and the display control unit 304 may control ON / OFF of the analysis result in response to the operation of the switching button by the operator. Note that the display control unit 304 can also switch the acquired label image to at least one of the corresponding radiation intensity image and material decomposition image and display it on the display unit 307.

[0164] In addition, the display control unit 304 may change whether or not to display a high image quality button or a switching button for receiving an instruction from an operator regarding processing using a trained model for segmentation, depending on the shooting conditions of the image. For example, when displaying an image shot under shooting conditions that do not match the shooting conditions of the learning data of the prepared trained model, the display control unit 304 may not display the high image quality button or the switching button for segmentation on the display unit 307. On the other hand, when displaying an image shot under shooting conditions that match the shooting conditions of the learning data of the prepared trained model, the display control unit 304 may display the high image quality button or the switching button for segmentation on the display unit 307.

[0165] Here, the photographing conditions may include the type of image. In this case, for example, when a trained model for CT images is not prepared, the display control unit 304 can not display a switching button for processing using the trained model for CT images on the display screen displaying the CT images.

[0166] In addition, when performing the image quality improvement process or the segmentation process using the learned model, the image quality improvement unit 907 or the image processing device 930 may determine whether or not the process can be performed based on the shooting conditions. In this case, the display control unit 304 may change whether or not to display the image quality improvement button or the switching button related to segmentation according to the result of the determination by the image quality improvement unit 907 or the image processing device 930. For example, when the image quality improvement unit 907 determines that the image quality improvement process cannot be performed based on the shooting conditions, the display control unit 304 may not display the image quality improvement button on the display screen based on the determination of the image quality improvement unit 907.

[0167] In these cases, when a process using the trained model cannot be performed, the button related to the process is not displayed on the display screen, thereby preventing the operator from performing an inappropriate operation. Note that when a process using the trained model cannot be performed, the display control unit 304 may display a message on the display screen indicating that the process cannot be performed.

[0168] The image quality improvement process and the segmentation process using the trained model may be executed in response to the operation of a corresponding image quality improvement button or a switching button. In this case, when the process using the trained model cannot be executed based on the shooting conditions or the like, the display control unit 304 may display a message on the display screen indicating that the process cannot be executed.

[0169] When an image is acquired using a machine learning model, an image showing a tissue or the like that does not actually exist may be generated. Therefore, when displaying a high-quality image or a labeled image acquired using a trained model, the display control unit 304 may display that these images are images acquired using a machine learning model. In this case, the operator can observe the image after understanding that the image was acquired using a machine learning model, thereby preventing erroneous judgments and the like caused by processing using the machine learning model.

[0170] The learning data for the image quality improvement model and the trained model for segmentation described above is not limited to data obtained using the image capture device itself that actually captures images. The image data may be data obtained using the same type of image capture device or the same kind of image capture device, depending on the desired configuration.

[0171] (Variation 5) In the various embodiments and modified examples described above, the examiner's instructions regarding display, analysis, image quality improvement processing, segmentation processing, etc. may be instructions by voice or the like in addition to manual instructions (e.g., instructions using a user interface, etc.). At this time, for example, a machine learning model including a voice recognition model (voice recognition engine, trained model for voice recognition) obtained by machine learning may be used. The manual instructions may also be instructions by character input, etc. using a keyboard, touch panel, etc. At this time, for example, a machine learning model including a character recognition model (character recognition engine, trained model for character recognition) obtained by machine learning may be used. The examiner's instructions may also be instructions by gestures, etc. At this time, a machine learning model including a gesture recognition model (gesture recognition engine, trained model for gesture recognition) obtained by machine learning may be used.

[0172] The instruction from the examiner may be a gaze detection result of the examiner on the monitor. The gaze detection result may be, for example, a pupil detection result using a moving image of the examiner captured around the monitor. In this case, the pupil detection from the moving image may use an object recognition engine as described above. The instruction from the examiner may be an instruction based on brain waves, weak electrical signals flowing through the body, or the like.

[0173] In such a case, for example, the learning data may be learning data in which character data or voice data (waveform data) indicating an instruction to display a radiation intensity image, a material decomposition image, a superimposed image, etc. is used as input data, and execution commands for actually displaying various images on the display unit 307 are used as output data. In addition, the learning data may be learning data in which character data or voice data indicating an instruction to display a high-quality image obtained by a high-quality model is used as input data, and an execution command to display the high-quality image and an execution command to change the image quality improvement button to an active state are used as correct answer data. Similarly, the learning data may be learning data in which character data or voice data indicating an instruction to display a label image obtained by a trained model for segmentation is used as input data, and an execution command to display the label image and an execution command to change the corresponding switching button to an active state are used as correct answer data. The learning data may be any data as long as the instruction content indicated by the character data or voice data, etc. corresponds to the execution command content. In addition, the voice data may be converted to character data using an acoustic model, a language model, etc. Furthermore, a process for reducing noise data superimposed on the voice data may be performed using waveform data obtained by a plurality of microphones. Also, the apparatus may be configured to be able to select between instructions by characters or voice, etc. and instructions by a mouse, touch panel, etc., according to an instruction from the examiner. Furthermore, the apparatus may be configured to be able to select whether instructions by characters or voice, etc. are turned on or off according to an instruction from the examiner.

[0174] Here, the machine learning includes deep learning as described above, and for example, a recurrent neural network (RNN) can be used for at least a part of a multi-layered neural network. Here, as an example of a machine learning model according to this modification, an RNN, which is a neural network that handles time-series information, will be described with reference to Figs. 13(a) and 13(b). In addition, a long short-term memory (hereinafter, LSTM), which is a type of RNN, will be described with reference to Figs. 14(a) and 14(b).

[0175] FIG. 13(a) shows the structure of an RNN, which is a machine learning model. The RNN 1320 has a loop structure in the network, and at time t, data x t Enter 1310 and enter data h t The RNN 1320 outputs 1330. Since the RNN 1320 has a loop function in the network, it can take over the current state as the next state, and can therefore handle time-series information. Figure 13(b) shows an example of input and output of a parameter vector at time t. Data x t 1310 contains N pieces of data (Params1 to ParamsN). In addition, the data h output from the RNN 1320 t 1330 includes N pieces of data (Params1 to ParamsN) corresponding to the input data.

[0176] However, since RNNs cannot handle long-term information during backpropagation, LSTMs are sometimes used. LSTMs can learn long-term information by providing a forget gate, an input gate, and an output gate. The structure of LSTM is shown in FIG. 14(a). In LSTM 1440, the information that the network takes over at the next time t is the internal state of the network called a cell, c t-1 and output data h t-1 The lowercase letters (c, h, x) in the figure represent vectors.

[0177] Next, the details of LSTM1440 are shown in Figure 14(b). In Figure 14(b), FG is the forget gate network, IG is the input gate network, and OG is the output gate network, each of which is a sigmoid layer. Therefore, it outputs a vector in which each element has a value between 0 and 1. The forget gate network FG decides how much past information to retain, and the input gate network IG decides which value to update. CU is the cell update candidate network, which is an activation function tanh layer. It creates a vector of new candidate values ​​to be added to the cell. The output gate network OG selects the cell candidate elements and selects how much information to convey at the next time.

[0178] The above-mentioned LSTM model is a basic form, and is not limited to the network shown here. The connections between networks may be changed. A QRNN (Quasi Recurrent Neural Network) may be used instead of LSTM. Furthermore, the machine learning model is not limited to a neural network, and boosting, a support vector machine, etc. may be used. In addition, when the instructions from the examiner are input by characters or voice, etc., a technology related to natural language processing (e.g., Sequence to Sequence) may be applied. In addition, a dialogue engine (dialogue model, trained model for dialogue) that responds to the examiner by outputting characters or voice, etc. may be applied.

[0179] In the high-image-quality model and the trained model for segmentation described in the second embodiment, the magnitude of the luminance value of the image of the input data, the order and gradient of the bright and dark areas, the position, distribution, continuity, etc. are extracted as part of the feature amount and used in the inference process. On the other hand, trained models for voice recognition, character recognition, gesture recognition, etc. are trained using time-series data, so the gradient between the input consecutive time-series data values ​​is also extracted as part of the feature amount and used in the estimation process. Therefore, such trained models are expected to be able to perform accurate estimation by using the effect of the time-dependent change in specific numerical values ​​in the estimation process.

[0180] The image processing device 930 can be provided with the image-enhancing model and the trained model for segmentation. The trained model may be configured, for example, by a software module executed by a processor such as a CPU, MPU, GPU, or FPGA, or may be configured by a circuit that performs a specific function such as an ASIC. The trained model may be provided in another server device connected to the image processing device 930. In this case, the image processing device 930 can use the trained model by connecting to a server or the like equipped with the trained model via any network such as the Internet. Here, the server equipped with the trained model may be, for example, a cloud server, a fog server, an edge server, or the like. In addition, when configuring a network within a facility, a site including a facility, or an area including multiple facilities to be capable of wireless communication, the reliability of the network may be improved by configuring the network to use radio waves in a dedicated wavelength band that is limited to the facility, site, area, or the like. In addition, the network may be configured by wireless communication that is capable of high speed, large capacity, low latency, and multiple simultaneous connections.

[0181] In the first to third embodiments, the generating unit 302 generates a radiation intensity image or a material decomposition image such as a CT image, but the acquiring unit 301 may acquire a radiation intensity image or a material decomposition image from an image processing device or a storage device (not shown) via an arbitrary network, and the acquired image may be used for display processing, other image processing, and the like.

[0182] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions. A computer may have one or more processors or circuits, and may include separate computers or a network of separate processors or circuits to read and execute computer-executable instructions.

[0183] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA), and the processor or circuitry may include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0184] The above disclosure includes the following configurations, methods, and programs. (Configuration 1) an acquisition unit that acquires a radiation intensity image obtained by imaging a subject using radiation, and a material decomposition image that is an image obtained by imaging the subject by counting photons of the radiation and indicates decomposed materials; a display control unit that causes the radiation intensity image and the material decomposition image to be displayed side by side, switched between, or superimposed on a display unit; An image processing device comprising: (Configuration 2) 2. The image processing device according to configuration 1, wherein the display control unit causes the display unit to display options for selecting a material to be decomposed around the material decomposition image. (Configuration 3) 3. The image processing device according to configuration 1 or 2, wherein the radiation intensity image and the material decomposition image are images generated using common data obtained by photographing the subject using radiation. (Configuration 4) 4. The image processing device according to any one of configurations 1 to 3, wherein the material decomposition image includes an image in which a plurality of material decomposition images obtained by decomposing different types of materials are superimposed on each other. (Configuration 5) 5. The image processing device according to any one of configurations 1 to 4, wherein the material decomposition image includes a plurality of material decomposition images obtained by decomposing different types of materials. (Configuration 6) 6. The image processing device according to configuration 5, wherein the display control unit causes the display unit to display each of the plurality of material decomposition images adjacent to the radiation intensity image. (Configuration 7) The display control unit is Switching between a plurality of radiation intensity images and a plurality of material decomposition images collectively and displaying them on the display unit, or 6. The image processing device according to any one of configurations 1 to 5, wherein the display unit is configured to switch between the plurality of radiation intensity images or the plurality of material decomposition images and a plurality of superimposed images of the plurality of radiation intensity images and the plurality of material decomposition images collectively and display them. (Configuration 8) 6. The image processing device according to any one of configurations 1 to 5, wherein the display control unit sets a transparency for one of the radiation intensity image and the material decomposition image in response to an instruction from an operator, and displays the one of the radiation intensity image and the material decomposition image on the display unit in a superimposed manner on the other of the radiation intensity image and the material decomposition image. (Configuration 9) 6. The image processing device according to any one of configurations 1 to 5, wherein the display control unit causes the display unit to superimpose the intensity image of the radiation and the material decomposition image on each other at a blend ratio between the intensity image of the radiation and the material decomposition image in accordance with an instruction from an operator, and the blend ratio is a transparency ratio such that increasing one of the transparency of the intensity image of the radiation and the transparency of the material decomposition image decreases the other. (Configuration 10) The image processing device according to any one of configurations 1 to 9, wherein when switching between the radiation intensity image and the material decomposition image and a radiation intensity image and a material decomposition image at a different position of the subject, the display control unit switches between the radiation intensity image and the material decomposition image collectively and displays them on the display unit in response to an instruction from an operator. (Configuration 11) 11. The image processing device according to any one of configurations 1 to 10, wherein the display control unit causes the display unit to display information indicating a type of image around the radiation intensity image and the material decomposition image, or superimposed on the radiation intensity image and the material decomposition image. (Configuration 12) an analysis unit configured to analyze at least one of the radiation intensity image and the material decomposition image, 12. The image processing device according to any one of configurations 1 to 11, wherein the display control unit causes the display unit to display a result of the analysis by the analysis unit around the analyzed image or superimposed on one of the radiation intensity image and the material decomposition image. (Configuration 13) The image processing device of configuration 12, wherein the display control unit causes the display unit to display, as an analysis result on a confirmation screen displayed immediately after photographing the subject, a simpler analysis result than the analysis result on an analysis screen for analyzing details of the subject. (Configuration 14) 14. The image processing device according to any one of configurations 1 to 13, wherein the display control unit causes the display unit to display information regarding imaging errors for at least one of the radiation intensity image and the material decomposition image in response to an instruction from an operator. (Configuration 15) the radiation intensity image and the material decomposition image are images generated using common data obtained by photographing the subject using radiation, 15. The image processing device according to any one of configurations 1 to 14, wherein the display control unit causes the display unit to display the radiation intensity image, and then causes the display unit to display the material decomposition image. (Configuration 16) The image processing device of any one of configurations 1 to 15, wherein the acquisition unit acquires data relating to a specific energy band obtained by photographing the subject in accordance with a prior setting or an instruction from an operator, earlier than data relating to other energy bands. (Configuration 17) a generating unit that generates the radiation intensity image and the material decomposition image using data obtained by photographing the subject, 17. The image processing device according to any one of configurations 1 to 16, wherein the generation unit generates a radiation intensity image and a material decomposition image for a specific position of the subject in response to a preset setting or an instruction from an operator, earlier than radiation intensity images and material decomposition images for other positions of the subject. (Configuration 18) 18. The image processing device according to configuration 16 or 17, wherein the advance settings include at least one of settings predetermined for each imaging condition including an imaging region and settings predetermined for each imaging mode according to a disease. (Configuration 19) 19. The image processing device according to any one of configurations 1 to 18, wherein the display control unit switches to an image having higher image quality than at least one of the radiation intensity image and the material decomposition image obtained by using at least one of the images as an input of a trained model, and displays the at least one image on the display unit in response to an instruction from an operator. (Configuration 20) 20. The image processing device according to any one of configurations 1 to 19, wherein the display control unit switches to an image in which each region of at least one of the radiation intensity image and the material decomposition image obtained by using the at least one of the images as an input for a trained model is labeled, and displays the at least one of the images on the display unit in response to an instruction from an operator. (Configuration 21) 21. The image processing device according to configuration 19 or 20, wherein the display control unit changes whether or not to display for receiving an instruction from the operator based on a shooting condition of the at least one image. (Configuration 22) The image processing device according to any one of configurations 19 to 21, wherein the trained model includes at least one of a plurality of trained models corresponding to a type of image and a plurality of trained models corresponding to a type of material to be discriminated. (Configuration 23) An image processing device according to any one of configurations 1 to 22, wherein an operator's instruction regarding display is information obtained using at least one of a trained model for character recognition, a trained model for voice recognition, and a trained model for gesture recognition. (Configuration 24) an imaging device that has a radiation detector capable of counting photons of radiation and that images the subject; An image processing device according to any one of configurations 1 to 23, An imaging system comprising: (Method 1) obtaining a radiation intensity image obtained by photographing an object using radiation, and a material decomposition image obtained by photographing the object by counting photons of the radiation, the material decomposition image indicating decomposed materials; displaying the radiation intensity image and the material decomposition image on a display unit side by side, switched between, or superimposed on each other; An image processing method comprising: (Program 1) A program that, when executed by a computer, causes the computer to execute each step of the image processing method described in Method 1.

[0185] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The present invention also includes inventions modified within the scope of the present invention and inventions equivalent to the present invention. In addition, the above-mentioned embodiments and modifications can be appropriately combined within the scope of the present invention. [Explanation of symbols]

[0186] 30: image processing device, 301: acquisition unit, 302: generation unit, 304: display control unit

Claims

1. an acquisition unit that acquires a radiation intensity image obtained by imaging a subject using radiation and a material decomposition image that is an image obtained by imaging the subject by counting photons of radiation and that indicates decomposed materials; a display control unit that displays the radiation intensity image and a plurality of types of material decomposition images obtained by decomposing different types of materials side by side on a display unit; Equipped with the display control unit, when switching between the radiation intensity image and the plurality of types of material decomposition images and between radiation intensity images and material decomposition images at different positions on the subject, switches between the radiation intensity image and the plurality of types of material decomposition images collectively and displays them on the display unit in response to an instruction from an operator.

2. The image processing device according to claim 1 , wherein the display control unit causes the display unit to display options for selecting a material to be decomposed around the material decomposition image.

3. The image processing apparatus according to claim 1 , wherein the radiation intensity image and the material decomposition image are images generated using common data obtained by imaging the subject using radiation.

4. The image processing apparatus according to claim 1 , wherein the display control unit causes the display unit to display each of the plurality of types of material decomposition images adjacent to the radiation intensity image.

5. The display control unit a superimposed image of the radiation intensity image and at least one material decomposition image of the plurality of types of material decomposition images is further displayed side by side on the display unit; 2. The image processing device according to claim 1, wherein, when switching the radiation intensity image, the plurality of types of material decomposition images, and the superimposed image to the radiation intensity image, the plurality of types of material decomposition images, and the superimposed image at different positions of the subject, the radiation intensity image, the plurality of types of material decomposition images, and the superimposed image are switched collectively and displayed on the display unit in response to an instruction from an operator.

6. 6. The image processing device according to claim 5, wherein the display control unit sets transparency for one of the radiation intensity image and the at least one material decomposition image in response to an instruction from an operator, and causes the display unit to display an image superimposed on the other of the radiation intensity image and the at least one material decomposition image as the superimposed image.

7. 6. The image processing device according to claim 5, wherein the display control unit causes the display unit to display, as the superimposed image, an image in which the radiation intensity image and the at least one material decomposition image are superimposed on each other at a blend ratio between the radiation intensity image and the at least one material decomposition image in accordance with an instruction from an operator, the blend ratio being a transparency ratio such that increasing one of the transparency of the radiation intensity image and the transparency of the at least one material decomposition image decreases the other.

8. 2. The image processing device according to claim 1, wherein the display control unit causes the display unit to display information indicating the type of image around the radiation intensity image and the material decomposition image or superimposed on the radiation intensity image and the material decomposition image.

9. an analysis unit that analyzes at least one of the radiation intensity image and the material decomposition image, The image processing apparatus according to claim 1 , wherein the display control unit causes the display unit to display the analysis result by the analysis unit around the analyzed image or superimposed on one of the radiation intensity image and the material decomposition image.

10. 10. The image processing device according to claim 9, wherein the display control unit causes the display unit to display, as the analysis result on the confirmation screen displayed immediately after photographing the subject, a simpler analysis result than the analysis result on an analysis screen for analyzing the details of the subject.

11. The image processing apparatus according to claim 1 , wherein the display control unit causes the display unit to display information about an imaging error in at least one of the radiation intensity image and the material decomposition image in response to an instruction from an operator.

12. the radiation intensity image and the material decomposition image are images generated using common data obtained by imaging the subject using radiation, The image processing apparatus according to claim 1 , wherein the display control unit causes the display unit to display the radiation intensity image and then causes the display unit to display the material decomposition image.

13. The image processing device according to claim 1 , wherein the acquisition unit acquires data relating to a specific energy band obtained by photographing the subject earlier than data relating to other energy bands, in accordance with a preset setting or an instruction from an operator.

14. a generating unit that generates the radiation intensity image and the material decomposition image using data obtained by imaging the subject, 2. The image processing device according to claim 1, wherein the generation unit generates a radiation intensity image and a material decomposition image for a specific position of the subject earlier than radiation intensity images and material decomposition images for other positions of the subject, in accordance with a preset setting or an instruction from an operator.

15. The image processing device according to claim 13 or 14, wherein the advance settings include at least one of a setting predetermined for each imaging condition including an imaging region, and a setting predetermined for each imaging mode according to a disease.

16. 2. The image processing device according to claim 1, wherein the display control unit switches to an image having higher image quality than at least one of the radiation intensity image and the material decomposition image acquired by using at least one of the radiation intensity image and the material decomposition image as an input of a trained model, and displays the at least one image on the display unit in response to an instruction from an operator.

17. 2. The image processing device according to claim 1, wherein the display control unit switches to an image in which each region of at least one of the radiation intensity image and the material decomposition image, acquired by using the at least one of the radiation intensity image and the material decomposition image as an input of a trained model, and displays the at least one of the images on the display unit in response to an instruction from an operator.

18. The image processing device according to claim 16 , wherein the display control unit changes whether or not a display for receiving an instruction from the operator is displayed based on a capture condition of the at least one image.

19. The image processing device according to claim 16 or 17, wherein the trained model includes at least one of a plurality of trained models according to the type of image and a plurality of trained models according to the type of material to be discriminated.

20. 2. The image processing device according to claim 1, wherein the operator's instructions regarding the display are information obtained using at least one of a trained model for character recognition, a trained model for speech recognition, and a trained model for gesture recognition.

21. an imaging device that includes a radiation detector capable of counting radiation photons and that images the subject; The image processing device according to claim 1 ; An imaging system comprising:

22. obtaining a radiation intensity image obtained by photographing a subject using radiation and a material decomposition image obtained by photographing the subject by counting photons of radiation, the material decomposition image indicating decomposed materials; displaying the radiation intensity image and a plurality of types of material decomposition images obtained by decomposing different types of materials side by side on a display unit; Including, the displaying includes, when switching the radiation intensity image and the plurality of types of material decomposition images to radiation intensity images and material decomposition images at different positions on the subject, switching the radiation intensity image and the plurality of types of material decomposition images collectively and displaying them on the display unit in response to an instruction from an operator.

23. A program that, when executed by a computer, causes the computer to execute each step of the image processing method according to claim 22.

Images