Image processing apparatus, radiation imaging system, image processing method, and storage medium

Abstract

An image processing device that generates a high-resolution ES image using a low-resolution image generated from a radiographic image using a learned model is provided.SOLUTION: An acquisition unit configured to acquire a first image that is a radiation image, and a processing unit configured to generate a second image that is an energy subtraction image using the first image, wherein the processing unit performs a process of reducing a resolution of the first image, acquires a third image that is output from the learned model by inputting the first image with the reduced resolution to the learned model, and is a different type of image from the first image, and performs a process of improving a resolution of the third image, generating a second image that is an image of a different type from the third image and has a higher resolution than the acquired third image by using the first image and the third image with the improved resolution.SELECTED DRAWING: Figure 20

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, a radiation imaging system, an image processing method, and a program. [Background technology]

[0002] Currently, radiation imaging devices using flat panel detectors (FPDs) made of semiconductor materials are widely used as imaging devices for X-ray medical image diagnosis and non-destructive testing. In medical image diagnosis, for example, such radiation imaging devices are used as digital imaging devices for capturing still images such as general radiography and for capturing moving images such as fluoroscopy.

[0003] One of the imaging methods using FPDs is energy subtraction (ES). In ES, multiple images with different energies are acquired by irradiating X-rays with different tube voltages. Then, by performing calculations using the multiple acquired images with different energies, it is possible to perform processing such as acquiring material-decomposed images such as bone images and soft tissue images.

[0004] Here, Patent Document 1 proposes a configuration in which a subject is irradiated with multiple radiation beams having different average energies in a short period of time and a video is captured using energy subtraction. Also, Patent Document 2 proposes a method for generating a high-quality energy subtraction image from multiple images of different radiation energies using an image generation model based on machine learning. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-162358 [Patent Document 2] Japanese Patent Publication No. 2023-120851 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when generating ES images using a machine learning model, as in the technology described in Patent Document 2, there is a possibility that the pixel pitch of the training images will not match the pixel pitch of the radiographic images used as input data for the trained model. For example, when low-resolution images are used to prepare a large amount of training data, the pixel pitch of the training images may be larger than the pixel pitch of the radiographic images used as input data for the trained model. In such cases, an ES image with a lower resolution than the resolution of the radiographic images used as input data for the trained model will be output.

[0007] Therefore, one object of one embodiment of the present disclosure is to generate a high-resolution ES image using a low-resolution image generated from a radiological image using a trained model. [Means for solving the problem]

[0008] An image processing device according to one embodiment of the present disclosure includes an acquisition unit that acquires a first image, which is a radiological image, and a processing unit that uses the first image to generate a second image, which is an energy subtraction image. The processing unit performs a process to reduce the resolution of the first image, inputs the first image with reduced resolution into a trained model, and thereby acquires a third image, which is an image of a different type from the first image, output from the trained model. The processing unit performs a process to improve the resolution of the third image, and uses the first image and the third image with improved resolution to generate the second image, which is an image of a different type from the third image and has a higher resolution than the acquired third image. [Effects of the Invention]

[0009] According to one embodiment of the present disclosure, a high-resolution ES image can be generated using a low-resolution image generated from a radiological image using a trained model. [Brief explanation of the drawings]

[0010] [Figure 1] 1 shows an example of the overall configuration of a radiation imaging system according to a first embodiment. [Figure 2] FIG. 2 is an equivalent circuit diagram of an example of a pixel of the radiation imaging device. [Figure 3] 10 is a timing chart showing an example of a radiation imaging operation. [Figure 4] 10 is a timing chart showing an example of a radiation imaging operation. [Figure 5] FIG. 10 is a block diagram of a correction process included in the preprocessing. [Figure 6] FIG. 1 is a block diagram of signal processing for ES processing. [Figure 7] FIG. 10 is a block diagram of a process for generating a total thickness image. [Figure 8] FIG. 10 is a block diagram of a process for generating a virtual monochromatic X-ray image. [Figure 9] 10 is a flowchart illustrating an example of a learning process. [Figure 10] FIG. 10 is a block diagram of a learning process. [Figure 11] 1 shows an example of the configuration of a machine learning model. [Figure 12] 3 shows an example of learning data according to the first embodiment. [Figure 13] 10 is a flowchart illustrating an example of an image generation process using a trained model. [Figure 14] FIG. 1 is a block diagram of an inference process using a trained model. [Figure 15] 3A to 3C show examples of input and output images during learning and inference according to the first embodiment. [Figure 16] FIG. 10 is a block diagram showing an example of a process for generating an ES image from a radiation image. [Figure 17] FIG. 10 is a block diagram showing another example of the process of generating an ES image from a radiation image. [Figure 18] FIG. 10 is a block diagram showing another example of the process of generating an ES image from a radiation image. [Figure 19]4 is a flowchart showing an example of a series of processes according to the first embodiment. [Figure 20] FIG. 2 is a block diagram showing an example of a series of processes according to the first embodiment. [Figure 21] FIG. 10 is a diagram for explaining a total thickness image. [Figure 22] 10 is a flowchart showing an example of a series of processes according to Modification 1. [Figure 23] FIG. 10 is a block diagram showing an example of a series of processes according to Modification 1. [Figure 24] FIG. 10 is a block diagram showing another example of a series of processes according to the first modification. [Figure 25] 10 shows an example of learning data according to the second embodiment. [Figure 26] 10A to 10C show examples of input and output images during learning and inference according to the second embodiment. [Figure 27] FIG. 10 is a block diagram showing an example of a process for generating an ES image from a radiation image. [Figure 28] 10 is a flowchart showing an example of a series of processes according to the second embodiment. [Figure 29] FIG. 10 is a block diagram showing an example of a series of processes according to the second embodiment. [Figure 30] 10 is a flowchart showing an example of a series of processes according to Modification 2. [Figure 31] FIG. 10 is a block diagram showing an example of a series of processes according to Modification 2. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments are arbitrary and can be changed depending on the configuration of an apparatus to which the present disclosure is applied or various conditions. In addition, the same reference numerals are used in the drawings to indicate identical or functionally similar elements.

[0012] In this disclosure, radiation includes α-rays, β-rays, γ-rays, and the like, which are beams created by particles (including photons) emitted by radioactive decay, as well as beams with the same or higher energy levels, such as X-rays, particle beams, and cosmic rays.

[0013] In the following, a machine learning model refers to a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor algorithms, naive Bayes algorithms, decision trees, and support vector machines. Another example is deep learning, which uses a neural network to generate features and connection weighting coefficients for learning. Any of the above algorithms that can be used can be used as appropriate and applied to the following embodiments. Furthermore, training data refers to training data, and is composed of a pair of input data and output data. Furthermore, supervised answer data refers to output data from training data (training data).

[0014] Furthermore, a trained model refers to a machine learning model that follows any machine learning algorithm, such as deep learning, and that has been trained (learned) in advance using appropriate training data (learning data). However, although a trained model is obtained in advance using appropriate training data, it does not mean that no further learning is performed, and additional learning can also be performed. Additional learning can also be performed after the device is installed at the site of use. Note that obtaining output data from input data using a trained model is sometimes referred to as inference.

[0015] (First embodiment) (Configuration of Radiation Imaging System) A radiation imaging system, an image processing device, and an image processing method according to this embodiment will be described below with reference to Fig. 1 to Fig. 21. Fig. 1 is a block diagram showing an example of the overall configuration of the radiation imaging system according to this embodiment. The radiation imaging system according to this embodiment includes a radiation generating device 101, a radiation control device 102, a control device 103, a radiation imaging device 104, an input unit 150, and a display unit 120.

[0016] The radiation generating device 101 includes a radiation source such as a tube. The radiation generating device 101 generates radiation under the control of the radiation control device 102. The radiation control device 102 includes a control circuit, a processor, etc. Based on the control of the control device 103, the radiation control device 102 controls the radiation generating device 101 so that radiation is irradiated toward the subject Su and the radiation imaging device 104. More specifically, the radiation control device 102 can control imaging conditions such as the irradiation angle of radiation emitted by the radiation generating device 101, the radiation focal position, the tube voltage, and the tube current.

[0017] The radiation control device 102, the radiation imaging device 104, the input unit 150, and the display unit 120 are connected to the control device 103, and the control device 103 can control these. The control device 103 can perform, for example, various controls related to radiation imaging and image processing for generating ES images.

[0018] The control device 103 is provided with an acquisition unit 131, a generation unit 132, a processing unit 133, a display control unit 134, and a storage unit 135. The acquisition unit 131 can acquire images captured by the radiation imaging device 104, images generated by the generation unit 132, etc. The acquisition unit 131 can also acquire various images from an external device (not shown) connected to the control device 103 via a network such as the Internet.

[0019] The generation unit 132 can generate a radiation image from an image (image information) captured using the radiation imaging device 104 and acquired by the acquisition unit 131. The generation unit 132 can generate energy images (high energy image and low energy image) related to radiation energy from an image captured using the radiation imaging device 104 by irradiating radiation of any energy, for example. A method for generating an energy image will be described later.

[0020] The processing unit 133 generates an ES image based on an energy image (radiographic image) using the trained model. The trained model and a method for generating an ES image will be described later. The processing unit 133 can also perform image processing, analysis processing, and the like using the generated ES image, etc. Furthermore, the processing unit 133 can also function as an example of a learning unit that trains a machine learning model.

[0021] The display control unit 134 controls the display of the display unit 120 and can display, for example, information on the subject Su (examination object), information on radiography, various acquired and generated images, etc. on the display unit 120. The storage unit 135 can store, for example, information on the subject Su, information on radiography, and various acquired and generated images. The storage unit 135 can also store programs and the like for the control device 103 to perform various processes.

[0022] Here, the control device 103 can be configured by a computer provided with a processor and a memory. The control device 103 may be configured by a general computer or a computer dedicated to the radiation imaging system. The control device 103 may be, for example, a personal computer, and a desktop PC, a notebook PC, a tablet PC (portable information terminal), or the like may be used. Furthermore, the control device 103 may be configured as a cloud-based computer in which some of the components are located in an external device.

[0023] Furthermore, each component of the control device 103 other than the storage unit 135 may be configured by a software module executed by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor may be, for example, a GPU (Graphical Processing Unit) or an FPGA (Field-Programmable Gate Array). Each component may be configured by a circuit that performs a specific function, such as an ASIC. The storage unit 135 may be configured by any storage medium, such as an optical disk such as a hard disk, or a memory.

[0024] The display unit 120 is configured with any monitor, and displays various information such as information about the subject Su, various images, and a mouse cursor in accordance with the operation of the input unit 150, under the control of the display control unit 134. The input unit 150 is an input device that issues instructions to the control device 103, and specifically includes a keyboard and a mouse. Note that the display unit 120 may be configured with a touch panel display, and in this case, the display unit 120 can also be used as the input unit 150.

[0025] The radiation imaging device 104 detects radiation that is irradiated from the radiation generation device 101 and passes through the subject Su, and captures a radiation image. The radiation imaging device 104 can be configured as, for example, an FPD. The radiation imaging device 104 is provided with a phosphor 141 that converts radiation into visible light, and a two-dimensional detector 142 that detects the visible light. The two-dimensional detector 142 includes a sensor in which pixels 20 that detect visible light are arranged in an array of X columns and Y rows, and outputs image information according to the detected radiation dose.

[0026] An example of the pixel 20 will now be described with reference to Fig. 2. Fig. 2 shows an equivalent circuit diagram of an example of the pixel 20. The pixel 20 is provided with a photoelectric conversion element 201 and an output circuit unit 202. The photoelectric conversion element 201 may typically include a photodiode. The output circuit unit 202 is provided with an amplifier circuit unit 204, a clamp circuit unit 206, a sample-and-hold circuit unit 207, and a selection circuit unit 208.

[0027] The photoelectric conversion element 201 includes a charge accumulation section, which is connected to the gate of a MOS transistor 204a in the amplifier circuit section 204. The source of the MOS transistor 204a is connected to a current source 204c via a MOS transistor 204b. The MOS transistor 204a and the current source 204c form a source follower circuit. The MOS transistor 204b is an enable switch that turns on when an enable signal EN supplied to its gate becomes active level, putting the source follower circuit into an operating state.

[0028] In the example shown in FIG. 2, the charge storage section of the photoelectric conversion element 201 and the gate of the MOS transistor 204a form a common node. This node functions as a charge-voltage converter that converts the charge stored in the charge storage section of the photoelectric conversion element 201 into a voltage. A voltage V (=Q / C) appears in the charge-voltage converter, which is determined by the charge Q stored in the charge storage section and the capacitance C of the charge-voltage converter. The charge-voltage converter is connected to a reset potential Vres via a reset switch 203. When a reset signal PRES becomes active level, the reset switch 203 is turned on, and the potential of the charge-voltage converter is reset to the reset potential Vres.

[0029] The clamp circuit unit 206 clamps the noise output by the amplifier circuit unit 204 in response to the reset potential of the charge-voltage converter using a clamp capacitor 206a. In other words, the clamp circuit unit 206 is a circuit for canceling the noise described above from the signal output from the source follower circuit in response to the charge generated by photoelectric conversion in the photoelectric conversion element 201. This noise includes kTC noise generated during reset. Clamping is achieved by first setting the clamp signal PCL to an active level to turn on the MOS transistor 206b, and then setting the clamp signal PCL to an inactive level to turn off the MOS transistor 206b. The output side of the clamp capacitor 206a is connected to the gate of the MOS transistor 206c. The source of the MOS transistor 206c is connected to a current source 206e via a MOS transistor 206d. The MOS transistor 206c and the current source 206e form a source follower circuit. The MOS transistor 206d is an enable switch that turns on when an enable signal EN0 supplied to its gate becomes active, activating the source follower circuit.

[0030] A signal output from the clamp circuit unit 206 in response to charges generated by photoelectric conversion of the photoelectric conversion element 201 is written as an optical signal to the capacitor 207Sb via the switch 207Sa when the optical signal sampling signal TS becomes active. The signal output from the clamp circuit unit 206 when the MOS transistor 206b is turned on immediately after resetting the potential of the charge-voltage converter is a clamp voltage. The clamp voltage is written as noise to the capacitor 207Nb via the switch 207Na when the noise sampling signal TN becomes active. This noise includes an offset component of the clamp circuit unit 206. The switch 207Sa and the capacitor 207Sb form a signal sample-and-hold circuit 207S, and the switch 207Na and the capacitor 207Nb form a noise sample-and-hold circuit 207N. The sample-and-hold circuit unit 207 includes the signal sample-and-hold circuit 207S and the noise sample-and-hold circuit 207N.

[0031] When the driving circuit unit drives the row selection signal to an active level, the signal (optical signal) held in the capacitor 207Sb is output to the signal line 21S via the MOS transistor 208Sa and the row selection switch 208Sb. At the same time, the signal (noise) held in the capacitor 207Nb is output to the signal line 21N via the MOS transistor 208Na and the row selection switch 208Nb. The MOS transistor 208Sa forms a source follower circuit together with a constant current source (not shown) provided on the signal line 21S. Similarly, the MOS transistor 208Na forms a source follower circuit together with a constant current source (not shown) provided on the signal line 21N. The MOS transistor 208Sa and the row selection switch 208Sb form the signal selection circuit unit 208S, and the MOS transistor 208Na and the row selection switch 208Nb form the noise selection circuit unit 208N. The selection circuit section 208 includes a signal selection circuit section 208S and a noise selection circuit section 208N.

[0032] The pixel 20 may have an addition switch 209S that adds optical signals from multiple adjacent pixels 20. In addition mode, an addition mode signal ADD becomes active, and the addition switch 209S is turned on. As a result, the capacitances 207Sb of adjacent pixels 20 are connected to each other by the addition switch 209S, and the optical signals are averaged. Similarly, the pixel 20 may have an addition switch 209N that adds noises from multiple adjacent pixels 20. When the addition switch 209N is turned on, the capacitances 207Nb of adjacent pixels 20 are connected to each other by the addition switch 209N, and the noises are averaged. The adder 209 includes an addition switch 209S and an addition switch 209N.

[0033] The pixel 20 may have a sensitivity change unit 205 for changing the sensitivity. The pixel 20 may include, for example, a first sensitivity change switch 205a, a second sensitivity change switch 205'a, and associated circuit elements. When the first change signal WIDE becomes active, the first sensitivity change switch 205a is turned on, and the capacitance value of the first additional capacitor 205b is added to the capacitance value of the charge-voltage conversion unit. This reduces the sensitivity of the pixel 20. When the second change signal WIDE2 becomes active, the second sensitivity change switch 205'a is turned on, and the capacitance value of the second additional capacitor 205'b is added to the capacitance value of the charge-voltage conversion unit. This further reduces the sensitivity of the pixel 20. By adding a function to reduce the sensitivity of the pixel 20 in this way, it becomes possible to receive a larger amount of light and widen the dynamic range. When the first change signal WIDE becomes active, the enable signal ENw may be turned on to cause the MOS transistor 204'a to operate as a source follower instead of the MOS transistor 204a.

[0034] The radiation imaging device 104 reads the output of the pixel circuit as described above and converts it into digital values ​​(image information) using an AD converter (not shown). The radiation imaging device 104 transfers the image information converted into digital values ​​to the control device 103. This allows the acquisition unit 131 of the control device 103 to acquire the radiographed image.

[0035] (Radiation imaging operation for ES processing) Next, before describing the processing according to this embodiment, a radiation imaging operation for performing energy subtraction (ES) processing and the ES processing will be described with reference to FIGS.

[0036] Figures 3 and 4 show examples of various drive timings in radiography operations for performing ES processing in a radiographic imaging system. Figure 3 shows an example of radiography operations when a relatively inexpensive tube that does not allow switching of tube voltage (energy) is used, while Figure 4 shows an example of radiography operations when a tube that does allow switching of tube voltage is used. The waveforms in the figures, with time on the horizontal axis, show the timing of X-ray exposure, synchronization signal, resetting of photoelectric conversion element 201, driving of sample-and-hold circuit unit 207, and readout of image from signal line 21. Note that in Figures 3 and 4, the waveform for X-rays represents tube voltage. Also, while black and white areas are provided for X-rays, this is simply done to make it easier to distinguish the timing.

[0037] First, the example shown in FIG. 3 will be described. In this example, X-rays are emitted after the photoelectric conversion element 201 is reset. Ideally, the X-ray tube voltage is a rectangular wave, but it takes a finite amount of time for the tube voltage to rise and fall. In particular, when the exposure time is short with pulsed X-rays, the tube voltage can no longer be considered a rectangular wave, and takes on a waveform like that shown in the X-ray waveform in FIG. 3. For this reason, the energy of the X-rays differs during the rise, stable, and fall periods of the X-rays.

[0038] Therefore, after X-rays 301 in the rising phase are irradiated, sampling is performed by the noise sample and hold circuit 207N, and further, after X-rays 302 in the stable phase are irradiated, sampling is performed by the signal sample and hold circuit 207S. Thereafter, the difference between the signal on signal line 21N and the signal on signal line 21S is read out as an image. At this time, the signal (R1) of X-rays 301 in the rising phase is held in the noise sample and hold circuit 207N, and the sum of the signal (R1) of X-rays 301 in the rising phase and the signal (B) of X-rays 302 in the stable phase is held in the signal sample and hold circuit 207S. Therefore, an image 304 corresponding to the signal (B) of X-rays 302 in the stable phase is read out as the difference between the signal on signal line 21N and the signal on signal line 21S.

[0039] Next, after the irradiation of X-rays 303 in the falling phase and the readout of image 304 are completed, sampling is again performed by signal sample-and-hold circuit 207S. Thereafter, photoelectric conversion element 201 is reset, and sampling is again performed by noise sample-and-hold circuit 207N, and the difference between the signal on signal line 21N and the signal on signal line 21S is read out as an image. At this time, the noise sample-and-hold circuit 207N holds a signal in a state where no X-rays are irradiated. Furthermore, the signal sample-and-hold circuit 207S holds the sum of the signal (R1) of X-rays 301 in the rising phase, the signal (B) of X-rays 302 in the stable phase, and the signal (R2) of X-rays 303 in the falling phase. Therefore, image 306, which corresponds to the sum of the signal (R1) of X-rays 301 in the rising phase, the signal (B) of X-rays 302 in the stable phase, and the signal (R2) of X-rays 303 in the falling phase, is read out as the difference between the signal on signal line 21N and the signal on signal line 21S. Thereafter, by calculating the difference between image 306 and image 304, image 305 corresponding to the sum of the signal (R1) of X-ray 301 in the rising phase and the signal (R2) of X-ray 303 in the falling phase is obtained.

[0040] The timing for resetting the sample-and-hold circuit unit 207 and the photoelectric conversion element 201 is determined using a synchronization signal 307 indicating that X-ray exposure has started from the radiation generation device 101. A method for detecting the start of radiation exposure may include measuring the tube current of the radiation generation device 101 and determining whether the current value exceeds a predetermined threshold. Alternatively, a configuration may be used in which, after resetting of the photoelectric conversion element 201 is completed, signals from the pixels 20 are repeatedly read out and whether the pixel values ​​exceed a predetermined threshold is determined. Furthermore, a configuration may be used in which an X-ray detector other than the two-dimensional detector 142 is built into the radiation imaging device 104 and whether the measured value exceeds a predetermined threshold is determined. In either case, sampling by the signal sample-and-hold circuit 207S, sampling by the noise sample-and-hold circuit 207N, and resetting of the photoelectric conversion element 201 are performed after a predetermined time has elapsed since the input of the synchronization signal 307.

[0041] In this way, it is possible to obtain an image 304 corresponding to the stable period of the pulsed X-rays and an image 305 corresponding to the sum of the rising and falling periods of the pulsed X-rays. Because the energy of the X-rays irradiated when generating these two images is different, energy subtraction processing can be performed by performing calculations between the two images.

[0042] Next, an example of radiation imaging operation when using a tube with switchable tube voltage will be described with reference to Fig. 4. This example differs from the example shown in Fig. 3 in that the X-ray tube voltage is actively switched.

[0043] In this example, first, the photoelectric conversion element 201 is reset, and then low-energy X-rays 401 are irradiated. Then, sampling is performed by the noise sample-and-hold circuit 207N, and the tube voltage is switched to irradiate high-energy X-rays 402. After the high-energy X-rays 402 are irradiated, sampling is performed by the signal sample-and-hold circuit 207S. Then, the tube voltage is switched to irradiate low-energy X-rays 403. Furthermore, the difference between the signals on the signal line 21N and the signal line 21S is read out as an image. At this time, the noise sample-and-hold circuit 207N holds the signal (R1) of the low-energy X-rays 401, and the signal sample-and-hold circuit 207S holds the sum of the signal (R1) of the low-energy X-rays 401 and the signal (B) of the high-energy X-rays 402. Therefore, an image 404 corresponding to the signal (B) of the high-energy X-ray 402 is read out as the difference between the signal on the signal line 21N and the signal on the signal line 21S.

[0044] Next, after the irradiation of low-energy X-rays 403 and the readout of image 404 are completed, sampling is again performed by signal sample-and-hold circuit 207S. Thereafter, photoelectric conversion element 201 is reset, and sampling is again performed by noise sample-and-hold circuit 207N, and the difference between the signal on signal line 21N and the signal on signal line 21S is read out as an image. At this time, the noise sample-and-hold circuit 207N holds a signal in a state where no X-rays are irradiated. Furthermore, the signal sample-and-hold circuit 207S holds the sum of the signal (R1) of low-energy X-rays 401, the signal (B) of high-energy X-rays 402, and the signal (R2) of low-energy X-rays 403. Therefore, an image 406 corresponding to the sum of the signal (R1) of low-energy X-rays 401, the signal (B) of high-energy X-rays 402, and the signal (R2) of low-energy X-rays 403 is read out as the difference between the signal on signal line 21N and the signal on signal line 21S. Thereafter, by calculating the difference between image 406 and image 404, image 405 corresponding to the sum of the signal (R1) of low-energy X-ray 401 and the signal (R2) of low-energy X-ray 403 is obtained.

[0045] The synchronization signal 407 is the same as in the example shown in Fig. 3. In this way, by acquiring images while actively switching the tube voltage, it is possible to make the energy difference between low-energy images and high-energy images larger than in the method of Fig. 3. The method of acquiring low-energy images and high-energy images is not limited to this. For example, an FPD including a stacked sensor formed of two types of materials (phosphors) with different X-ray absorption rates may be used as the radiation imaging device 104, and low-energy and high-energy images may be acquired based on the outputs from the sensors of each layer obtained by a single exposure.

[0046] (ES processing) Next, the ES processing method will be described. In addition to the signal processing of the ES processing, the ES processing can include correction processing as pre-processing and image processing as post-processing.

[0047] First, the correction process, which is pre-processing, will be described with reference to Fig. 5. Fig. 5 is a block diagram of the correction process. Note that in the following, this embodiment will be described assuming that a radiation imaging operation is performed in accordance with the example shown in Fig. 3.

[0048] 3 without irradiating the radiation imaging device 104 with X-rays, and the acquisition unit 131 acquires the captured images. At this time, two images corresponding to images 304 and 306 are read out, with the first image (image 304) being image F_Odd and the second image (image 306) being image F_Even. Images F_Odd and F_Even are images corresponding to fixed pattern noise (FPN) of the radiation imaging device 104.

[0049] Next, in a state where there is no subject, the radiation imaging device 104 is irradiated with X-rays to perform imaging using the driving method shown in Fig. 3, and the acquisition unit 131 acquires the captured images. At this time, two images corresponding to images 304 and 306 are read out, with the first image (image 304) being image W_Odd and the second image (image 306) being image W_Even. Images W_Odd and W_Even are images corresponding to the sum of signals due to the FPN of the radiation imaging device 104 and X-rays.

[0050] Therefore, by subtracting image F_Odd from image W_Odd and image F_Even from image W_Even, images WF_Odd and WF_Even can be obtained from which the FPN of the radiation imaging device 104 has been removed. In this embodiment, such correction processing is called offset correction.

[0051] Image WF_Odd is an image corresponding to X-rays 302 in the stable period, and image WF_Even is an image corresponding to the sum of X-rays 301 in the rising period, X-rays 302 in the stable period, and X-rays 303 in the falling period. Therefore, by subtracting image WF_Odd from image WF_Even, an image corresponding to the sum of X-rays 301 in the rising period and X-rays 303 in the falling period is obtained. The energies of X-rays 301 in the rising period and X-rays 303 in the falling period are lower than the energy of X-rays 302 in the stable period. Therefore, by subtracting image WF_Odd from image WF_Even, a low-energy image W_Low in the absence of an object is obtained. Furthermore, a high-energy image W_High in the absence of an object is obtained from image WF_Odd. In this embodiment, this correction process is called color correction.

[0052] Next, with a subject present, the radiation imaging device 104 is irradiated with X-rays to perform imaging using the driving method shown in Fig. 3, and the acquisition unit 131 acquires the captured images. At this time, two images corresponding to images 304 and 306 are read out, with the first image (image 304) designated as image X_Odd and the second image (image 306) designated as image X_Even. The generation unit 132 performs offset correction and color correction on these images in the same manner as when there is no subject, thereby obtaining a low-energy image X_Low when there is a subject and a high-energy image X_High when there is a subject.

[0053] Here, if the thickness of the subject is d, the linear attenuation coefficient of the subject is μ, the output of pixel 20 when there is no subject is I0, and the output of pixel 20 when there is a subject is I, the following equation (1) holds.

number

number

[0054] Therefore, by dividing the low-energy image X_Low when there is an object by the low-energy image W_Low when there is no object, the image of the attenuation rate at low energy (low-energy image Im L ) is obtained. Similarly, by dividing the high-energy image X_High when there is an object by the high-energy image W_High when there is no object, the image H of the attenuation rate at high energy (high-energy image Im H Hereinafter, such a correction process will be referred to as gain correction. In the radiation imaging system, the generation unit 132 performs correction processes including the offset correction, color correction, and gain correction as described above, thereby obtaining a low-energy image Im L and high-energy image Im H can be generated and obtained.

[0055] Next, the signal processing of the ES processing will be described with reference to Fig. 6. Fig. 6 is a block diagram of the signal processing of the ES processing. In the signal processing of the ES processing, the low-energy image Im obtained by the correction processing described with reference to Fig. L and high energy images H From the bone thickness image (bone image Im B ) and soft tissue thickness image (soft tissue image Im S ) is found.

[0056] First, let E be the energy of the X-ray photon, N(E) be the number of photons at energy E, B be the thickness of the bone, and S be the thickness of the soft tissue. Let μ be the linear attenuation coefficient of the bone at energy E. B (E), the linear attenuation coefficient of soft tissue at energy E is μ S (E), if the attenuation ratio is I / I0, the following equation (3) holds.

number

[0057] The number of photons N(E) at energy E is the X-ray spectrum. The X-ray spectrum can be obtained by simulation or actual measurement for the tube voltage. Also, the linear attenuation coefficient μ of bone at energy E isB (E) and the linear attenuation coefficient μ of soft tissue at energy E S (E) can be obtained from databases such as those of the National Institute of Standards and Technology (NIST). Therefore, by using equation (3), it is possible to calculate the attenuation ratio I / I0 for any bone thickness B, soft tissue thickness S, and X-ray spectrum N(E).

[0058] Here, the spectrum of low energy X-rays is N L (E) High-energy X-ray spectrum H If (E), the following equation (4) holds.

number

[0059] Here, we will explain the case where the Newton-Raphson method, which is a type of iterative solution, is used as a typical method for solving nonlinear simultaneous equations. First, let us define the number of iterations of the Newton-Raphson method as m, and the bone thickness after the mth iteration as B. m , the thickness of the soft tissue after the mth iteration is S m Then, the attenuation rate of high energy after the mth iteration, H m , and the low-energy attenuation rate L after the mth iteration m is expressed by the following equation (5).

number

number

[0060] At this time, the bone thickness B after the m+1th iteration m+1 and soft tissue thickness S m+1 is expressed by the following equation (7) using the attenuation rate of high energy H and the attenuation rate of low energy L.

number

number

number

[0061] By repeating this calculation, the attenuation rate of high energy after the mth iteration, H m The difference between the measured high-energy attenuation rate H and the measured high-energy attenuation rate L approaches 0. The same is true for the low-energy attenuation rate L. As a result, the bone thickness B after the mth iteration m converges to the bone thickness B, and the m-th soft tissue thickness S m converges to the thickness S of the soft tissue. In this way, the nonlinear simultaneous equations shown in equation (4) can be solved. Therefore, by calculating equation (4) for all pixels, the low-energy image Im L and high energy images H From the bone image B and soft tissue imaging S can be obtained.

[0062] In this example, for the sake of simplicity, the bone thickness B and the soft tissue thickness S are calculated by ES processing, but this embodiment is not limited to this. For example, the thickness of water and the thickness of a contrast agent may be calculated by ES processing. In this case, the linear attenuation coefficient of water at energy E and the linear attenuation coefficient of a contrast agent at energy E may also be obtained from a database such as NIST. The ES processing makes it possible to calculate the thickness of any two types of material. Furthermore, an image of the effective atomic number Z and an image of the surface density D may be obtained from an image of the attenuation rate L at low energy and an image of the attenuation rate H at high energy obtained by the correction shown in FIG. 5. The effective atomic number Z is the equivalent atomic number of the mixture, and the surface density D is the density [g / cm] of the subject. 3 ] and the thickness of the subject [cm].

[0063] In the above description, the nonlinear simultaneous equations are solved using the Newton-Raphson method. However, the method for solving the nonlinear simultaneous equations is not limited to this. For example, iterative methods such as the least squares method and bisection method may also be used. Furthermore, the method for calculating the bone thickness B and the soft tissue thickness S is not limited to solving the nonlinear simultaneous equations using an iterative method. For example, the bone thickness B and the soft tissue thickness S for various combinations of high-energy attenuation rate H and low-energy attenuation rate L may be calculated in advance to generate a table, and the bone thickness B and the soft tissue thickness S may be calculated quickly by referring to the table.

[0064] Next, image processing for generating a total thickness image and a virtual monochromatic X-ray image in post-processing of such ES processing will be described. First, the processing for generating a total thickness image will be described with reference to FIG. 7. FIG. 7 is a block diagram of image processing for generating a total thickness image. In the image processing shown in FIG. 7, the image of bone thickness (bone image Im B ) and soft tissue thickness image (soft tissue image Im S ) and the sum of these images is the total thickness image Im T can be obtained.

[0065] In the human body, bones are surrounded by soft tissue, so the thickness of the soft tissue in the area where the bone is located is reduced by the thickness of the bone. S When gradation processing is applied to the image, the soft tissue image Im S On the other hand, bone contrast appears in the soft tissue image Im S and bone image Im B Adding , backfills the loss of bone thickness in the soft tissue image. Therefore, the total thickness image Im T is an image in which the contrast of bones has been removed. T A time-direction filter such as a recursive filter or a space-direction filter such as a Gaussian filter may be applied to the image to reduce noise, and then the image may be subjected to gradation processing and displayed.

[0066] In the following, energy subtraction processing (ES processing) refers to processing for calculating differences between images (energy images) related to different radiation energies to obtain, for example, material decomposition images of bone and soft tissue, contrast agent and water, etc., information on effective atomic number and surface density, etc. The energy subtraction processing may also include, for example, correction processing such as offset correction processing as pre-processing, and image processing such as contrast adjustment processing and total thickness image generation processing as post-processing.

[0067] Next, image processing for generating a virtual monochromatic X-ray image will be described with reference to Fig. 8. Fig. 8 is a block diagram of image processing for generating a virtual monochromatic X-ray image. In the image processing shown in Fig. 8, the bone image Im obtained by the signal processing shown in Fig. 6 is B and soft tissue imaging Im S Here, a virtual monochromatic X-ray image is an image that is assumed to be obtained when X-rays of a single energy are irradiated.

[0068] Virtual monochromatic X-ray images are used in Dual Energy CT, which combines ES processing and three-dimensional reconstruction. Virtual monochromatic X-ray images can suppress beam hardening artifacts and metal artifacts. For example, when the energy of virtual monochromatic X-rays is set to E V Then, the virtual monochromatic X-ray image V is calculated by the following equation (10).

number

[0069] In addition, the energy E of the virtual monochromatic X-ray V By changing the linear attenuation coefficient μ of bone, the CNR (contrast to noise ratio) of the virtual monochromatic X-ray image can be improved. B (E) is the linear attenuation coefficient μ of soft tissue S (E). However, the energy E V The larger the difference between the linear attenuation coefficient μB(E) of bone and the linear attenuation coefficient μS(E) of soft tissue becomes smaller. Therefore, the energy E of the virtual monochromatic X-ray V By setting a large value, the increase in noise in the virtual monochromatic X-ray image due to noise in the bone image is suppressed. On the other hand, the virtual monochromatic X-ray energy E V The smaller the linear attenuation coefficient μ of bone, B (E) and the linear attenuation coefficient μ of soft tissue S The contrast of the virtual monochromatic X-ray image increases because the difference in energy E V By setting an appropriate value, the CNR of the virtual monochromatic X-ray image can be improved, and for example, the amount of contrast agent used in radiography can be reduced.

[0070] In this embodiment, a virtual monochromatic X-ray image is generated from the bone thickness B and the soft tissue thickness S, but the present disclosure is not limited to this form. After calculating the effective atomic number Z and the surface density D by ES processing, a virtual monochromatic X-ray image may be generated using the effective atomic number Z and the surface density D. In addition, when a plurality of energies E VA composite X-ray image may be generated by combining multiple virtual monochromatic X-ray images generated in step 1. A composite X-ray image is an image that is expected to be obtained when irradiating an object with X-rays of a given spectrum.

[0071] (Learning process) In the above description, bone images and soft tissue images are generated by ES processing using high-energy images and low-energy images. In contrast, in this embodiment, a machine learning model is used to generate a total thickness image based on a radiographic image, which is an input image, and an ES image of a type different from the total thickness image is generated based on the total thickness image and the input image using an approximation formula for ES processing. Here, the learning process of the machine learning model according to this embodiment will be described with reference to Figs. 9 to 12. Fig. 9 is a flowchart showing an example of the learning process of the machine learning model according to this embodiment.

[0072] In the learning process of the machine learning model according to this embodiment, learning data is prepared in step S901. In the learning data according to this embodiment, radiation images such as high-energy images or low-energy images are used as input data, and total thickness images are used as output data. Note that either high-energy images or low-energy images may be used as input data for the learning data. In the following, an example will be described in which high-energy images are used as input data and total thickness images are used as output data.

[0073] As shown in FIGS. 1 to 6, the training data can be prepared by acquiring captured high-energy images and low-energy images, performing ES processing to acquire bone images and soft-tissue images, and generating a total thickness image by adding the bone images and soft-tissue images. More specifically, the acquisition unit 131 acquires the captured high-energy images and low-energy images. Then, the processing unit 133 performs ES processing on the high-energy images and low-energy images to generate bone images and soft-tissue images. The processing unit 133 also adds the generated bone images and soft-tissue images to generate a total thickness image. This allows the control device 103 to acquire radiographic images to be used as input data for the training data and a total thickness image to be used as output data for the training data.

[0074] Alternatively, a virtual monochromatic X-ray image may be generated from the bone image and the soft tissue image, the virtual monochromatic X-ray image may be used as input data for the learning data, and the total thickness image may be used as output data for the learning data. In this case, the processing unit 133 may generate the virtual monochromatic X-ray image from the bone image and the soft tissue image.

[0075] In the above description, it is assumed that a high-energy image and a low-energy image are acquired and then energy subtraction processing is performed to obtain a bone image and a soft tissue image. However, the present disclosure is not limited to such a configuration.

[0076] The acquisition unit 131 may acquire high-energy images, low-energy images, bone images, soft-tissue images, total thickness images, and virtual monochromatic X-ray images from an external device such as a server (not shown) connected to the control device 103. Furthermore, the processing unit 133 may generate bone images and soft-tissue images from the high-energy images and low-energy images acquired from the external device by the acquisition unit 131. Similarly, the processing unit 133 may generate total thickness images and virtual monochromatic X-ray images from the bone images and soft-tissue images acquired from the external device by the acquisition unit 131. Furthermore, the processing unit 133 may perform thinning or binning on the images acquired by the acquisition unit 131 to generate images with reduced resolution, and include the generated images in the learning data.

[0077] Alternatively, images for training data may be prepared by obtaining two-dimensional images by forward projection from three-dimensional data acquired by CT (Computed Tomography). For example, after calculating the ratio of bone to soft tissue from the CT value, two-dimensional bone images and soft tissue images can be obtained separately by forward projection.

[0078] As an example, the acquisition unit 131 can acquire three-dimensional data obtained by CT from an external device. Then, the processing unit 133 acquires three-dimensional data of bone and three-dimensional data of soft tissue based on the ratio of bone to soft tissue calculated from the CT value for the acquired three-dimensional data. Furthermore, the processing unit 133 can perform a known forward projection process on these to acquire two-dimensional bone images and soft tissue images. Then, the processing unit 133 can generate a total thickness image based on the acquired bone images and soft tissue images. The acquisition unit 131 may also acquire bone images and soft tissue images based on the three-dimensional data obtained by CT from an external device. Note that these processes are merely examples, and the control device 103 may acquire bone images and soft tissue images based on the three-dimensional data obtained by CT using any known process.

[0079] In this case, a virtual monochromatic X-ray image can be generated from the bone image and soft tissue image obtained by forward projection using the method shown in Fig. 8, and the pair of the virtual monochromatic X-ray image and the total thickness image can be used as learning data. In this case as well, the processing unit 133 can generate a virtual monochromatic X-ray image from the bone image and soft tissue image obtained by forward projection.

[0080] In the following, the term "energy subtraction image (ES image)" can include, for example, a material decomposition image obtained using ES processing, an image showing effective atomic number and surface density, etc. Furthermore, the term "ES image" can also include a material decomposition image obtained using an approximation formula for ES processing described below, a radiation image of any energy, etc. Furthermore, the term "ES image" can also include an image inferred using a trained model obtained by training an image obtained using the above-mentioned ES processing, a material decomposition image obtained using three-dimensional CT data, etc. Furthermore, in the following, the term "ES image" can also include a total thickness image generated by adding together material decomposition images such as bone images and soft tissue images.

[0081] In this embodiment, an example will be described in which a total thickness image is used as output data of the training data, but the configuration of the training data is not limited to this. ES images, such as material decomposition images of bone images and soft tissue images, can also be used as output data of the training data. Therefore, the training data may be, for example, a pair of a high-energy image and a bone image, a pair of a low-energy image and a bone image, or a pair of a virtual monochromatic X-ray image and a bone image. Similarly, the training data may be, for example, a pair of a high-energy image and a soft-tissue image, a pair of a low-energy image and a soft-tissue image, or a pair of a virtual monochromatic X-ray image and a soft-tissue image.

[0082] Next, in step S902, the processing unit 133 performs preprocessing of the training data. Machine learning requires a huge amount of data. If the number of data pieces in the training data prepared in step S901 is insufficient, the processing unit 133 can increase the training data by data expansion. Methods of data expansion that can be used include clipping, which extracts a portion of an image, and rotating, scaling, and flipping an image.

[0083] Alternatively, the processing unit 133 may perform data expansion using an image generation AI. For example, the image generation AI can be trained using bone images and soft tissue images obtained by imaging and ES processing or forward projection of CT three-dimensional data, and new bone images and soft tissue images can be generated using the image generation AI. In this case, virtual monochromatic X-ray images and total thickness images can be generated from the new bone images and soft tissue images obtained by the image generation AI, and the pair of virtual monochromatic X-ray image and total thickness image can be used as training data. Note that when bone images and soft tissue images are used as output data of the training data, the new bone images and soft tissue images obtained by the image generation AI can be used as output data of the training data.

[0084] Next, in step S903, a learning process for the machine learning model is performed. Fig. 10 shows a block diagram of the machine learning according to this embodiment. The ES image generation model according to the present disclosure is a trained model generated by training (learning) based on a machine learning algorithm.

[0085] Here, we will briefly explain a general trained model. A trained model is a machine learning model that has been trained (learned) in advance using appropriate training data for an arbitrary machine learning algorithm. The training data consists of one or more pairs of input data and output data (ground truth data). The format and combination of input data and output data in the pairs that make up the training data may be suitable for a desired configuration, such as one being an image and the other being a numerical value, one being composed of a group of multiple images and the other being a character string, or both being images. Furthermore, a machine learning algorithm includes a method or system for performing training.

[0086] As shown in Fig. 10, in this embodiment, training data is loaded into a computer, and features such as regularities and patterns inherent in the training data are trained based on a machine learning algorithm to generate a trained model. Here, the machine learning algorithm includes deep learning techniques such as convolutional neural networks (CNNs). In deep learning techniques, different parameter settings for layers and nodes constituting a neural network may affect the degree to which trends trained using the training data can be reproduced in inference data.

[0087] Specifically, parameters in a CNN can include, for example, the filter kernel size, number of filters, stride value, and dilation value set for the convolution layer, as well as the number of nodes output by the fully connected layer. The parameter set and the number of training epochs can be set to values ​​suitable for the usage of the trained model based on the training data. For example, the parameter set and the number of epochs can be set based on the training data to enable the output of ES images such as total thickness images with higher accuracy.

[0088] Here is an example of one method for determining such parameter sets and the number of epochs. First, 70% of the pairs constituting the learning data are used for training, and the remaining 30% are randomly set as those for evaluation. Next, the training pairs are used to train the machine learning model, and at the end of each training epoch, the evaluation pairs are used to calculate a training evaluation value. The training evaluation value is, for example, the average value of a group of values ​​obtained by evaluating the output when input data constituting each pair is input to the machine learning model being trained, and the output data corresponding to the input data, using a loss function. Finally, the parameter set and the number of epochs that result in the smallest training evaluation value are determined as the parameter set and the number of epochs for the machine learning model.

[0089] In this way, by dividing the pairs that make up the learning data into those for training and those for evaluation and determining the number of epochs, it is possible to prevent the machine learning model from overfitting the training pairs.

[0090] An example of the configuration of a CNN related to an ES image generation model according to this embodiment will be described below with reference to Fig. 11. Fig. 11 shows an example of the configuration of an ES image model. The configuration shown in Fig. 11 is composed of a plurality of layer groups that are responsible for processing and outputting a group of input values. As shown in Fig. 11, the types of layers included in this configuration include a convolution layer, a downsampling layer, an upsampling layer, and a merging layer.

[0091] The convolution layer is a layer that performs convolution processing on a group of input values ​​according to parameters such as the set filter kernel size, the number of filters, the stride value, the dilation value, etc. Note that the number of dimensions of the filter kernel size may also be changed depending on the number of dimensions of the input image.

[0092] The downsampling layer is a layer that performs processing to reduce the number of output value groups to be less than the number of input value groups by thinning out or combining input value groups. Specifically, such processing includes, for example, max pooling processing.

[0093] An upsampling layer is a layer that performs processing to make the number of output values ​​greater than the number of input values ​​by duplicating input values ​​or adding values ​​interpolated from the input values, such as linear interpolation.

[0094] A synthesis layer is a layer that inputs a group of values, such as a group of output values ​​from a layer or a group of pixel values ​​that make up an image, from multiple sources and performs processing to combine them by concatenating or adding them.

[0095] In this configuration, the pixel values ​​constituting the input image Im1101 are output through a convolution processing block and then combined in a composition layer with the pixel values ​​constituting the input image Im1101. The combined pixel values ​​are then shaped into an ES image Im1102 in the final convolution layer.

[0096] It should be noted that different parameter settings for the layers and nodes that make up the neural network may affect the degree to which trends trained from learning data can be reproduced during inference. In other words, in many cases, appropriate parameters differ depending on the implementation form, so they can be changed to preferred values ​​as needed.

[0097] In addition to changing the parameters as described above, changing the configuration of the CNN can sometimes result in better CNN characteristics, such as outputting more accurate ES images, shortening processing time, or shortening the time required to train a machine learning model.

[0098] The CNN used in this embodiment is a U-net type machine learning model having an encoder function consisting of multiple layers including multiple downsampling layers and a decoder function consisting of multiple layers including multiple upsampling layers. That is, the CNN has a U-shaped structure having an encoder function and a decoder function. The U-net type machine learning model is configured (for example, using skip connections) so that position information (spatial information) obscured in multiple layers configured as encoders can be used in layers of the same dimension (layers corresponding to each other) in multiple layers configured as decoders.

[0099] Although not shown in the figure, as an example of a modification to the CNN configuration, for example, a batch normalization layer or an activation layer using a rectifier linear unit may be incorporated after the convolution layer.

[0100] The ES image generation model of this embodiment is configured to be stored in the storage unit 135 and executed by the processing unit 133, but these may also be provided in an external device (not shown) connected to the control device 103.

[0101] Here, a GPU can perform efficient calculations by processing a larger amount of data in parallel. Therefore, when performing learning multiple times using a machine learning algorithm such as deep learning, it is effective to use a GPU for processing. Therefore, in this embodiment, a GPU is used in addition to a CPU for processing by the processing unit 133, which functions as an example of a learning unit. Specifically, when executing a learning program including a learning model, the CPU and GPU work together to perform calculations to perform learning. Note that calculations may be performed only by the CPU or the GPU in the processing of the learning unit. Furthermore, the ES processing according to this embodiment may also be realized using a GPU, as with the learning unit. Note that if the trained model is provided in an external device, the processing unit 133 does not need to function as a learning unit.

[0102] The learning unit may also include an error detection unit and an update unit (not shown). The error detection unit obtains the error between correct data and output data output from the output layer of the neural network in response to input data input to the input layer. The error detection unit may use a loss function to calculate the error between the output data from the neural network and correct data. The update unit updates the connection weighting coefficients between the nodes of the neural network based on the error obtained by the error detection unit so as to reduce the error. This update unit updates the connection weighting coefficients using, for example, an error backpropagation method. The error backpropagation method is a technique for adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the error.

[0103] FIG. 12 shows the configuration of training data according to this embodiment. In this embodiment, both input and output data of the training data are image data. As described above, the input data of the training data are radiological images such as high-energy images, low-energy images, or virtual monochromatic X-ray images, and the output data are ES images such as total thickness images, bone images, or soft tissue images. As shown in FIG. 12, the training data is collective data including a plurality of these image pairs (training data 1 to n). Note that the training data according to this embodiment may include images generated by the data augmentation performed in step S902.

[0104] By using the trained model trained in this way, when a radiographic image is input to the trained model, the processing unit 133 can acquire an ES image such as a total thickness image corresponding to the input radiographic image. Furthermore, by constructing a trained model using such training data, many nonlinear arithmetic processes included in the ES processing can be included in inference using a machine learning algorithm such as deep learning.

[0105] (Image generation processing using a trained model) Next, with reference to Fig. 13, an image generation process using the above-described trained model used in the image processing according to this embodiment will be described. Fig. 13 is a flowchart showing an example of the image generation process using the trained model. First, in step S1301, the acquisition unit 131 acquires a high-energy image acquired by normal imaging using the radiation imaging device 104. Note that the acquisition unit 131 may also acquire the high-energy image from an external device connected to the control device 103. Furthermore, the acquired radiographic image may be any radiographic image that corresponds to the input data of the training data, and may be, for example, a low-energy image.

[0106] Next, in step S1302, the processing unit 133 inputs the acquired high-energy image into the trained model and acquires a total thickness image output from the trained model. Here, FIG. 14 is a block diagram of inference processing using the trained model. When target data (input data) is input to the trained model, inference data according to the design of the trained model is output. At this time, the trained model outputs inference data that is likely to correspond to the target data, for example, according to a trend trained using the training data. The trained model according to this embodiment outputs a total thickness image that is likely to correspond to the input high-energy image, according to a trained trend.

[0107] The combination of target data and inference data corresponds to the combination of input data and output data of the training data. Therefore, the target data may be a radiological image such as a high-energy image, a low-energy image, or an energy image obtained by imaging using the voltage of a virtual monochromatic X-ray image as the tube voltage during imaging, according to the input data of the training data. Furthermore, the inference data may be an ES image such as a total thickness image, a bone image, or a soft tissue image, according to the output data of the training data.

[0108] Here, with reference to FIGS. 15(a) and 15(b), the relationship between the input data and output data of the training data and the target data and inference data during inference will be described. FIG. 15(a) is a block diagram of a training process using exemplary training data, and FIG. 15(b) is a block diagram of an inference process using exemplary target data and inference data. As described above, in this embodiment, the input data and target data of the training data are high-energy images, and the output data of the training data and the inference data of the trained model are total thickness images. As shown in FIG. 15(a), the trained model trained using high-energy images and total thickness images as training data trains features such as regularity and regularity inherent in the high-energy images and total thickness images. Therefore, as shown in FIG. 15(b), when a high-energy image is input as target data, the trained model can output a total thickness image as inference data that is likely to correspond to the target data according to the training trend.

[0109] As explained with reference to Figure 7, bones in the human body are surrounded by soft tissue, and when a soft-tissue image and a bone image are added together, the reduced bone thickness in the soft-tissue image is refilled. Therefore, the total thickness image obtained by adding the soft-tissue image and the bone image is an image from which bone contrast has been removed, in other words, an image with less subject structure. Machine learning models are trained on the features of the training data, but when training is performed using data containing complex features, unnecessary artifacts due to the complex features may occur in the inference data. In contrast, training a machine learning model using images with less subject structure can reduce artifacts due to the subject structure that may occur in the inference data. Therefore, when a high-energy image captured as target data is input to a trained model trained using a high-energy image and a total thickness image as output data, a total thickness image with reduced artifacts can be output as inference data.

[0110] According to the processing of step S1302, an ES image can be generated from a single radiation image, such as a high-energy image or a low-energy image, using an ES image generation model based on machine learning. Therefore, when the control device 103 that performs such processing acquires an ES image using a trained model, there is no need to capture two images, a high-energy image and a low-energy image.

[0111] In contrast, in general ES processing, bone images and soft tissue images are generated using high-energy images and low-energy images. In order to acquire high-energy images and low-energy images with little time lag, such processing requires a detector equipped with a sample-and-hold circuit as shown in Figure 2 and an X-ray source capable of switching tube voltage as shown in Figure 4. This requires large-scale hardware modifications to the radiation imaging device 104 and the radiation generator 101.

[0112] In contrast, the processing in step S1302 can be performed using a single radiographic image, which eliminates the need for the hardware modifications described above to the radiographic imaging system used for inference, thereby reducing the cost of the radiographic imaging system. Furthermore, the processing in step S1302 can be performed using a single radiographic image, which can also reduce the radiation exposure of the subject associated with capturing the radiographic image. Furthermore, since processing can be performed using a single radiographic image, there is an advantage in that motion artifacts due to misalignment that occur when processing is performed using two radiographic images are eliminated.

[0113] Next, in step S1303, the processing unit 133 performs post-processing on the ES image acquired in step S1302. Specifically, the processing unit 133 uses an approximate equation for ES processing to acquire an ES image such as a bone image, a soft tissue image, or a radiographic image of any energy based on the high-energy image used as input data and the total thickness image acquired using the trained model. FIG. 16 is a block diagram showing an example of post-processing using the approximate equation for ES processing. In the example shown in FIG. 16, the processing unit 133 performs logarithmic transformation on the radiographic image used as input data. Furthermore, the processing unit 133 generates an ES image of a type different from the total thickness image by performing weighted addition using the approximate equation for ES processing using the logarithm of the radiographic image and the total thickness image acquired using the trained model.

[0114] In the explanation so far, the ES image generation model has learned the relationship between a radiographic image and a total thickness image, a radiographic image and a bone image, or a radiographic image and a soft tissue image. In such a configuration, an ES image generation model must be prepared for each ES image to be generated. However, the relationship shown in equation (4) holds for the pixel value X of the radiographic image, the bone thickness B, and the soft tissue thickness S. Here, for simplicity, the spectrum N(E) of radiation when a radiographic image is acquired at any energy is expressed as the average energy E X Approximating this to a virtual monochromatic X-ray with a peak at , it is expressed by the following equation (11).

number

[0115] From equations (4) and (11), the pixel value X of a radiographic image acquired at any energy is calculated by the average energy E X The linear attenuation coefficient μ of bone in XB and the average energy E X The linear attenuation coefficient μ of soft tissue in XS Using this, it is expressed by the following equation (12).

number

[0116] Furthermore, the total thickness T is the sum of the bone thickness B and the soft tissue thickness S. Therefore, the bone thickness B and the soft tissue thickness S are expressed using the total thickness T by the following equation (13).

number

[0117] For simplicity, the spectrum N(E) of radiation when a radiological image is acquired at any energy is expressed as the average energy E Y In this case, the pixel value Y of a radiation image of any energy can be approximated by E Y The linear attenuation coefficient μ of bone in YB And, E Y The linear attenuation coefficient μ of soft tissue in YS Using this, it is expressed by the following equation (14).

number

[0118] Equation (13) multiplies the logarithm of the radiographic image and the total thickness image by a coefficient determined by the linear attenuation coefficient, and then adds them together. The same applies to equation (14). Therefore, as shown in Figure 16, ES images such as bone images, soft tissue images, virtual monochromatic X-ray images, and radiographic images of any energy can be obtained by weighted addition of the logarithm of the radiographic image and the total thickness image.

[0119] Furthermore, in this embodiment, a configuration was assumed in which the logarithms of the total thickness image generated by the ES image generation model and the radiographic image used as input data were weighted and added. However, this embodiment is not limited to such a configuration. As shown in FIG. 17 , a configuration may be adopted in which the logarithms of the bone image and the radiographic image generated by the ES image generation model are weighted and added to obtain ES images such as soft tissue images and total thickness images. In this case, the pixel values ​​S of the soft tissue image and the pixel values ​​T of the total thickness image are calculated using the logarithms of the bone image and the radiographic image as shown in the following equation (15).

number

[0120] Similarly, as shown in Fig. 18, a configuration may be adopted in which ES images such as bone images and total thickness images are obtained by weighted addition of the logarithms of the soft tissue image and radiological image generated by the ES image generation model. In this case, the method for calculating the pixel value B of the bone image and the pixel value T of the total thickness image using the logarithms of the soft tissue image and radiological image is as shown in the following equation (16).

number

[0121] Each of equations (13) to (16) is a weighted addition of the logarithm of the radiographic image and the ES image generated by the ES image generation model. Furthermore, the ES image obtained by this weighted addition is a different type of ES image from the ES image generated by the ES image generation model. Therefore, in the post-processing in step S1303, a different type of ES image from the ES image generated by the ES image generation model can be generated by weighting the logarithm of the radiographic image and the ES image generated by the ES image generation model.

[0122] In the above process, for simplicity, the radiation spectrum N(E) when a radiological image is acquired at any energy is expressed as the average energy E XThe ES image was approximated using virtual monochromatic X-rays with a peak at 1000 MHz. However, the inventors of the present application have found that calculations using approximation using virtual monochromatic X-rays may not result in accurate ES images. Therefore, a method for generating an ES image of a different type from the ES image generated by the ES image generation model without approximation using virtual monochromatic X-rays will be described.

[0123] The pixel value X of a radiographic image is calculated by the linear attenuation coefficient μ of bone at energy E. B (E) and the linear attenuation coefficient μ of soft tissue at energy E S Using (E), this is expressed as the following equation (17).

number

[0124] By solving the nonlinear equation of Equation (17), the thickness S of the soft tissue can be calculated from the bone thickness B and the pixel value X of the radiographic image. The method for solving Equation (17) may be the Newton-Raphson method, or an iterative method such as the least squares method or bisection method. When the Newton-Raphson method is used, for example, the value obtained by the approximation of Equation (17) can be used as the initial value. Alternatively, a table may be generated by calculating the soft tissue thickness S for various combinations of pixel value X of the radiographic image and bone thickness B, and the soft tissue thickness S may be calculated quickly by referencing the table.

[0125] Note that a similar process may be used to calculate the bone thickness S from the soft tissue thickness S and the pixel value X of the radiographic image. A total thickness image may also be obtained by adding a bone image or soft tissue image obtained using the ES image generation model and a soft tissue image or bone image obtained using Equation (17). A virtual monochromatic X-ray image may also be obtained using a bone image or soft tissue image obtained using the ES image generation model and a soft tissue image or bone image obtained using Equation (17). A bone image or soft tissue image may also be obtained using Equation (17) with the total thickness T and the pixel value X of the radiographic image. For example, by substituting (TS) for B or (TB) for S in Equation (17), a bone image or soft tissue image may be obtained using the total thickness image and the radiographic image. A bone image or soft tissue image may also be obtained using a table with the same process as above with the total thickness image and the radiographic image.

[0126] (A series of processes according to this embodiment) Next, a series of processes according to this embodiment, including image generation processing using the trained model, will be described with reference to Fig. 19 and Fig. 20. Fig. 19 is a flowchart showing an example of a series of processes according to this embodiment. Fig. 20 is a block diagram showing an example of a series of processes according to this embodiment.

[0127] In the image generation process using the trained model described above, it has been assumed that the resolution of the image processed by the trained model matches the resolution of the radiographic image used as input data. However, as described above, when generating ES images using a machine learning model, the pixel pitch of the training image may not match the pixel pitch of the radiographic image used as input data for the trained model. For example, when low-resolution images are used to prepare a large amount of training data, the pixel pitch of the training image may be larger than the pixel pitch of the radiographic image used as input data for the trained model.

[0128] As an example, when bone images or soft tissue images are used as training data for a machine learning model, bone images or soft tissue images generated from three-dimensional CT data can be used. However, the pixel pitch of radiographic images obtained by a general radiation sensor for still images or videos is approximately 50 to 200 μm, whereas the voxel pitch of three-dimensional CT data is approximately 500 μm. Therefore, when training images are obtained from three-dimensional CT data, the pixel pitch of the training images becomes larger than the pixel pitch of the radiographic images used as input data for the trained model.

[0129] In addition, depending on the performance of the radiographic imaging device, it may be difficult to prepare as training data only ES images with a pixel pitch larger than the pixel pitch of the radiographic images used as input data to the trained model. In this case, too, the pixel pitch of the training images becomes larger than the pixel pitch of the radiographic images used as input data to the trained model.

[0130] If the pixel pitch of the training image is larger than the pixel pitch of the radiographic image used as input data for the trained model, an ES image with a lower resolution than the resolution of the radiographic image used as input data for the trained model will be output. In such cases, the resolution of the obtained ES image is low compared to the captured radiographic image, which may prevent appropriate diagnosis, etc.

[0131] Therefore, in this embodiment, a process of increasing the resolution is performed when generating ES images such as bone images and soft tissue images using a total thickness image acquired using a trained model that has been trained using ES images with a large pixel pitch, as shown in Fig. 20. Hereinafter, an example of performing processing using a trained model that has been trained using training data with a large pixel pitch (low resolution) will be described.

[0132] First, in step S1901, the acquisition unit 131 acquires a radiographic image captured using the radiation imaging device 104. The acquired radiographic image is an image having a higher resolution than the resolution of the learning data of the ES image generation model. The acquisition unit 131 may acquire the radiographic image from an external device connected to the control device 103.

[0133] In step S1902, the processing unit 133 performs resolution reduction processing such as thinning or binning on the high-resolution radiographic image acquired in step S1901, as shown in Fig. 20, to obtain a low-resolution radiographic image. Through this resolution reduction processing, the processing unit 133 can acquire a radiographic image having a resolution similar to that of the radiographic image used as learning data. Note that the resolution reduction processing such as thinning or binning for reducing the resolution of the image may be performed by any known method.

[0134] Next, in step S1903, the processing unit 133 inputs the low-resolution radiation image into the ES image generation model to generate a low-resolution total thickness image.

[0135] Thereafter, in step S1904, the processing unit 133 performs high-resolution processing such as interpolation on the low-resolution total thickness image generated in step S1903 to obtain a high-resolution total thickness image. Through this high-resolution processing, the processing unit 133 can obtain a total thickness image having a resolution similar to that of the radiographic image used as input data. Note that the high-resolution processing such as interpolation may be performed by any known method.

[0136] The resolution enhancement process may be a process using a machine learning model. In this case, a low-resolution total thickness image may be used as input data for the training data, and a total thickness image that has been enhanced in resolution by interpolation or the like may be used as output data. For images that are easy to acquire, a trained model obtained by performing transfer learning using the total thickness image may be used on a trained model obtained using training data in which a low-resolution image is used as input data and a high-resolution image is used as output data. In this case, a trained model with high accuracy can be prepared using a relatively small number of total thickness images.

[0137] In step S1905, the processing unit 133 performs filtering such as a low-pass filter on the high-resolution total thickness image acquired in step S1904. Note that by applying a low-pass filter (LPF), it is possible to reduce artifacts of high-frequency components caused by the ES image generation model.

[0138] In step S1906, the processing unit 133 performs logarithmic conversion processing on the radiation image acquired in step S1901. The logarithmic conversion processing may be performed by any known method.

[0139] In step S1907, the processing unit 133 acquires a high-resolution ES image using an approximation equation for ES processing based on the high-resolution total thickness image after filtering obtained in step S1905 and the logarithm of the high-resolution radiographic image obtained in step S1906. More specifically, the processing unit 133 performs weighted addition according to equation (13) or equation (14) using the high-resolution total thickness image after filtering obtained in step S1905 and the logarithm of the high-resolution radiographic image obtained in step S1906. In this way, the processing unit 133 acquires an ES image such as a high-resolution bone image, soft tissue image, or radiographic image of any energy.

[0140] In step S1908, the display control unit 134 causes the display unit 120 to display the ES image acquired in step S1907. Note that the display control unit 134 may also cause the display unit 120 to display the analysis results of any analysis process performed by the processing unit 133 on the ES image acquired in step S1907. In addition, the control device 103 may transmit the ES image acquired in step S1907 and the analysis results of the ES image to an external device.

[0141] According to this processing, even when a trained model that has been trained using ES images with a large pixel pitch is used, it is possible to acquire ES images with improved resolution. Therefore, the control device 103 can provide appropriate support for diagnosis, etc. by providing ES images with improved resolution.

[0142] In this embodiment, a high-energy image is used as input data, and a total thickness image is output from the ES image generation model. However, as described above, the input data and the inferred data output from the ES image generation model are not limited to this. The input data and the inferred data may be data corresponding to the training data. For example, the input data may be a low-energy image, or a radiological image such as an energy image obtained by imaging using the voltage of the virtual monochromatic X-ray image used as input data for the training data as the tube voltage during imaging. Furthermore, the ES image output from the ES image generation model and used in subsequent processing may be an ES image such as a bone image or a soft tissue image.

[0143] When the total thickness image is used as the inference data, the strength of the low-pass filter processing can be increased, and the artifacts of the high-frequency components generated by the ES image generation model can be further reduced. Here, this effect will be described with reference to FIG.

[0144] Figure 21 is a diagram for explaining a total thickness image, and shows a schematic cross section of a human body. The surface of the human bone, i.e., the cortical bone, is covered with high-density bone. On the other hand, the inside of the bone, i.e., the trabecular bone, is formed of spongy bone. Therefore, radiographic images and bone images contain high-frequency components resulting from the structure of the cortical bone and trabecular bone.

[0145] However, the spaces between the trabeculae are filled with bone marrow, which is classified as soft tissue. Therefore, in the total thickness image, which is the sum of the bone image and soft tissue, the structures of the bone trabeculae and bone marrow cancel each other out, reducing the high-frequency components of the image. Therefore, in the total thickness image, signal components are less likely to be lost even if the strength of the low-pass filter applied is increased. Here, increasing the strength of the low-pass filter processing can further reduce high-frequency artifacts caused by the ES image generation model. Therefore, when using the total thickness image as inference data, the strength of the low-pass filter processing can be increased, further reducing high-frequency artifacts caused by the ES image generation model.

[0146] In this embodiment, the logarithmic conversion process in step S1906 is performed after the process in step S1905. However, the logarithmic conversion process in step S1906 may be performed between steps S1901 and S1907. The logarithmic conversion process in step S1906 may also be performed in parallel with the processes in steps S1901 to S1905.

[0147] Furthermore, in this embodiment, in step S1907, a high-resolution ES image is acquired by weighted addition of the logarithm of the high-resolution radiographic image and the high-resolution total thickness image according to equation (13) or equation (14) as an approximation equation for ES processing. Alternatively, a high-resolution ES image may be acquired by calculation using a nonlinear equation of the high-resolution radiographic image and the high-resolution total thickness image according to equation (17) as an approximation equation for ES processing. Furthermore, when the image output from the ES image generation model is a bone image or a soft tissue image, a high-resolution ES image can be acquired according to equation (15), equation (16), or equation (17) as an approximation equation for ES processing. Specifically, the processing unit 133 can acquire a high-resolution ES image by weighted addition of the logarithm of the high-resolution radiographic image and an image obtained by increasing the resolution of the image output from the ES image generation model according to equation (15) or equation (16). Furthermore, the processing unit 133 may acquire a high-resolution ES image by performing a calculation using a high-resolution radiographic image and an image obtained by increasing the resolution of an image output from the ES image generation model, according to equation (17) as an approximate equation for ES processing. When performing the calculation according to equation (17), step S1906 may be omitted.

[0148] Furthermore, when images clipped by data augmentation are used as training data for a trained model, the processing unit 133 clips the low-resolution radiographic images to be input to the ES image generation model to make the image size corresponding to the training data. After that, the processing unit 133 can connect the images to each other to make the image output from the ES image generation model by inputting the clipped low-resolution radiographic images the same size as before clipping.

[0149] However, in this case, tile-like artifacts may occur due to differences in levels at the joints between the images. Therefore, the processing unit 133 may provide overlapping portions (overlapping portions) between the images when clipping the low-resolution radiographic images, and join the overlapping portions of the images output from the ES image generation model by performing weighted addition. For example, weighting may be performed based on the coordinates of the pixels in the overlapping portions of the clipped low-resolution radiographic images.

[0150] As described above, the radiation imaging system according to this embodiment includes the radiation imaging device 104 that detects radiation irradiated from the radiation generation device 101, and the control device 103 that is communicatively connected to the radiation imaging device 104. The control device 103 can function as an example of an image processing device that performs image processing. The control device 103 includes an acquisition unit 131 that acquires a first image, which is a radiation image, and a processing unit 133 that performs processing to generate a second image, which is an energy subtraction image, using the acquired first image.

[0151] The processing unit 133 performs a process of reducing the resolution of the first image. The processing unit 133 can also acquire a third image output from the trained model by inputting the first image with reduced resolution to the trained model. The processing unit 133 also performs a process of improving the resolution of the third image. The processing unit 133 also uses the first image and the third image with improved resolution to generate a second image that is a different type of image from the third image and has a higher resolution than the acquired third image.

[0152] Note that different types of images include not only images of different image categories such as radiation images and material decomposition images, but also images of the same category but with different energies. For example, different types of images also include radiation images with different energies.

[0153] With this configuration, the control device 103 according to this embodiment can acquire high-resolution ES images based on normal radiographic images, even when using a trained model trained using ES images with a large pixel pitch. Therefore, ES images with a large pixel pitch, such as total thickness images generated from 3D CT data, bone images, and soft tissue images, can be used as training data, facilitating the preparation of training data for the trained model. As a result, it is expected that the training accuracy will be improved, resulting in reduced artifacts and an increase in the number of regions that can be handled by the trained model. Furthermore, by providing ES images with improved resolution, the control device 103 can provide support for appropriate diagnosis, etc.

[0154] Furthermore, with the above configuration, a high-resolution ES image can be generated using a single radiographic image. This eliminates the need for significant hardware modifications to the radiographic imaging system used for inference using a trained model, thereby reducing the cost of the radiographic imaging system. Furthermore, since processing can be performed using a single radiographic image, it is also possible to reduce the amount of radiation exposure of the subject associated with capturing the radiographic image.

[0155] The third image may be a total thickness image showing the sum of the thicknesses of different materials in the subject, or a material decomposition image showing the thickness of materials in the subject. The material decomposition image may be, for example, a bone image, a soft tissue image, an image showing the thickness of water, or an image showing the thickness of a contrast agent. In this case, the processing unit 133 may generate the second image by weighted addition of the logarithm of the first image and the third image with improved resolution. The processing unit 133 may also generate the second image by calculation using a nonlinear equation between the first image and the third image with improved resolution.

[0156] Furthermore, the processing unit 133 can apply a filter to the third image with improved resolution and generate a second image using the third image that has been subjected to the filter process. For example, the processing unit 133 can reduce artifacts of high-frequency components in the image output by the trained model by applying a low-pass filter to the third image. Furthermore, when the third image is a total thickness image, the filter strength can be increased, thereby further reducing artifacts in the image output by the trained model.

[0157] The processing unit 133 can improve the resolution of the third image by performing interpolation processing, and can reduce the resolution of the first image by performing thinning processing or binning processing.

[0158] The second image may be a material decomposition image showing the thickness of the material of the subject, a total thickness image showing the sum of the thicknesses of different materials of the subject, a virtual monochromatic X-ray image, or a radiation image of any energy.

[0159] The trained model can be obtained using training data including a radiological image having a lower resolution than the first image and a third image having a lower resolution than the first image and being a different type of image from the first image. Here, the training data can include images obtained using forward projection based on three-dimensional CT data. Furthermore, the training data can also include images whose resolution has been reduced by performing thinning or binning.

[0160] (Variation 1) In the first embodiment, it has been described that the ES image generation model may be trained by preparing low-resolution radiographic images and low-resolution bone images as training data. In such a case, a total thickness image may be generated using the bone images output from the ES image generation model, and the total thickness image may be subjected to high-resolution processing and low-pass filtering.

[0161] 22 and 23, a first modification will be described in which, when low-resolution bone images are used as output data of the learning data, a total thickness image is generated using the bone images output from the ES image generation model, and a high-resolution ES image is generated based on the total thickness image. Fig. 22 is a flowchart showing an example of a series of processes according to this modification. Fig. 23 is a block diagram showing an example of a series of processes according to this modification.

[0162] The series of processes according to this modification differs from the series of processes according to the first embodiment in that steps S2201 and S2202 are added after step S1903. The series of processes according to this embodiment will be described below, focusing on the differences from the first embodiment.

[0163] The processing of steps S1901 to S1903 is the same as the processing of the first embodiment, and therefore description thereof will be omitted. However, in step S1903, a low-resolution bone image is generated using an ES image generation model in accordance with the training data.

[0164] In step S2201, the processing unit 133 performs logarithmic conversion processing on the low-resolution radiation image acquired in step S1902, in the same manner as in step S1906.

[0165] In step S2202, the processing unit 133 performs weighted addition of the low-resolution bone image obtained in step S1903 and the logarithm of the low-resolution radiographic image obtained in step S2201 to obtain a low-resolution total thickness image. Equation (15) can be used for the weighted addition.

[0166] After that, in step S1904, the processing unit 133 performs a high-resolution process on the total thickness image obtained in step S2201. The subsequent processes may be the same as the series of processes according to the first embodiment.

[0167] In this modification, when a low-resolution bone image is generated using an ES image generation model, a low-resolution total thickness image is generated first, and then a high-resolution ES image is generated. With this configuration, the low-pass filter is applied to the total thickness image, so that the strength of the low-pass filter processing can be increased as described above, and high-frequency artifacts caused by the ES image generation model can be further reduced.

[0168] Similarly, a low-resolution radiographic image and a low-resolution soft-tissue image may be prepared, an ES image generation model may be trained, a total thickness image may be generated using the soft-tissue image output from the ES image generation model, and the total thickness image may be subjected to high-resolution processing, low-pass filtering, etc. Fig. 24 is a block diagram showing an example of a series of processes according to this modified example when soft-tissue images are used as output data of the training data.

[0169] In this case, the same processing as the series of processing shown in Fig. 22 may be performed. However, in step S2202, the processing unit 133 performs weighted addition of the low-resolution soft tissue image obtained in step S1903 and the logarithm of the low-resolution radiographic image obtained in step S2201 to obtain a low-resolution total thickness image. Equation (16) can be used for the weighted addition. The subsequent processing may be the same as the series of processing shown in Fig. 22.

[0170] In this series of processes, when generating a low-resolution soft-tissue image using an ES image generation model, a low-resolution total thickness image is first generated, and then a high-resolution ES image is generated. Even in this case, the low-pass filter is applied to the total thickness image, so the strength of the low-pass filter processing can be increased as described above, and high-frequency artifacts caused by the ES image generation model can be further reduced.

[0171] In this modification, the logarithmic conversion process in step S2201 is performed after the process of step S1903. However, the logarithmic conversion process in step S2201 may be performed between steps S1902 and S2202. Also, the logarithmic conversion process in step S2201 may be performed in parallel with the processes from steps S1902 to S1903. Similarly, the logarithmic conversion process in step S1906 may be performed between steps S1901 and S1907. Also, the logarithmic conversion process in step S1906 may be performed in parallel with the processes from steps S1901 to S1905.

[0172] In this modification, in step S2202, the processing unit 133 acquires a low-resolution total thickness image by weighted addition of the logarithm of the low-resolution radiographic image and the image output from the ES image generation model according to equation (15) or equation (16) as an approximation equation for ES processing. Alternatively, the processing unit 133 may acquire a low-resolution total thickness image using equation (17) as an approximation equation for ES processing. More specifically, the processing unit 133 may acquire a low-resolution soft tissue image or bone image by performing a calculation using the low-resolution radiographic image and the low-resolution bone image or soft tissue image output from the ES image generation model according to equation (17). The processing unit 133 can acquire a low-resolution total thickness image by adding the low-resolution bone image or soft tissue image output from the ES image generation model and the low-resolution soft tissue image or bone image acquired according to the approximation equation for ES processing. Alternatively, the processing unit 133 may acquire a low-resolution total thickness image by devising a solution using the Newton-Raphson method of equation (17). In this case, step S2201 may be omitted. Note that, when acquiring a material decomposition image in step S2202, the material decomposition image may be acquired by the same method as described in step S1907.

[0173] As described above, the processing unit 133 according to this modification can generate a fourth image, which is an energy subtraction image, using the first image and the third image with reduced resolution. The processing unit 133 can also perform processing to improve the resolution of the fourth image. The processing unit 133 can improve the resolution of the fourth image by, for example, performing interpolation processing. The processing unit 133 can also generate a second image, which is a different type of image from the fourth image and has a higher resolution than the generated fourth image, using the acquired first image and the fourth image with improved resolution. This configuration can also achieve the same effects as the first embodiment.

[0174] The processing unit 133 can generate the second image by weighted addition of the logarithm of the first image and the fourth image with improved resolution. The processing unit 133 can also generate the second image by calculation using a nonlinear equation between the first image and the fourth image with improved resolution.

[0175] Furthermore, the processing unit 133 can apply a filter process to the fourth image with improved resolution and generate a second image using the fourth image that has been subjected to the filter process. For example, the processing unit 133 can apply a low-pass filter to the fourth image to reduce artifacts of high-frequency components in the image output by the trained model.

[0176] The fourth image used to generate the second image can be a total thickness image that shows the sum of the thicknesses of different materials in the subject, or a material decomposition image that shows the thickness of the material in the subject. When the fourth image is a total thickness image, the filter strength can be improved, and artifacts in the image output by the trained model can be further reduced.

[0177] In this regard, if the third image is not a total thickness image indicating the sum of thicknesses of different materials in the subject, the processing unit 133 may generate a fourth image, which is a total thickness image, using the first image with reduced resolution and the third image. For example, if the third image is a material decomposition image indicating the thicknesses of materials in the subject, the processing unit 133 can generate the fourth image by using a weighted addition of the logarithm of the first image with reduced resolution and the third image. In this case, the processing unit 133 can also generate the fourth image by performing an operation using a nonlinear equation between the first image with reduced resolution and the third image.

[0178] (Second embodiment) In the first embodiment, a trained model obtained by using a total thickness image, a bone image, or a soft tissue image as output data of training data is used as the ES image generation model. In contrast, in the second embodiment of the present disclosure, a trained model is used that uses a radiological image such as a low-energy image, a high-energy image, or a virtual monochromatic X-ray image as output data of training data.

[0179] The radiation imaging system according to this embodiment will be described below with reference to Figs. 25 to 29. The configuration of the radiation imaging system according to this embodiment is similar to that of the radiation imaging system according to the first embodiment, and therefore the same reference numerals are used and a description thereof will be omitted. The learning process according to this embodiment is similar to that according to the first embodiment, and therefore a description thereof will be omitted. The radiation imaging system according to this embodiment will be described below, focusing on the differences from the radiation imaging system according to the first embodiment.

[0180] (Training data) In this embodiment, as in the first embodiment, a trained model may be generated by loading training data into a computer and training features such as regularities and patterns inherent in the training data based on a machine learning algorithm. Also in this embodiment, both input and output training data are image data. However, in this embodiment, both input and output training data are radiological images such as high-energy images, low-energy images, or virtual monochromatic X-ray images. Note that radiological images relating to different energies can be used as the input and output training data according to this embodiment.

[0181] As for the training data according to this embodiment, the input data may be a high-energy image and the output data may be a low-energy image. Conversely, as for the training data, the input data may be a low-energy image and the output data may be a high-energy image. Furthermore, the input data and output data of the training data may be virtual monochromatic X-ray images of different energies. Furthermore, as for the training data, the input data may be a high-energy image or a low-energy image and the output data may be a virtual monochromatic X-ray image of different energy from the input data. Conversely, as for the training data, the input data may be a virtual monochromatic X-ray image and the output data may be a radiation image of different energy from the input data.

[0182] Here, Fig. 25 shows an example of training data according to this embodiment. As shown in Fig. 25, when input data of the training data is a radiographic image A and output data of the training data is a radiographic image B, the training data is a set of data including a plurality of these image pairs (training data 1 to n).

[0183] As in the first embodiment, when target data is input to a trained model, inference data according to the design of the trained model is output. The combination of images of the target data and inference data corresponds to the combination of images of input data and output data of training data. Therefore, the target data in this embodiment may be a radiological image such as a high-energy image, a low-energy image, or an energy image obtained by imaging using the voltage of a virtual monochromatic X-ray image as the tube voltage during imaging according to the training data. Furthermore, the inference data may be a radiological image such as an energy image of an energy different from the energy of the target data or a virtual monochromatic X-ray image according to the output data of the training data.

[0184] Here, with reference to Figures 26(a) and 26(b), the relationship between the input data and output data of the training data according to this embodiment, and the target data and inference data during inference will be described. Hereinafter, in this embodiment, an example will be described in which the input data of the training data is a high-energy image and the output image of the training data is a low-energy image. Figure 26(a) is a block diagram of the training process using an example of the training data according to this embodiment, and Figure 26(b) is a block diagram of the inference process using an example of the target data and inference data according to this embodiment.

[0185] As described above, in this embodiment, the input data and target data of the training data are high-energy images, and the output data and inference data are low-energy images. As shown in Figure 26(a), a trained model trained using high-energy images and low-energy images as training data trains features such as regularity and regularity inherent in the high-energy images and low-energy images. Therefore, as shown in Figure 26(b), when a high-energy image is input as target data, the trained model can output a low-energy image as inference data that is likely to correspond to the target data according to the training trend.

[0186] Here, the radiation images of different energies used as input data and output data for the training data have different transmittances (pixel values ​​on the images) of the subject depending on the radiation energy, but the structure (arrangement) of the subject is the same. Therefore, ideally, these images differ only in contrast. In this way, training with images in which the subject structure is the same or has only minor differences can reduce the occurrence of artifacts caused by differences in the subject structure compared to training with different types of images. Therefore, the trained model according to this embodiment can output inference data in which artifacts caused by the subject structure are reduced.

[0187] Using the training data described above, a trained model according to this embodiment can be generated by performing a training process similar to the training process according to the first embodiment on the machine learning model according to this embodiment. By using the trained model according to this embodiment, it is possible to generate radiographic images of different energies based on radiographic images of a single energy.

[0188] Next, the image generation process using the trained model according to this embodiment will be described, focusing on the differences from the image generation process according to the first embodiment. Note that the image generation process using the trained model according to this embodiment is similar to that of the first embodiment, except that radiographic images are generated using the trained model and that radiographic images generated by the trained model are used in post-processing. Therefore, the image generation process using the trained model according to this embodiment will be described below with reference to FIG. 13.

[0189] In the image generation process using the trained model according to this embodiment, similarly to the first embodiment, the acquisition unit 131 acquires a high-energy image in step S1301. Then, in step S1302, the processing unit 133 acquires a low-energy image output from the trained model by using the high-energy image acquired in step S1301 as input data for the trained model.

[0190] In step S1303, the processing unit 133 acquires an ES image such as a bone image, a soft tissue image, or a radiation image of any energy using an approximation formula for ES processing using a high-energy image and a low-energy image. More specifically, the processing unit 133 acquires an ES image using weighted addition using the logarithm of the high-energy image and the logarithm of the low-energy image.

[0191] Here, the post-processing according to this embodiment will be explained with reference to FIG. 27. The pixel values ​​of the low-energy image and the high-energy image generated by the trained model according to this embodiment are expressed by Equation (5). For simplicity, the spectrum N of high-energy X-rays is H (E) is the average energy E H It is a virtual monochromatic X-ray with a peak at the low energy X-ray spectrum N L (E) is the average energy E L Approximating this to a virtual monochromatic X-ray with a peak at , it is expressed by the following equation (18).

number

[0192] From equations (5) and (18), the pixel value of the radiation image, X H ,X L is the energy E H The linear attenuation coefficient μ of bone in HB and the linear attenuation coefficient of soft tissue μ HS and energy E L The linear attenuation coefficient μ of bone in LB and the linear attenuation coefficient of soft tissue μ LS Using the above, it is expressed by the following equation (19).

number

[0193] Equation (19) can be rearranged to obtain the following equation (20).

number

[0194] Furthermore, as described in the first embodiment, the pixel value X of a radiographic image acquired at any energy is expressed by equation (12). Therefore, the following equation (21) holds from equation (12) and equation (20).

number

[0195] Therefore, the processing unit 133 can acquire ES images such as bone images, soft tissue images, or radiation images of any energy by performing processing using an approximation formula for ES processing on the logarithm of the high-energy image and the logarithm of the low-energy image. The processing unit 133 can also generate a total thickness image by adding the bone images and soft-tissue images thus obtained.

[0196] (A series of processes according to this embodiment) Next, a series of processes according to this embodiment, including image generation processing using the trained model, will be described with reference to Fig. 28 and Fig. 29. Fig. 28 is a flowchart showing an example of the series of processes according to this embodiment. Fig. 29 is a block diagram showing an example of the series of processes according to this embodiment.

[0197] Even in the image generation process using the trained model, when low-resolution images are used to prepare a large amount of training data, the pixel pitch of the training images may become larger than the pixel pitch of the radiographic images used as input data to the trained model. If the pixel pitch of the training images is larger than the pixel pitch of the radiographic images used as input data to the trained model, a radiographic image with a lower resolution than the resolution of the radiographic image used as input data to the trained model will be output. In such cases, there is a possibility that an appropriate diagnosis or the like cannot be performed due to the low resolution of the obtained radiographic image compared to the captured radiographic image.

[0198] Therefore, in this embodiment, a process for increasing the resolution is performed when an ES image such as a bone image or a soft tissue image is obtained by using a radiographic image acquired using a trained model that has been trained using radiographic images with a large pixel pitch. Below, an example will be described in which processing is performed using a trained model that has been trained using training data with a large pixel pitch (low resolution).

[0199] Note that, when using radiographic images with a large pixel pitch as training data, it is conceivable to use radiographic images obtained simply using a low-resolution radiographic imaging device as training data, or to obtain training data from three-dimensional CT data. Here, when obtaining training data from three-dimensional CT data, for example, two-dimensional bone images and soft tissue images can be obtained from the three-dimensional CT data using forward projection, and then these can be converted to obtain training data. In this case, for example, high-energy images and low-energy images used as training data can be obtained from the bone images and soft tissue images using Equation (19). Furthermore, images obtained by performing thinning or binning on radiographic images obtained using a high-resolution radiographic imaging device to reduce the resolution can be included in the training data.

[0200] In the series of processes according to this embodiment, first, in step S2801, the acquisition unit 131 acquires a high-energy image captured using the radiation imaging device 104. Note that the acquired high-energy image has a higher resolution than the resolution of the learning data of the trained model. Note that the acquisition unit 131 may acquire the high-energy image from an external device connected to the control device 103.

[0201] In step S2802, the processing unit 133 performs resolution reduction processing such as thinning or binning on the high-resolution high-energy image acquired in step S2801, as shown in Fig. 29, to obtain a low-resolution high-energy image. Through this resolution reduction processing, the processing unit 133 can acquire a high-energy image having a resolution comparable to that of the high-energy image used as training data. Note that the resolution reduction processing such as thinning or binning for reducing the image resolution may be performed by any known method.

[0202] Next, in step S2803, the processing unit 133 inputs the low-resolution high-energy image into the trained model to generate a low-resolution low-energy image.

[0203] In step S2804, the processing unit 133 performs logarithmic transformation on the low-resolution high-energy image acquired in step S2802 and on the low-resolution low-energy image acquired in step S2803. The logarithmic transformation may be performed by any known method.

[0204] In step S2805, the processing unit 133 performs weighted addition of the logarithm of the low-resolution low-energy image and the logarithm of the low-resolution high-energy image obtained in step S2804 according to equation (20) as an approximation equation for ES processing, to obtain low-resolution bone and soft-tissue images. The processing unit 133 also adds the obtained low-resolution bone and soft-tissue images to obtain a low-resolution total thickness image.

[0205] In step S2806, the processing unit 133 performs high-resolution processing such as interpolation on the low-resolution total thickness image acquired in step S2805 to obtain a high-resolution total thickness image. Through this high-resolution processing, the processing unit 133 can acquire a total thickness image having a resolution similar to that of the radiographic image used as input data. Note that the high-resolution processing such as interpolation may be performed by any known method. Note that the high-resolution processing may be processing using a machine learning model, as described in the first embodiment.

[0206] In step S2807, the processing unit 133 performs filtering such as a low-pass filter on the high-resolution total thickness image acquired in step S2806. Note that by applying a low-pass filter, it is possible to reduce artifacts of high-frequency components caused by the trained model.

[0207] In step S2808, the processing unit 133 performs logarithmic transformation processing on the high-energy image acquired in step S2801. The logarithmic transformation processing may be performed by any known method.

[0208] In step S2809, the processing unit 133 performs weighted addition on the logarithm of the high-resolution total thickness image after filtering obtained in step S2807 and the high-resolution high-energy image obtained in step S2809 according to equation (13) or equation (14). As a result, the processing unit 133 acquires an ES image such as a high-resolution bone image, soft tissue image, or radiation image.

[0209] In step S2810, the display control unit 134 causes the display unit 120 to display the ES image acquired in step S2809. Note that the display control unit 134 may also cause the display unit 120 to display the analysis results of any analysis process performed by the processing unit 133 on the ES image acquired in step S2809. In addition, the control device 103 may transmit the ES image acquired in step S2809 and the analysis results of the ES image to an external device.

[0210] In the description of this embodiment, a high-energy image is used as input data, and a low-energy image is output from the trained model. However, as described above, the input data and the inference data output from the trained model are not limited to this. The input data and the inference data may be data corresponding to the training data. For example, the input data may be a low-energy image, or a radiological image such as an energy image obtained by imaging using the voltage of the virtual monochromatic X-ray image used as input data for the training data as the tube voltage during imaging. Furthermore, the radiological image output from the trained model and used in subsequent processing may be a radiological image such as a high-energy image or a virtual monochromatic X-ray image.

[0211] In this embodiment, a total thickness image is acquired in step S2805, but the image acquired in step S2805 and used in subsequent processing may be an ES image such as a material decomposition image, such as a bone image or a soft tissue image. When a total thickness image is acquired in step S2805 and used in subsequent processing, the strength of the low-pass filter processing can be increased, as described in the first embodiment. This can provide the effect of further reducing high-frequency component artifacts caused by the trained model.

[0212] In this embodiment, the logarithmic transformation processing of the low-resolution input data in step S2804 is performed after the processing of step S2803. However, the logarithmic transformation processing of the low-resolution input data in step S2804 may be performed between steps S2802 and S2805. The logarithmic transformation processing of the low-resolution input data in step S2804 may be performed in parallel with the processing of steps S2802 to S2803. Similarly, the logarithmic transformation processing of step S2808 may be performed between steps S2801 and S2809. The logarithmic transformation processing of step S2808 may be performed in parallel with the processing of steps S2801 to S2807.

[0213] Furthermore, in this embodiment, in step S2809, a high-resolution ES image is acquired by weighted addition of the logarithm of the high-resolution radiographic image and the high-resolution total thickness image according to equation (13) or equation (14) as an approximation formula for ES processing. On the other hand, if the image acquired in step S2805 is a bone image or a soft tissue image, a high-resolution ES image can be acquired according to equation (15), equation (16), or equation (17) as an approximation formula for ES processing. Specifically, the processing unit 133 can acquire a high-resolution ES image by weighted addition of the logarithm of the high-resolution radiographic image and an image obtained by increasing the resolution of the image acquired in step S2805 according to equation (15) or equation (16). Alternatively, the processing unit 133 may acquire a high-resolution ES image by calculation using the high-resolution radiographic image and an image obtained by increasing the resolution of the image acquired in step S2805 according to equation (17) as an approximation formula for ES processing. In this case, step S2808 may be omitted.

[0214] Furthermore, in this embodiment, when images clipped by data augmentation are used as training data for a trained model, the processing unit 133 may perform processing using the overlapping portion as described in the first embodiment.

[0215] As described above, in this embodiment, the third image output from the trained model may be a radiographic image or a virtual monochromatic X-ray image with a different energy from the acquired first image. In this regard, the third image used as training data may be a different type of image from the first image (a radiographic image or a virtual monochromatic X-ray image with a different energy). Furthermore, the processing unit 133 may generate a fourth image by using a weighted addition of the logarithm of the first image with reduced resolution and the logarithm of the third image acquired using the trained model. Furthermore, the processing unit 133 may perform a process to improve the resolution of the fourth image. Furthermore, the processing unit 133 may use the acquired first image and the fourth image with improved resolution to generate a second image that is a different type of image from the fourth image and has a higher resolution than the generated fourth image.

[0216] With this configuration, even when using a trained model trained using radiographic images with a large pixel pitch, it is possible to acquire ES images with improved resolution based on normal radiographic images. Therefore, radiographic images with a large pixel pitch, such as radiographic images generated from 3D CT data, can be used as training data, making it easier to prepare training data for the trained model. As a result, it is expected that artifacts will be reduced due to improved training accuracy, and the number of areas that can be handled by the trained model will increase. Furthermore, by providing ES images with improved resolution, the control device 103 can provide support for appropriate diagnosis, etc.

[0217] Furthermore, with the above configuration, a high-resolution ES image can be generated using a single radiographic image. This eliminates the need for significant hardware modifications to the radiographic imaging system used for inference using a trained model, thereby reducing the cost of the radiographic imaging system. Furthermore, since processing can be performed using a single radiographic image, it is also possible to reduce the amount of radiation exposure of the subject associated with capturing the radiographic image.

[0218] (Variation 2) In the second embodiment, a configuration was assumed in which a total thickness image is acquired from a high-energy image and a low-energy image, and an ES image is generated by weighted addition of the logarithm of the high-energy image and the total thickness image. However, the configuration in which an ES image is generated using a radiological image output from a trained model is not limited to this. Here, as Modification 2, a configuration in which an ES image is generated using the logarithms of radiological images associated with different energies will be described.

[0219] The input data (target data) and inference data may be data corresponding to the training data, and the input data and output data of the training data may be radiation images relating to different energies, as in the second embodiment. For example, the input data may be a radiation image such as a high-energy image, a low-energy image, or an energy image obtained by imaging using the voltage of the virtual monochromatic X-ray image used as input data for the training data as the tube voltage during imaging. Furthermore, the radiation image output from the trained model and used in subsequent processing may be a radiation image such as a low-energy image, a high-energy image, or a virtual monochromatic X-ray image. Below, an example will be described in which the input data is a high-energy image and the inference data is a low-energy image.

[0220] Here, a series of processes according to this modified example will be described with reference to Fig. 30 and Fig. 31. Fig. 30 is a flowchart showing an example of a series of processes according to this modified example. Fig. 31 is a block diagram showing an example of a series of processes according to this modified example. Note that processes similar to those in the second embodiment will be designated by similar reference numerals and will not be described again.

[0221] The processing of steps S2801 to S2803 is omitted here because it is the same as the processing of the second embodiment. In step S2803, when a low-resolution, low-energy image is acquired using the trained model, the processing proceeds to step S3001.

[0222] In step S3001, the processing unit 133 performs logarithmic transformation on the low-resolution, low-energy image obtained in step S2803. The logarithmic transformation may be performed by any known method.

[0223] In step S3002, the processing unit 133 performs high-resolution processing such as interpolation on the logarithm of the low-resolution low-energy image to obtain the logarithm of the high-resolution low-energy image. Note that the high-resolution processing such as interpolation may be performed by any known method. Note that the high-resolution processing may be processing using a machine learning model, as described in the first embodiment.

[0224] In step S3003, the processing unit 133 performs filtering, such as low-pass filtering, on the logarithm of the high-resolution low-energy image.

[0225] When the process in step S3003 is completed, the process proceeds to step S2808. In step S2808, the processing unit 133 performs logarithmic transformation on the high-resolution high-energy image, which is the input image, as in the second embodiment. In step S2808, when the high-energy image is logarithmically transformed, the process proceeds to step S3004.

[0226] In step S3004, the processing unit 133 performs filtering such as low-pass filtering and high-pass filtering on the logarithm of the high-resolution, high-energy image.

[0227] In step S3005, the processing unit 133 performs weighted addition on the logarithm of the high-resolution low-energy image after the low-pass filter, the logarithm of the high-resolution high-energy image after the low-pass filter, and the logarithm of the high-resolution high-energy image after the high-pass filter. The processing unit 133 performs this weighted addition to obtain a high-resolution ES image. Note that this weighted addition may be performed by multiplying each image by an arbitrary coefficient and adding them together using, for example, equation (20) or equation (21) as an approximate equation for ES processing. The subsequent processing may be the same as in the second embodiment.

[0228] In the description of this modified example, a high-energy image is used as input data and a low-energy image is output from the trained model. However, as described above, the input data and the inference data output from the trained model are not limited to this. The input data and the inference data may be data corresponding to the training data. For example, the input data may be a low-energy image, or a radiological image such as an energy image obtained by imaging using the voltage of the virtual monochromatic X-ray image used as input data for the training data as the tube voltage during imaging. Furthermore, the radiological image output from the trained model and used in subsequent processing may be a radiological image such as a high-energy image or a virtual monochromatic X-ray image.

[0229] In this modification, the logarithmic transformation processing in step S3001 is performed after the processing of step S2803. However, the logarithmic transformation processing in step S3001 may be performed between steps S2801 and S3002. The logarithmic transformation processing in step S3001 may also be performed in parallel with the processing of steps S2801 to S2803. Similarly, the logarithmic transformation processing in step S2808 may be performed between steps S2801 and S3004. The logarithmic transformation processing in step S2808 may also be performed in parallel with the processing of steps S2801 to S3003.

[0230] As described above, in this modification, the third image can be a radiographic image or a virtual monochromatic X-ray image with a different energy from that of the acquired first image. In this case, the processing unit 133 can generate the second image by performing weighted addition of the logarithm of the acquired first image and the logarithm of the third image with improved resolution. Even with this configuration, the same effects as those of the second embodiment can be achieved.

[0231] The processing unit 133 can apply filter processing to the logarithm of the acquired first image. Here, the filter applied to the logarithm of the first image can be, for example, a high-pass filter and a low-pass filter. The processing unit 133 can also perform processing to improve the resolution of the logarithm of the third image acquired using the trained model. Furthermore, the processing unit 133 can apply filter processing to the logarithm of the third image with improved resolution. Here, the filter applied to the logarithm of the third image with improved resolution can be, for example, a low-pass filter. In this case, the processing unit 133 can generate the second image by using weighted addition of the logarithm of the first image to which the filter processing has been applied and the logarithm of the third image to which the filter processing has been applied. In such a case, it is possible to reduce the occurrence of artifacts in the second image caused by low-frequency components and high-frequency components contained in the first image and artifacts caused by high-frequency components contained in the third image.

[0232] Note that the post-processing using the trained model according to the above embodiment and modified example is not limited to the above-described processing. For example, it may include tone conversion processing, any image quality improvement processing, any image conversion processing, etc. Furthermore, in the post-processing according to the above embodiment and modified example, filtering processing using a low-pass filter, a high-pass filter, etc. is performed, but filtering may not be performed. Furthermore, the filter used in filtering processing may be a band-pass filter that passes signals in a desired frequency band.

[0233] In addition, in the trained models described in the above embodiments and variants, it is considered that the magnitude of the brightness values ​​of the input data image, the order and gradient of the bright and dark areas, position, distribution, continuity, etc. are extracted as part of the features and used in the inference process.

[0234] The flowcharts described in the above embodiments and modifications show only examples of the flow of operations, and the order of the operations is not limited to the order shown in each flowchart. For example, some operations may be performed in a different order or in parallel.

[0235] (Other Examples) The present disclosure can also be realized by providing a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions. A computer may have one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0236] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0237] The above disclosure includes the following configurations, methods, and programs. (Configuration 1) an acquisition unit that acquires a first image that is a radiation image; a processing unit that uses the first image to generate a second image, which is an energy subtraction image; Equipped with The processing unit performing a process to reduce the resolution of the first image; By inputting the first image with reduced resolution into a trained model, a third image that is an image of a different type from the first image is obtained and output from the trained model; performing a resolution enhancement process on the third image; an image processing device that uses the first image and the third image with improved resolution to generate a second image that is a different type of image from the third image and has a higher resolution than the acquired third image. (Configuration 2) 2. The image processing device according to claim 1, wherein the third image is a total thickness image showing the sum of thicknesses of different materials of the object, or a material decomposition image showing the thickness of materials of the object. (Configuration 3) The processing unit a weighted addition of the logarithm of the first image and the resolution-enhanced third image; or 3. The image processing device according to configuration 2, wherein the second image is generated by performing an operation using a nonlinear equation between the first image and the third image with improved resolution. (Configuration 4) 2. The image processing device according to claim 1, wherein the third image is a radiation image or a virtual monochromatic X-ray image of a different energy from the acquired first image. (Configuration 5) 5. The image processing device of claim 4, wherein the processing unit generates the second image using a weighted addition of the logarithm of the first image and the logarithm of the resolution-enhanced third image. (Configuration 6) 6. The image processing device according to any one of configurations 1 to 5, wherein the processing unit applies a filter process to the third image whose resolution has been improved, and generates the second image using the third image that has been subjected to the filter process. (Configuration 7) The processing unit applying a filter to the logarithm of the first image; performing a resolution enhancement process on the logarithm of the third image; applying a filter to the logarithm of the resolution-enhanced third image; 2. The image processing device of claim 1, wherein the second image is generated using a weighted addition of the logarithm of the filtered first image and the logarithm of the filtered third image. (Configuration 8) 8. The image processing device according to any one of configurations 1 to 7, wherein the processing unit performs interpolation processing to improve the resolution of the third image. (Configuration 9) 9. The image processing device according to any one of configurations 1 to 8, wherein the processing unit reduces the resolution of the first image by performing thinning processing or binning processing. (Configuration 10) 10. The image processing device according to any one of configurations 1 to 9, wherein the second image is a material decomposition image showing the thickness of a material in the subject, a total thickness image showing the sum of the thicknesses of different materials in the subject, a virtual monochromatic X-ray image, or a radiation image of any energy. (Configuration 11) 11. The image processing device according to any one of configurations 1 to 10, wherein the trained model is obtained using training data including a radiological image having a lower resolution than the first image and a third image having a lower resolution than the first image and being a different type of image from the first image. (Configuration 12) 12. The image processing device according to claim 11, wherein the training data includes images obtained using forward projection based on three-dimensional CT data. (Configuration 13) 12. The image processing device according to claim 11, wherein the learning data includes images whose resolution has been reduced by performing thinning or binning. (Configuration 14) an acquisition unit that acquires a first image that is a radiation image; a processing unit that uses the first image to generate a second image, which is an energy subtraction image; Equipped with The processing unit performing a process to reduce the resolution of the first image; By inputting the first image with reduced resolution into a trained model, a third image is obtained, which is an image of a different type from the first image, output from the trained model; generating a fourth image, which is an energy subtraction image, using the reduced resolution first image and the third image; performing a resolution enhancement process on the fourth image; an image processing device that uses the acquired first image and the fourth image with improved resolution to generate a second image that is a different type of image from the fourth image and has a higher resolution than the generated fourth image. (Configuration 15) 15. The image processing device according to configuration 14, wherein the fourth image is a total thickness image showing the sum of thicknesses of different materials of the object, or a material decomposition image showing the thickness of materials of the object. (Configuration 16) The processing unit a weighted addition of the logarithm of the first image and the resolution-enhanced fourth image; or 16. The image processing device according to claim 15, wherein the second image is generated using an operation using a nonlinear equation between the first image and the fourth image with improved resolution. (Configuration 17) the third image is a material decomposition image showing a thickness of a material of the object, The processing unit a weighted addition of the logarithm of the reduced resolution first image and the third image; or 15. The image processing device according to configuration 14, wherein the fourth image is generated by performing an operation using a nonlinear equation between the first image with reduced resolution and the third image. (Configuration 18) 15. The image processing device of claim 14, wherein the third image is a radiation image of a different energy than the first image or a virtual monochromatic X-ray image. (Configuration 19) 19. The image processing device of claim 18, wherein the processing unit generates the fourth image using a weighted addition of the logarithm of the reduced resolution first image and the logarithm of the third image. (Configuration 20) 20. The image processing device according to any one of configurations 14 to 19, wherein the processing unit applies a filter process to the fourth image with improved resolution, and generates the second image using the fourth image that has been subjected to the filter process. (Configuration 21) 21. The image processing device according to any one of configurations 14 to 20, wherein the processing unit performs interpolation processing to improve the resolution of the fourth image. (Configuration 22) 22. The image processing device according to any one of configurations 14 to 21, wherein the processing unit reduces the resolution of the first image by performing thinning processing or binning processing. (Configuration 23) 23. The image processing device according to any one of configurations 14 to 22, wherein the second image is a material decomposition image showing the thickness of a material in the subject, a total thickness image showing the sum of the thicknesses of different materials in the subject, a virtual monochromatic X-ray image, or a radiation image of any energy. (Configuration 24) 24. The image processing device according to any one of configurations 14 to 23, wherein the trained model is obtained using training data including a radiological image having a lower resolution than the first image and a third image having a lower resolution than the first image and being a different type of image from the first image. (Configuration 25) 25. The image processing device of claim 24, wherein the training data includes images obtained using forward projection based on three-dimensional CT data. (Configuration 26) 25. The image processing device according to claim 24, wherein the learning data includes images whose resolution has been reduced by performing thinning or binning. (Configuration 27) a radiation imaging device for detecting radiation; an image processing device according to any one of configurations 1 to 26, which is communicably connected to the radiation imaging device; A radiation imaging system comprising: (Method 1) acquiring a first image, the first image being a radiographic image; generating a second image using the first image, the second image being an energy subtraction image; Including, The generating step comprises: performing a process to reduce the resolution of the first image; inputting the first image with reduced resolution into a trained model to obtain a third image output from the trained model, the third image being a different type of image from the first image; performing a resolution enhancement process on the third image; generating a second image using the first image and the resolution-enhanced third image, the second image being a different type of image from the third image and having a higher resolution than the acquired third image; An image processing method comprising: (Method 2) acquiring a first image, the first image being a radiographic image; generating a second image using the first image, the second image being an energy subtraction image; Including, The generating step comprises: performing a process to reduce the resolution of the first image; inputting the first image with reduced resolution into a trained model to obtain a third image output from the trained model, the third image being a different type of image from the first image; generating a fourth image, which is an energy subtraction image, using the reduced resolution first image and the third image; performing a resolution enhancement process on the fourth image; generating a second image using the acquired first image and the resolution-enhanced fourth image, the second image being a different type of image from the fourth image and having a higher resolution than the generated fourth image; An image processing method comprising: (Program 1) A program that, when executed by a computer, causes the computer to execute the image processing method according to Method 1 or 2.

[0238] Although the present disclosure has been described above with reference to the embodiments and modifications, the present disclosure is not limited to the above embodiments and modifications. The present disclosure also includes inventions that have been modified within the scope of the present disclosure and inventions equivalent to the present disclosure. Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of the present invention. [Explanation of symbols]

[0239] 103: Control device (image processing device) 131: Acquisition Department 133: Processing section

Claims

1. an acquisition unit that acquires a first image that is a radiation image; a processing unit that uses the first image to generate a second image, which is an energy subtraction image; Equipped with The processing unit performing a resolution reduction process on the first image; By inputting the first image with reduced resolution into a trained model, a third image is obtained, which is an image of a different type from the first image and is output from the trained model; performing a resolution enhancement process on the third image; an image processing device that uses the first image and the third image with improved resolution to generate a second image that is a different type of image from the third image and has a higher resolution than the acquired third image.

2. The image processing device according to claim 1 , wherein the third image is a total thickness image showing the sum of thicknesses of different materials of the object, or a material decomposition image showing the thickness of materials of the object.

3. The processing unit a weighted addition of the logarithm of the first image and the resolution-enhanced third image; or The image processing device according to claim 2 , wherein the second image is generated by performing an operation using a nonlinear equation between the first image and the third image with improved resolution.

4. The image processing apparatus according to claim 1 , wherein the third image is a radiation image or a virtual monochromatic X-ray image of a different energy from the acquired first image.

5. The image processing device according to claim 4 , wherein the processing unit generates the second image using a weighted addition of the logarithm of the first image and the logarithm of the third image with improved resolution.

6. The image processing device according to claim 1 , wherein the processing unit applies a filter process to the third image whose resolution has been improved, and generates the second image using the third image that has been subjected to the filter process.

7. The processing unit applying a filter to the logarithm of the first image; performing a resolution enhancement process on the logarithm of the third image; applying a filter to the logarithm of the resolution-enhanced third image; The image processing device of claim 1 , wherein the second image is generated using a weighted addition of the logarithm of the filtered first image and the logarithm of the filtered third image.

8. The image processing device according to claim 1 , wherein the processing unit performs an interpolation process to improve the resolution of the third image.

9. The image processing device according to claim 1 , wherein the processing unit reduces the resolution of the first image by performing thinning or binning.

10. The image processing device according to claim 1 , wherein the second image is a material decomposition image showing the thickness of a material of the object, a total thickness image showing the sum of thicknesses of different materials of the object, a virtual monochromatic X-ray image, or a radiation image of any energy.

11. 2. The image processing device according to claim 1, wherein the trained model is obtained using training data including a radiological image having a lower resolution than the first image and a third image having a lower resolution than the first image and being a different type of image from the first image.

12. The image processing device according to claim 11 , wherein the learning data includes images obtained using forward projection based on three-dimensional CT data.

13. The image processing device according to claim 11 , wherein the learning data includes images whose resolution has been reduced by performing thinning or binning.

14. an acquisition unit that acquires a first image that is a radiation image; a processing unit that uses the first image to generate a second image, which is an energy subtraction image; Equipped with The processing unit performing a resolution reduction process on the first image; By inputting the first image with reduced resolution into a trained model, a third image is obtained, which is an image of a different type from the first image and is output from the trained model; generating a fourth image, which is an energy subtraction image, using the reduced resolution first image and the third image; performing a resolution enhancement process on the fourth image; an image processing device that uses the acquired first image and the fourth image with improved resolution to generate a second image that is a different type of image from the fourth image and has a higher resolution than the generated fourth image.

15. The image processing device according to claim 14 , wherein the fourth image is a total thickness image showing a sum of thicknesses of different materials of the object, or a material decomposition image showing thicknesses of materials of the object.

16. The processing unit a weighted addition of the logarithm of the first image and the resolution-enhanced fourth image; or The image processing device according to claim 15 , wherein the second image is generated using a calculation using a non-linear equation between the first image and the fourth image with improved resolution.

17. the third image is a material decomposition image showing a thickness of a material of the object, The processing unit a weighted addition of the logarithm of the reduced resolution first image and the third image; or The image processing device according to claim 14 , wherein the fourth image is generated by performing an operation using a nonlinear equation between the first image with reduced resolution and the third image.

18. The image processing device according to claim 14 , wherein the third image is a radiation image of a different energy from the first image or a virtual monochromatic X-ray image.

19. The image processing device according to claim 18 , wherein the processing unit generates the fourth image using a weighted addition of the logarithm of the first image with reduced resolution and the logarithm of the third image.

20. The image processing device according to claim 14 , wherein the processing unit applies a filter process to the fourth image whose resolution has been improved, and generates the second image using the fourth image that has been subjected to the filter process.

21. The image processing device according to claim 14 , wherein the processing unit performs an interpolation process to improve the resolution of the fourth image.

22. The image processing device according to claim 14 , wherein the processing unit reduces the resolution of the first image by performing thinning processing or binning processing.

23. The image processing device according to claim 14 , wherein the second image is a material decomposition image showing the thickness of a material of the object, a total thickness image showing the sum of thicknesses of different materials of the object, a virtual monochromatic X-ray image, or a radiation image of any energy.

24. 15. The image processing device according to claim 14, wherein the trained model is obtained using training data including a radiological image having a lower resolution than the first image and a third image having a lower resolution than the first image and being a different type of image from the first image.

25. The image processing device according to claim 24 , wherein the training data includes images obtained using forward projection based on three-dimensional CT data.

26. The image processing device according to claim 24 , wherein the learning data includes images whose resolution has been reduced by performing thinning or binning.

27. a radiation imaging device for detecting radiation; an image processing device according to any one of claims 1 to 26, which is communicably connected to the radiation imaging device; A radiation imaging system comprising:

28. acquiring a first image, the first image being a radiographic image; generating a second image using the first image, the second image being an energy subtraction image; Including, The generating step comprises: performing a process to reduce the resolution of the first image; By inputting the first image with reduced resolution into a trained model, a third image that is an image of a different type from the first image is obtained, which is output from the trained model; and performing a resolution enhancement process on the third image; generating a second image using the first image and the resolution-enhanced third image, the second image being a different type of image from the third image and having a higher resolution than the acquired third image; An image processing method comprising:

29. acquiring a first image, the first image being a radiographic image; generating a second image using the first image, the second image being an energy subtraction image; Including, The generating step comprises: performing a process to reduce the resolution of the first image; By inputting the first image with reduced resolution into a trained model, a third image that is an image of a different type from the first image is obtained, which is output from the trained model; and generating a fourth image, which is an energy subtraction image, using the reduced resolution first image and the third image; performing a resolution enhancement process on the fourth image; generating a second image using the acquired first image and the resolution-enhanced fourth image, the second image being a different type of image from the fourth image and having a higher resolution than the generated fourth image; An image processing method comprising:

30. 30. A program that, when executed by a computer, causes the computer to execute the image processing method according to claim 28 or 29.

Images