Image processing apparatus, radiation imaging system, image processing method, and storage medium

Abstract

Provided is an image processing device that can reduce artifacts caused by the structure of a subject in a case in which an ES image is generated from a radiographic image using a learned model.SOLUTION: An image processing apparatus comprising: an acquisition unit configured to acquire a radiation image; and a processing unit configured to acquire a total thickness image output from a learned model by inputting the acquired radiation image to the learned model obtained using learning data including the radiation image and the total thickness image indicating a sum of thicknesses of different substances of a subject.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, a radiation imaging system, an image processing method, and a program. [Background technology]

[0002] A commonly known imaging technique is time-multiplexed spectral imaging. In time-multiplexed spectral imaging, multiple radiation beams with different average energies are irradiated onto the subject over a short period of time, and the constituent materials of the subject are distinguished by measuring the proportion of radiation beams with each average energy that pass through the subject and reach the radiation measurement surface. Specifically, energy subtraction (ES) processing of acquired radiation images with different energies (pairs of high- and low-energy images in the case of two energies) can be used to obtain energy subtraction images such as bone images and soft tissue images.

[0003] This type of time-division spectral imaging is also used to generate medical radiological images, as in the technology described in Patent Document 1. Patent Document 2 proposes a method for improving the image quality of energy subtraction images acquired by these methods using an image generation model based on machine learning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-162358 [Patent Document 2] Japanese Patent Publication No. 2023-120851 Summary of the Invention [Problem to be solved by the invention]

[0005] However, as with the technology described in Patent Document 2, when training a machine learning model using ES images with complex subject structures, such as bone images or soft tissue images, artifacts due to the subject structure may occur in images generated using the trained model.

[0006] Therefore, one object of one embodiment of the present disclosure is to provide an image processing device that can reduce artifacts caused by the structure of the subject when generating an ES image from a radiological image using a trained model. [Means for solving the problem]

[0007] An image processing device according to one embodiment of the present disclosure includes an acquisition unit that acquires a radiological image, and a processing unit that acquires a total thickness image output from a trained model obtained using training data including the radiological image and a total thickness image that indicates the sum of the thicknesses of different materials in a subject, by inputting the acquired radiological image into the trained model. [Effects of the Invention]

[0008] According to one embodiment of the present disclosure, when generating an ES image from a radiological image using a trained model, artifacts caused by the structure of the subject can be reduced. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows an example of the overall configuration of a radiation imaging system according to a first embodiment. [Figure 2] FIG. 2 is an equivalent circuit diagram of an example of a pixel of the radiation imaging device. [Figure 3] 10 is a timing chart showing an example of a radiation imaging operation. [Figure 4] 10 is a timing chart showing an example of a radiation imaging operation. [Figure 5] FIG. 10 is a block diagram of a correction process included in the preprocessing. [Figure 6] FIG. 1 is a block diagram of signal processing for ES processing. [Figure 7] FIG. 10 is a block diagram of a process for generating a total thickness image. [Figure 8] FIG. 10 is a block diagram of a process for generating a virtual monochromatic X-ray image. [Figure 9] 10 is a flowchart illustrating an example of a learning process. [Figure 10] FIG. 10 is a block diagram of a learning process. [Figure 11] 1 shows an example of the configuration of a machine learning model. [Figure 12] 3 shows an example of learning data according to the first embodiment. [Figure 13] 4 is a flowchart showing an example of a series of processes according to the first embodiment. [Figure 14] FIG. 1 is a block diagram of an inference process using a trained model. [Figure 15] 3A to 3C show examples of input and output images during learning and inference according to the first embodiment. [Figure 16] FIG. 10 is a diagram for explaining a total thickness image. [Figure 17] 10 is a flowchart showing an example of a series of processes according to the second embodiment. [Figure 18] FIG. 10 is a block diagram showing an example of a series of processes according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments are arbitrary and can be changed depending on the configuration of an apparatus to which the present disclosure is applied or various conditions. In addition, the same reference numerals are used in the drawings to indicate identical or functionally similar elements.

[0011] In this disclosure, radiation includes α-rays, β-rays, γ-rays, and the like, which are beams created by particles (including photons) emitted by radioactive decay, as well as beams with the same or higher energy levels, such as X-rays, particle beams, and cosmic rays.

[0012] Energy subtraction processing (ES processing) refers to processing that calculates differences using images (energy images) related to different radiation energies to obtain, for example, material decomposition images of bone and soft tissue, contrast agent and water, etc., information on effective atomic number and surface density, etc. The energy subtraction processing may include, for example, correction processing such as offset correction processing as pre-processing, and image processing such as contrast adjustment processing and total thickness image generation processing as post-processing.

[0013] The term "energy subtraction image (ES image)" can include, for example, a material decomposition image obtained using ES processing, an image showing effective atomic number and surface density, etc. Furthermore, the term "ES image" can also include a material decomposition image obtained using an approximation formula for ES processing described below, a radiation image of any energy, etc. Furthermore, the term "ES image" can also include an image inferred using a trained model obtained by training an image obtained using the above-mentioned ES processing, a material decomposition image obtained using three-dimensional data from CT (Computed Tomography), etc. Furthermore, hereinafter, the term "ES image" can also include a total thickness image generated by adding together material decomposition images such as bone images and soft tissue images.

[0014] In the following, a machine learning model refers to a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor algorithms, naive Bayes algorithms, decision trees, and support vector machines. Another example is deep learning, which uses a neural network to generate features and connection weighting coefficients for learning. Any of the above algorithms that can be used can be used as appropriate and applied to the following embodiments. Furthermore, training data refers to training data, and is composed of a pair of input data and output data. Furthermore, supervised answer data refers to output data from training data (training data).

[0015] Furthermore, a trained model refers to a machine learning model that follows any machine learning algorithm, such as deep learning, and that has been trained (learned) in advance using appropriate training data (learning data). However, although a trained model is obtained in advance using appropriate training data, it does not mean that no further learning is performed, and additional learning can also be performed. Additional learning can also be performed after the device is installed at the site of use. Note that obtaining output data from input data using a trained model is sometimes referred to as inference.

[0016] (First embodiment) As described above, when a machine learning model is trained using ES images with complex subject structures, such as bone images or soft tissue images, as learning data, artifacts due to the subject structure may appear in the ES images generated using the trained model. In such cases, appropriate diagnosis, image analysis, etc. may not be possible based on the artifacts that appear in the generated ES images.

[0017] Therefore, in this embodiment, a total thickness image, which is an ES image that simplifies the structure of the object to be depicted, is used as training data to obtain a trained model, and the trained model is used to acquire a total thickness image, which is an ES image, from a radiographic image. Hereinafter, a radiographic imaging system, an image processing device, and an image processing method according to this embodiment will be described with reference to FIGS.

[0018] (Configuration of Radiation Imaging System) 1 is a block diagram showing an example of the overall configuration of a radiation imaging system according to this embodiment. The radiation imaging system according to this embodiment includes a radiation generating device 101, a radiation control device 102, a control device 103, a radiation imaging device 104, an input unit 150, and a display unit 120.

[0019] The radiation generating device 101 includes a radiation source such as a tube. The radiation generating device 101 generates radiation under the control of the radiation control device 102. The radiation control device 102 includes a control circuit, a processor, etc. Based on the control of the control device 103, the radiation control device 102 controls the radiation generating device 101 so that radiation is irradiated toward the subject Su and the radiation imaging device 104. More specifically, the radiation control device 102 can control imaging conditions such as the irradiation angle of radiation emitted by the radiation generating device 101, the radiation focal position, the tube voltage, and the tube current.

[0020] The radiation control device 102, the radiation imaging device 104, the input unit 150, and the display unit 120 are connected to the control device 103, and the control device 103 can control these. The control device 103 can perform, for example, various controls related to radiation imaging and image processing for generating ES images.

[0021] The control device 103 is provided with an acquisition unit 131, a generation unit 132, a processing unit 133, a display control unit 134, and a storage unit 135. The acquisition unit 131 can acquire images captured by the radiation imaging device 104, images generated by the generation unit 132, etc. The acquisition unit 131 can also acquire various images from an external device (not shown) connected to the control device 103 via a network such as the Internet.

[0022] The generation unit 132 can generate a radiation image from an image (image information) captured using the radiation imaging device 104 and acquired by the acquisition unit 131. The generation unit 132 can generate energy images (high energy image and low energy image) related to radiation energy from an image captured using the radiation imaging device 104 by irradiating radiation of any energy, for example. A method for generating an energy image will be described later.

[0023] The processing unit 133 generates an ES image based on an energy image (radiographic image) using the trained model. The trained model and a method for generating an ES image will be described later. The processing unit 133 can also perform image processing, analysis processing, and the like using the generated ES image, etc. Furthermore, the processing unit 133 can also function as an example of a learning unit that trains a machine learning model.

[0024] The display control unit 134 controls the display of the display unit 120 and can display, for example, information on the subject Su (examination object), information on radiography, various acquired and generated images, etc. on the display unit 120. The storage unit 135 can store, for example, information on the subject Su, information on radiography, and various acquired and generated images. The storage unit 135 can also store programs and the like for the control device 103 to perform various processes.

[0025] Here, the control device 103 can be configured by a computer provided with a processor and a memory. The control device 103 may be configured by a general computer or a computer dedicated to the radiation control system. The control device 103 may be, for example, a personal computer, and a desktop PC, a notebook PC, a tablet PC (portable information terminal), or the like may be used. Furthermore, the control device 103 may be configured as a cloud-based computer in which some of the components are located in an external device.

[0026] Furthermore, each component of the control device 103 other than the storage unit 135 may be configured by a software module executed by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor may be, for example, a GPU (Graphical Processing Unit) or an FPGA (Field-Programmable Gate Array). Each component may be configured by a circuit that performs a specific function, such as an ASIC. The storage unit 135 may be configured by any storage medium, such as an optical disk such as a hard disk, or a memory.

[0027] The display unit 120 is configured with any monitor, and displays various information such as information about the subject Su, various images, and a mouse cursor in accordance with the operation of the input unit 150, under the control of the display control unit 134. The input unit 150 is an input device that issues instructions to the control device 103, and specifically includes a keyboard and a mouse. Note that the display unit 120 may be configured with a touch panel display, and in this case, the display unit 120 can also be used as the input unit 150.

[0028] The radiation imaging device 104 detects radiation that is irradiated from the radiation generation device 101 and passes through the subject Su, and captures a radiation image. The radiation imaging device 104 can be configured as, for example, an FPD. The radiation imaging device 104 is provided with a phosphor 141 that converts radiation into visible light, and a two-dimensional detector 142 that detects the visible light. The two-dimensional detector 142 includes a sensor in which pixels 20 that detect visible light are arranged in an array of X columns and Y rows, and outputs image information according to the detected radiation dose.

[0029] An example of the pixel 20 will now be described with reference to Fig. 2. Fig. 2 shows an equivalent circuit diagram of an example of the pixel 20. The pixel 20 is provided with a photoelectric conversion element 201 and an output circuit unit 202. The photoelectric conversion element 201 may typically include a photodiode. The output circuit unit 202 is provided with an amplifier circuit unit 204, a clamp circuit unit 206, a sample-and-hold circuit unit 207, and a selection circuit unit 208.

[0030] The photoelectric conversion element 201 includes a charge accumulation section, which is connected to the gate of a MOS transistor 204a in the amplifier circuit section 204. The source of the MOS transistor 204a is connected to a current source 204c via a MOS transistor 204b. The MOS transistor 204a and the current source 204c form a source follower circuit. The MOS transistor 204b is an enable switch that turns on when an enable signal EN supplied to its gate becomes active level, putting the source follower circuit into an operating state.

[0031] In the example shown in FIG. 2, the charge storage section of the photoelectric conversion element 201 and the gate of the MOS transistor 204a form a common node. This node functions as a charge-voltage converter that converts the charge stored in the charge storage section of the photoelectric conversion element 201 into a voltage. A voltage V (=Q / C) appears in the charge-voltage converter, which is determined by the charge Q stored in the charge storage section and the capacitance C of the charge-voltage converter. The charge-voltage converter is connected to a reset potential Vres via a reset switch 203. When a reset signal PRES becomes active level, the reset switch 203 is turned on, and the potential of the charge-voltage converter is reset to the reset potential Vres.

[0032] The clamp circuit unit 206 clamps the noise output by the amplifier circuit unit 204 in response to the reset potential of the charge-voltage converter using a clamp capacitor 206a. In other words, the clamp circuit unit 206 is a circuit for canceling the noise described above from the signal output from the source follower circuit in response to the charge generated by photoelectric conversion in the photoelectric conversion element 201. This noise includes kTC noise generated during reset. Clamping is achieved by first setting the clamp signal PCL to an active level to turn on the MOS transistor 206b, and then setting the clamp signal PCL to an inactive level to turn off the MOS transistor 206b. The output side of the clamp capacitor 206a is connected to the gate of the MOS transistor 206c. The source of the MOS transistor 206c is connected to a current source 206e via a MOS transistor 206d. The MOS transistor 206c and the current source 206e form a source follower circuit. The MOS transistor 206d is an enable switch that turns on when an enable signal EN0 supplied to its gate becomes active, activating the source follower circuit.

[0033] A signal output from the clamp circuit unit 206 in response to charges generated by photoelectric conversion of the photoelectric conversion element 201 is written as an optical signal to the capacitor 207Sb via the switch 207Sa when the optical signal sampling signal TS becomes active. The signal output from the clamp circuit unit 206 when the MOS transistor 206b is turned on immediately after resetting the potential of the charge-voltage converter is a clamp voltage. The clamp voltage is written as noise to the capacitor 207Nb via the switch 207Na when the noise sampling signal TN becomes active. This noise includes an offset component of the clamp circuit unit 206. The switch 207Sa and the capacitor 207Sb form a signal sample-and-hold circuit 207S, and the switch 207Na and the capacitor 207Nb form a noise sample-and-hold circuit 207N. The sample-and-hold circuit unit 207 includes the signal sample-and-hold circuit 207S and the noise sample-and-hold circuit 207N.

[0034] When the driving circuit unit drives the row selection signal to an active level, the signal (optical signal) held in the capacitor 207Sb is output to the signal line 21S via the MOS transistor 208Sa and the row selection switch 208Sb. At the same time, the signal (noise) held in the capacitor 207Nb is output to the signal line 21N via the MOS transistor 208Na and the row selection switch 208Nb. The MOS transistor 208Sa forms a source follower circuit together with a constant current source (not shown) provided on the signal line 21S. Similarly, the MOS transistor 208Na forms a source follower circuit together with a constant current source (not shown) provided on the signal line 21N. The MOS transistor 208Sa and the row selection switch 208Sb form the signal selection circuit unit 208S, and the MOS transistor 208Na and the row selection switch 208Nb form the noise selection circuit unit 208N. The selection circuit section 208 includes a signal selection circuit section 208S and a noise selection circuit section 208N.

[0035] The pixel 20 may have an addition switch 209S that adds optical signals from multiple adjacent pixels 20. In addition mode, an addition mode signal ADD becomes active, and the addition switch 209S is turned on. As a result, the capacitances 207Sb of adjacent pixels 20 are connected to each other by the addition switch 209S, and the optical signals are averaged. Similarly, the pixel 20 may have an addition switch 209N that adds noises from multiple adjacent pixels 20. When the addition switch 209N is turned on, the capacitances 207Nb of adjacent pixels 20 are connected to each other by the addition switch 209N, and the noises are averaged. The adder 209 includes an addition switch 209S and an addition switch 209N.

[0036] The pixel 20 may have a sensitivity change unit 205 for changing the sensitivity. The pixel 20 may include, for example, a first sensitivity change switch 205a, a second sensitivity change switch 205'a, and associated circuit elements. When the first change signal WIDE becomes active, the first sensitivity change switch 205a is turned on, and the capacitance value of the first additional capacitor 205b is added to the capacitance value of the charge-voltage conversion unit. This reduces the sensitivity of the pixel 20. When the second change signal WIDE2 becomes active, the second sensitivity change switch 205'a is turned on, and the capacitance value of the second additional capacitor 205'b is added to the capacitance value of the charge-voltage conversion unit. This further reduces the sensitivity of the pixel 20. By adding a function to reduce the sensitivity of the pixel 20 in this way, it becomes possible to receive a larger amount of light and widen the dynamic range. When the first change signal WIDE becomes active, the enable signal ENw may be turned on to cause the MOS transistor 204'a to operate as a source follower instead of the MOS transistor 204a.

[0037] The radiation imaging device 104 reads the output of the pixel circuit as described above and converts it into digital values ​​(image information) using an AD converter (not shown). The radiation imaging device 104 transfers the image information converted into digital values ​​to the control device 103. This allows the acquisition unit 131 of the control device 103 to acquire the radiographed image.

[0038] (Radiation imaging operation for ES processing) Next, before describing the processing according to this embodiment, the radiation imaging operation for performing the ES processing and the ES processing will be described with reference to FIGS.

[0039] Figures 3 and 4 show examples of various drive timings in radiography operations for performing ES processing in a radiographic imaging system. Figure 3 shows an example of radiography operations when a relatively inexpensive tube that does not allow switching of tube voltage (energy) is used, while Figure 4 shows an example of radiography operations when a tube that does allow switching of tube voltage is used. The waveforms in the figures, with time on the horizontal axis, show the timing of X-ray exposure, synchronization signal, resetting of photoelectric conversion element 201, driving of sample-and-hold circuit unit 207, and readout of image from signal line 21. Note that in Figures 3 and 4, the waveform for X-rays represents tube voltage. Also, while black and white areas are provided for X-rays, this is simply done to make it easier to distinguish the timing.

[0040] First, the example shown in FIG. 3 will be described. In this example, X-rays are emitted after the photoelectric conversion element 201 is reset. Ideally, the X-ray tube voltage is a rectangular wave, but it takes a finite amount of time for the tube voltage to rise and fall. In particular, when the exposure time is short with pulsed X-rays, the tube voltage can no longer be considered a rectangular wave, and takes on a waveform like that shown in the X-ray waveform in FIG. 3. For this reason, the energy of the X-rays differs during the rise, stable, and fall periods of the X-rays.

[0041] Therefore, after X-rays 301 in the rising phase are irradiated, sampling is performed by the noise sample and hold circuit 207N, and further, after X-rays 302 in the stable phase are irradiated, sampling is performed by the signal sample and hold circuit 207S. Thereafter, the difference between the signal on signal line 21N and the signal on signal line 21S is read out as an image. At this time, the signal (R1) of X-rays 301 in the rising phase is held in the noise sample and hold circuit 207N, and the sum of the signal (R1) of X-rays 301 in the rising phase and the signal (B) of X-rays 302 in the stable phase is held in the signal sample and hold circuit 207S. Therefore, an image 304 corresponding to the signal (B) of X-rays 302 in the stable phase is read out as the difference between the signal on signal line 21N and the signal on signal line 21S.

[0042] Next, after the irradiation of X-rays 303 in the falling phase and the readout of image 304 are completed, sampling is again performed by signal sample-and-hold circuit 207S. Thereafter, photoelectric conversion element 201 is reset, and sampling is again performed by noise sample-and-hold circuit 207N, and the difference between the signal on signal line 21N and the signal on signal line 21S is read out as an image. At this time, the noise sample-and-hold circuit 207N holds a signal in a state where no X-rays are irradiated. Furthermore, the signal sample-and-hold circuit 207S holds the sum of the signal (R1) of X-rays 301 in the rising phase, the signal (B) of X-rays 302 in the stable phase, and the signal (R2) of X-rays 303 in the falling phase. Therefore, image 306, which corresponds to the sum of the signal (R1) of X-rays 301 in the rising phase, the signal (B) of X-rays 302 in the stable phase, and the signal (R2) of X-rays 303 in the falling phase, is read out as the difference between the signal on signal line 21N and the signal on signal line 21S. Thereafter, by calculating the difference between image 306 and image 304, image 305 corresponding to the sum of the signal (R1) of X-ray 301 in the rising phase and the signal (R2) of X-ray 303 in the falling phase is obtained.

[0043] The timing for resetting the sample-and-hold circuit unit 207 and the photoelectric conversion element 201 is determined using a synchronization signal 307 indicating that X-ray exposure has started from the radiation generation device 101. A method for detecting the start of radiation exposure may include measuring the tube current of the radiation generation device 101 and determining whether the current value exceeds a predetermined threshold. Alternatively, a configuration may be used in which, after resetting of the photoelectric conversion element 201 is completed, signals from the pixels 20 are repeatedly read out and whether the pixel values ​​exceed a predetermined threshold is determined. Furthermore, a configuration may be used in which an X-ray detector other than the two-dimensional detector 142 is built into the radiation imaging device 104 and whether the measured value exceeds a predetermined threshold is determined. In either case, sampling by the signal sample-and-hold circuit 207S, sampling by the noise sample-and-hold circuit 207N, and resetting of the photoelectric conversion element 201 are performed after a predetermined time has elapsed since the input of the synchronization signal 307.

[0044] In this way, it is possible to obtain an image 304 corresponding to the stable period of the pulsed X-rays and an image 305 corresponding to the sum of the rising and falling periods of the pulsed X-rays. Because the energy of the X-rays irradiated when generating these two images is different, energy subtraction processing can be performed by performing calculations between the two images.

[0045] Next, an example of radiation imaging operation when using a tube with switchable tube voltage will be described with reference to Fig. 4. This example differs from the example shown in Fig. 3 in that the X-ray tube voltage is actively switched.

[0046] In this example, first, the photoelectric conversion element 201 is reset, and then low-energy X-rays 401 are irradiated. Then, sampling is performed by the noise sample-and-hold circuit 207N, and the tube voltage is switched to irradiate high-energy X-rays 402. After the high-energy X-rays 402 are irradiated, sampling is performed by the signal sample-and-hold circuit 207S. Then, the tube voltage is switched to irradiate low-energy X-rays 403. Furthermore, the difference between the signals on the signal line 21N and the signal line 21S is read out as an image. At this time, the noise sample-and-hold circuit 207N holds the signal (R1) of the low-energy X-rays 401, and the signal sample-and-hold circuit 207S holds the sum of the signal (R1) of the low-energy X-rays 401 and the signal (B) of the high-energy X-rays 402. Therefore, an image 404 corresponding to the signal (B) of the high-energy X-ray 402 is read out as the difference between the signal on the signal line 21N and the signal on the signal line 21S.

[0047] Next, after the irradiation of low-energy X-rays 403 and the readout of image 404 are completed, sampling is again performed by signal sample-and-hold circuit 207S. Thereafter, photoelectric conversion element 201 is reset, and sampling is again performed by noise sample-and-hold circuit 207N, and the difference between the signal on signal line 21N and the signal on signal line 21S is read out as an image. At this time, the noise sample-and-hold circuit 207N holds a signal in a state where no X-rays are irradiated. Furthermore, the signal sample-and-hold circuit 207S holds the sum of the signal (R1) of low-energy X-rays 401, the signal (B) of high-energy X-rays 402, and the signal (R2) of low-energy X-rays 403. Therefore, an image 406 corresponding to the sum of the signal (R1) of low-energy X-rays 401, the signal (B) of high-energy X-rays 402, and the signal (R2) of low-energy X-rays 403 is read out as the difference between the signal on signal line 21N and the signal on signal line 21S. Thereafter, by calculating the difference between image 406 and image 404, image 405 corresponding to the sum of the signal (R1) of low-energy X-ray 401 and the signal (R2) of low-energy X-ray 403 is obtained.

[0048] The synchronization signal 407 is the same as in the example shown in Fig. 3. In this way, by acquiring images while actively switching the tube voltage, it is possible to make the energy difference between low-energy images and high-energy images larger than in the method of Fig. 3. The method of acquiring low-energy images and high-energy images is not limited to this. For example, an FPD including a stacked sensor formed of two types of materials (phosphors) with different X-ray absorption rates may be used as the radiation imaging device 104, and low-energy and high-energy images may be acquired based on the outputs from the sensors of each layer obtained by a single exposure.

[0049] (ES processing) Next, the ES processing method will be described. In addition to the signal processing of the ES processing, the ES processing can include correction processing as pre-processing and image processing as post-processing.

[0050] First, the correction process, which is pre-processing, will be described with reference to Fig. 5. Fig. 5 is a block diagram of the correction process. Note that in the following, this embodiment will be described assuming that a radiation imaging operation is performed in accordance with the example shown in Fig. 3.

[0051] 3 without irradiating the radiation imaging device 104 with X-rays, and the acquisition unit 131 acquires the captured images. At this time, two images corresponding to images 304 and 306 are read out, with the first image (image 304) being image F_Odd and the second image (image 306) being image F_Even. Images F_Odd and F_Even are images corresponding to fixed pattern noise (FPN) of the radiation imaging device 104.

[0052] Next, in a state where there is no subject, the radiation imaging device 104 is irradiated with X-rays to perform imaging using the driving method shown in Fig. 3, and the acquisition unit 131 acquires the captured images. At this time, two images corresponding to images 304 and 306 are read out, with the first image (image 304) being image W_Odd and the second image (image 306) being image W_Even. Images W_Odd and W_Even are images corresponding to the sum of signals due to the FPN of the radiation imaging device 104 and X-rays.

[0053] Therefore, by subtracting image F_Odd from image W_Odd and image F_Even from image W_Even, images WF_Odd and WF_Even can be obtained from which the FPN of the radiation imaging device 104 has been removed. In this embodiment, such correction processing is called offset correction.

[0054] Image WF_Odd is an image corresponding to X-rays 302 in the stable period, and image WF_Even is an image corresponding to the sum of X-rays 301 in the rising period, X-rays 302 in the stable period, and X-rays 303 in the falling period. Therefore, by subtracting image WF_Odd from image WF_Even, an image corresponding to the sum of X-rays 301 in the rising period and X-rays 303 in the falling period is obtained. The energies of X-rays 301 in the rising period and X-rays 303 in the falling period are lower than the energy of X-rays 302 in the stable period. Therefore, by subtracting image WF_Odd from image WF_Even, a low-energy image W_Low in the absence of an object is obtained. Furthermore, a high-energy image W_High in the absence of an object is obtained from image WF_Odd. In this embodiment, this correction process is called color correction.

[0055] Next, with a subject present, the radiation imaging device 104 is irradiated with X-rays to perform imaging using the driving method shown in Fig. 3, and the acquisition unit 131 acquires the captured images. At this time, two images corresponding to images 304 and 306 are read out, with the first image (image 304) designated as image X_Odd and the second image (image 306) designated as image X_Even. The generation unit 132 performs offset correction and color correction on these images in the same manner as when there is no subject, thereby obtaining a low-energy image X_Low when there is a subject and a high-energy image X_High when there is a subject.

[0056] Here, if the thickness of the subject is d, the linear attenuation coefficient of the subject is μ, the output of pixel 20 when there is no subject is I0, and the output of pixel 20 when there is a subject is I, the following equation (1) holds.

number

number

[0057] Therefore, by dividing the low-energy image X_Low when there is an object by the low-energy image W_Low when there is no object, the image of the attenuation rate at low energy (low-energy image Im L ) is obtained. Similarly, by dividing the high-energy image X_High when there is an object by the high-energy image W_High when there is no object, an image of the attenuation rate H at high energy (high-energy image Im H Hereinafter, such a correction process will be referred to as gain correction. In the radiation imaging system, the generation unit 132 performs correction processes including the offset correction, color correction, and gain correction as described above, thereby obtaining a low-energy image Im L and high-energy image Im H can be generated and obtained.

[0058] Next, the signal processing of the ES processing will be described with reference to Fig. 6. Fig. 6 is a block diagram of the signal processing of the ES processing. In the signal processing of the ES processing, the low-energy image Im obtained by the correction processing described with reference to Fig. L and high energy images H From the bone thickness image (bone image Im B ) and soft tissue thickness image (soft tissue image Im S ) is found.

[0059] First, let E be the energy of the X-ray photon, N(E) be the number of photons at energy E, B be the thickness of the bone, and S be the thickness of the soft tissue. Let μ be the linear attenuation coefficient of the bone at energy E. B (E), the linear attenuation coefficient of soft tissue at energy E is μ S (E), if the attenuation ratio is I / I0, the following equation (3) holds.

number

[0060] The number of photons N(E) at energy E is the X-ray spectrum. The X-ray spectrum can be obtained by simulation or actual measurement for the tube voltage. Also, the linear attenuation coefficient μ of bone at energy E isB (E) and the linear attenuation coefficient μ of soft tissue at energy E S (E) can be obtained from databases such as those of the National Institute of Standards and Technology (NIST). Therefore, by using equation (3), it is possible to calculate the attenuation ratio I / I0 for any bone thickness B, soft tissue thickness S, and X-ray spectrum N(E).

[0061] Here, the spectrum of low energy X-rays is N L (E) High-energy X-ray spectrum H If (E), the following equation (4) holds.

number

[0062] Here, we will explain the case where the Newton-Raphson method, which is a type of iterative solution, is used as a typical method for solving nonlinear simultaneous equations. First, let us define the number of iterations of the Newton-Raphson method as m, and the bone thickness after the mth iteration as B. m , the thickness of the soft tissue after the mth iteration is S m Then, the attenuation rate of high energy after the mth iteration, H m , and the low-energy attenuation rate L after the mth iteration m is expressed by the following equation (5).

number

number

[0063] At this time, the bone thickness B after the m+1th iteration m+1 and soft tissue thickness S m+1 is expressed by the following equation (7) using the attenuation rate of high energy H and the attenuation rate of low energy L.

number

number

number

[0064] By repeating this calculation, the high-energy attenuation rate H m The difference between the measured high-energy attenuation rate H and the measured high-energy attenuation rate L approaches 0. The same is true for the low-energy attenuation rate L. As a result, the bone thickness B after the mth iteration m converges to the bone thickness B, and the m-th soft tissue thickness S m converges to the thickness S of the soft tissue. In this way, the nonlinear simultaneous equations shown in equation (4) can be solved. Therefore, by calculating equation (4) for all pixels, the low-energy image Im L and high energy images H From the bone image B and soft tissue imaging S can be obtained.

[0065] In this example, for the sake of simplicity, the bone thickness B and the soft tissue thickness S are calculated by ES processing, but this embodiment is not limited to this. For example, the thickness of water and the thickness of a contrast agent may be calculated by ES processing. In this case, the linear attenuation coefficient of water at energy E and the linear attenuation coefficient of a contrast agent at energy E may also be obtained from a database such as NIST. The ES processing makes it possible to calculate the thickness of any two types of material. Furthermore, an image of the effective atomic number Z and an image of the surface density D may be obtained from an image of the attenuation rate L at low energy and an image of the attenuation rate H at high energy obtained by the correction shown in FIG. 5. The effective atomic number Z is the equivalent atomic number of the mixture, and the surface density D is the density [g / cm] of the subject. 3 ] and the thickness of the subject [cm].

[0066] In the above description, the nonlinear simultaneous equations are solved using the Newton-Raphson method. However, the method for solving the nonlinear simultaneous equations is not limited to this. For example, iterative methods such as the least squares method and bisection method may also be used. Furthermore, the method for calculating the bone thickness B and the soft tissue thickness S is not limited to solving the nonlinear simultaneous equations using an iterative method. For example, the bone thickness B and the soft tissue thickness S for various combinations of high-energy attenuation rate H and low-energy attenuation rate L may be calculated in advance to generate a table, and the bone thickness B and the soft tissue thickness S may be calculated quickly by referring to the table.

[0067] Next, image processing for generating a total thickness image and a virtual monochromatic X-ray image in post-processing of such ES processing will be described. First, the processing for generating a total thickness image will be described with reference to FIG. 7. FIG. 7 is a block diagram of image processing for generating a total thickness image. Here, the total thickness image is an image showing the sum of the thicknesses of different materials in the subject, and can be generated by adding together multiple material decomposition images each showing the thickness of a different material. In the image processing shown in FIG. 7, the bone thickness image (bone image Im) obtained by the signal processing shown in FIG. 6 is B ) and soft tissue thickness image (soft tissue image Im S) and the sum of these images is the total thickness image Im T can be obtained.

[0068] In the human body, bones are surrounded by soft tissue, so the thickness of the soft tissue in the area where the bone is located is reduced by the thickness of the bone. S When gradation processing is applied to the image, the soft tissue image Im S On the other hand, bone contrast appears in the soft tissue image Im S and bone image Im B Adding , backfills the loss of bone thickness in the soft tissue image. Therefore, the total thickness image Im T is an image in which the contrast of bones has been removed, in other words, an image in which the structure of the object to be depicted is simplified. T A time-direction filter such as a recursive filter or a space-direction filter such as a Gaussian filter may be applied to the image to reduce noise, and then the image may be subjected to gradation processing and displayed.

[0069] Next, image processing for generating a virtual monochromatic X-ray image will be described with reference to Fig. 8. Fig. 8 is a block diagram of image processing for generating a virtual monochromatic X-ray image. In the image processing shown in Fig. 8, the bone image Im obtained by the signal processing shown in Fig. 6 is B and soft tissue imaging Im S Here, a virtual monochromatic X-ray image is an image that is assumed to be obtained when X-rays of a single energy are irradiated.

[0070] Virtual monochromatic X-ray images are used in Dual Energy CT, which combines ES processing and three-dimensional reconstruction. Virtual monochromatic X-ray images can suppress beam hardening artifacts and metal artifacts. For example, when the energy of virtual monochromatic X-rays is set to E V Then, the virtual monochromatic X-ray image V is calculated by the following equation (10).

number

[0071] In addition, the energy E of the virtual monochromatic X-ray V By changing the linear attenuation coefficient μ of bone, the CNR (contrast to noise ratio) of the virtual monochromatic X-ray image can be improved. B (E) is the linear attenuation coefficient μ of soft tissue S (E). However, the energy E V The larger the difference between the linear attenuation coefficient μB(E) of bone and the linear attenuation coefficient μS(E) of soft tissue becomes smaller. Therefore, the energy E of the virtual monochromatic X-ray V By setting a large value, the increase in noise in the virtual monochromatic X-ray image due to noise in the bone image is suppressed. On the other hand, the virtual monochromatic X-ray energy E V The smaller the linear attenuation coefficient μ of bone, B (E) and the linear attenuation coefficient μ of soft tissue S The contrast of the virtual monochromatic X-ray image increases because the difference in energy E V By setting an appropriate value, the CNR of the virtual monochromatic X-ray image can be improved, and for example, the amount of contrast agent used in radiography can be reduced.

[0072] In this embodiment, a virtual monochromatic X-ray image is generated from the bone thickness B and the soft tissue thickness S, but the present disclosure is not limited to this form. After calculating the effective atomic number Z and the surface density D by ES processing, a virtual monochromatic X-ray image may be generated using the effective atomic number Z and the surface density D. In addition, when a plurality of energies E V A composite X-ray image may be generated by combining multiple virtual monochromatic X-ray images generated in step 1. A composite X-ray image is an image that is expected to be obtained when irradiating an object with X-rays of a given spectrum.

[0073] (Learning process) In the above description, bone images and soft tissue images are generated by ES processing using high-energy images and low-energy images. In contrast, in this embodiment, a machine learning model is used to generate a total thickness image based on a radiological image, which is an input image. Here, the learning process of the machine learning model according to this embodiment will be described with reference to Figs. 9 to 12. Fig. 9 is a flowchart showing an example of the learning process of the machine learning model according to this embodiment.

[0074] In the learning process of the machine learning model according to this embodiment, learning data is prepared in step S901. In the learning data according to this embodiment, radiation images such as high-energy images or low-energy images are used as input data, and total thickness images are used as output data. Note that either high-energy images or low-energy images may be used as input data for the learning data. In the following, an example will be described in which high-energy images are used as input data and total thickness images are used as output data.

[0075] As shown in FIGS. 1 to 6, the training data can be prepared by acquiring captured high-energy images and low-energy images, performing ES processing to acquire bone images and soft-tissue images, and generating a total thickness image by adding the bone images and soft-tissue images. More specifically, the acquisition unit 131 acquires the captured high-energy images and low-energy images. Then, the processing unit 133 performs ES processing on the high-energy images and low-energy images to generate bone images and soft-tissue images. The processing unit 133 also adds the generated bone images and soft-tissue images to generate a total thickness image. This allows the control device 103 to acquire radiographic images to be used as input data for the training data and a total thickness image to be used as output data for the training data.

[0076] Alternatively, a virtual monochromatic X-ray image may be generated from the bone image and the soft tissue image, the virtual monochromatic X-ray image may be used as input data for the learning data, and the total thickness image may be used as output data for the learning data. In this case, the processing unit 133 may generate the virtual monochromatic X-ray image from the bone image and the soft tissue image.

[0077] In the above description, it is assumed that a high-energy image and a low-energy image are acquired and then energy subtraction processing is performed to obtain a bone image and a soft tissue image. However, the present disclosure is not limited to such a configuration.

[0078] The acquisition unit 131 may acquire high-energy images, low-energy images, bone images, soft-tissue images, total thickness images, and virtual monochromatic X-ray images from an external device such as a server (not shown) connected to the control device 103. Furthermore, the processing unit 133 may generate bone images and soft-tissue images from the high-energy images and low-energy images acquired from the external device by the acquisition unit 131. Similarly, the processing unit 133 may generate total thickness images and virtual monochromatic X-ray images from the bone images and soft-tissue images acquired from the external device by the acquisition unit 131.

[0079] Alternatively, images for training data may be prepared by obtaining two-dimensional images by forward projection from three-dimensional data acquired by CT. For example, after calculating the ratio of bone to soft tissue from the CT value, two-dimensional bone images and soft tissue images can be obtained separately by forward projection.

[0080] As an example, the acquisition unit 131 can acquire three-dimensional data obtained by CT from an external device. Then, the processing unit 133 acquires three-dimensional data of bone and three-dimensional data of soft tissue based on the ratio of bone to soft tissue calculated from the CT value for the acquired three-dimensional data. Furthermore, the processing unit 133 can perform a known forward projection process on these to acquire two-dimensional bone images and soft tissue images. Then, the processing unit 133 can generate a total thickness image based on the acquired bone images and soft tissue images. The acquisition unit 131 may also acquire bone images and soft tissue images based on the three-dimensional data obtained by CT from an external device. Note that these processes are merely examples, and the control device 103 may acquire bone images and soft tissue images based on the three-dimensional data obtained by CT using any known process.

[0081] In this case, a virtual monochromatic X-ray image can be generated from the bone image and soft tissue image obtained by forward projection using the method shown in Fig. 8, and the pair of the virtual monochromatic X-ray image and the total thickness image can be used as learning data. In this case as well, the processing unit 133 can generate a virtual monochromatic X-ray image from the bone image and soft tissue image obtained by forward projection.

[0082] Next, in step S902, the processing unit 133 performs preprocessing of the training data. Machine learning requires a huge amount of data. If the number of data pieces in the training data prepared in step S901 is insufficient, the processing unit 133 can increase the training data by data expansion. Methods of data expansion that can be used include clipping, which extracts a portion of an image, and rotating, scaling, and flipping an image.

[0083] Alternatively, the processing unit 133 may perform data expansion using an image generation AI. For example, the image generation AI can be trained using bone images and soft tissue images obtained by imaging and ES processing or forward projection of CT three-dimensional data, and new bone images and soft tissue images can be generated using the image generation AI. In this case, virtual monochromatic X-ray images and total thickness images can be generated from the new bone images and soft tissue images obtained by the image generation AI, and pairs of virtual monochromatic X-ray images and total thickness images can be used as training data.

[0084] Next, in step S903, a learning process for the machine learning model is performed. Fig. 10 shows a block diagram of the machine learning according to this embodiment. The ES image generation model (total thickness image generation model) according to the present disclosure is a trained model generated by training (learning) based on a machine learning algorithm.

[0085] Here, we will briefly explain a general trained model. A trained model is a machine learning model that has been trained (learned) in advance using appropriate training data for an arbitrary machine learning algorithm. The training data consists of one or more pairs of input data and output data (ground truth data). The format and combination of input data and output data in the pairs that make up the training data may be suitable for a desired configuration, such as one being an image and the other being a numerical value, one being composed of a group of multiple images and the other being a character string, or both being images. Furthermore, a machine learning algorithm includes a method or system for performing training.

[0086] As shown in Fig. 10, in this embodiment, training data is loaded into a computer, and features such as regularities and patterns inherent in the training data are trained based on a machine learning algorithm to generate a trained model. Here, the machine learning algorithm includes deep learning techniques such as convolutional neural networks (CNNs). In deep learning techniques, different parameter settings for layers and nodes constituting a neural network may affect the degree to which trends trained using the training data can be reproduced in inference data.

[0087] Specifically, parameters in a CNN can include, for example, the filter kernel size, number of filters, stride value, and dilation value set for the convolution layer, as well as the number of nodes output by the fully connected layer. The parameter set and the number of training epochs can be set to values ​​suitable for the usage of the trained model based on the training data. For example, the parameter set and the number of epochs can be set based on the training data to output a more accurate total thickness image.

[0088] Here is an example of one method for determining such parameter sets and the number of epochs. First, 70% of the pairs constituting the learning data are used for training, and the remaining 30% are randomly set as those for evaluation. Next, the training pairs are used to train the machine learning model, and at the end of each training epoch, the evaluation pairs are used to calculate a training evaluation value. The training evaluation value is, for example, the average value of a group of values ​​obtained by evaluating the output when input data constituting each pair is input to the machine learning model being trained, and the output data corresponding to the input data, using a loss function. Finally, the parameter set and the number of epochs that result in the smallest training evaluation value are determined as the parameter set and the number of epochs for the machine learning model.

[0089] In this way, by dividing the pairs that make up the learning data into those for training and those for evaluation and determining the number of epochs, it is possible to prevent the machine learning model from overfitting the training pairs.

[0090] An example of the configuration of a CNN related to an ES image generation model according to this embodiment will be described below with reference to Fig. 11. Fig. 11 shows an example of the configuration of an ES image model. The configuration shown in Fig. 11 is composed of a plurality of layer groups that are responsible for processing and outputting a group of input values. As shown in Fig. 11, the types of layers included in this configuration include a convolution layer, a downsampling layer, an upsampling layer, and a merging layer.

[0091] The convolution layer is a layer that performs convolution processing on a group of input values ​​according to parameters such as the set filter kernel size, the number of filters, the stride value, the dilation value, etc. Note that the number of dimensions of the filter kernel size may also be changed depending on the number of dimensions of the input image.

[0092] The downsampling layer is a layer that performs processing to reduce the number of output value groups to be less than the number of input value groups by thinning out or combining input value groups. Specifically, such processing includes, for example, max pooling processing.

[0093] An upsampling layer is a layer that performs processing to make the number of output values ​​greater than the number of input values ​​by duplicating input values ​​or adding values ​​interpolated from the input values, such as linear interpolation.

[0094] A synthesis layer is a layer that inputs a group of values, such as a group of output values ​​from a layer or a group of pixel values ​​that make up an image, from multiple sources and performs processing to combine them by concatenating or adding them.

[0095] In this configuration, the pixel values ​​constituting the input radiographic image Im1101 are output through a convolution processing block and then combined in a combination layer with the pixel values ​​constituting the input radiographic image Im1101. The combined pixel values ​​are then converted into a total thickness image Im1102 in the final convolution layer.

[0096] It should be noted that different parameter settings for the layers and nodes that make up the neural network may affect the degree to which trends trained from learning data can be reproduced during inference. In other words, in many cases, appropriate parameters differ depending on the implementation form, so they can be changed to preferred values ​​as needed.

[0097] In addition to changing the parameters as described above, changing the configuration of the CNN can sometimes result in better CNN characteristics, such as outputting a more accurate total thickness image, shortening processing time, or shortening the time required to train a machine learning model.

[0098] The CNN used in this embodiment is a U-net type machine learning model having an encoder function consisting of multiple layers including multiple downsampling layers and a decoder function consisting of multiple layers including multiple upsampling layers. That is, the CNN has a U-shaped structure having an encoder function and a decoder function. The U-net type machine learning model is configured (for example, using skip connections) so that position information (spatial information) obscured in multiple layers configured as encoders can be used in layers of the same dimension (layers corresponding to each other) in multiple layers configured as decoders.

[0099] Although not shown in the figure, as an example of a modification to the CNN configuration, for example, a batch normalization layer or an activation layer using a rectifier linear unit may be incorporated after the convolution layer.

[0100] The ES image generation model of this embodiment is configured to be stored in the storage unit 135 and executed by the processing unit 133, but these may also be provided in an external device (not shown) connected to the control device 103.

[0101] Here, a GPU can perform efficient calculations by processing a larger amount of data in parallel. Therefore, when performing learning multiple times using a machine learning algorithm such as deep learning, it is effective to use a GPU for processing. Therefore, in this embodiment, a GPU is used in addition to a CPU for processing by the processing unit 133, which functions as an example of a learning unit. Specifically, when executing a learning program including a learning model, the CPU and GPU work together to perform calculations to perform learning. Note that calculations may be performed only by the CPU or the GPU in the processing of the learning unit. Furthermore, the ES processing according to this embodiment may also be realized using a GPU, as with the learning unit. Note that if the trained model is provided in an external device, the processing unit 133 does not need to function as a learning unit.

[0102] The learning unit may also include an error detection unit and an update unit (not shown). The error detection unit obtains the error between correct data and output data output from the output layer of the neural network in response to input data input to the input layer. The error detection unit may use a loss function to calculate the error between the output data from the neural network and correct data. The update unit updates the connection weighting coefficients between the nodes of the neural network based on the error obtained by the error detection unit so as to reduce the error. This update unit updates the connection weighting coefficients using, for example, an error backpropagation method. The error backpropagation method is a technique for adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the error.

[0103] FIG. 12 shows the configuration of training data according to this embodiment. In this embodiment, both input and output data are image data. As described above, the input data of the training data is a radiological image such as a high-energy image, a low-energy image, or a virtual monochromatic X-ray image, and the output data is a total thickness image. As shown in FIG. 12, the training data is collective data including a plurality of these image pairs (training data 1 to n). Note that the training data according to this embodiment may include images generated by the data expansion performed in step S902.

[0104] By using the trained model trained in this manner, when a radiographic image is input to the trained model, the processing unit 133 can acquire a total thickness image corresponding to the input radiographic image. Furthermore, by constructing a trained model using such training data, many nonlinear arithmetic processes included in the ES processing can be included in inference using a machine learning algorithm such as deep learning.

[0105] (A series of processes according to this embodiment) Next, a series of processes according to this embodiment will be described with reference to Fig. 13 to Fig. 16. Fig. 13 is a flowchart showing an example of a series of processes according to this embodiment.

[0106] First, in step S1301, the acquisition unit 131 acquires a high-energy image acquired by normal imaging using the radiation imaging device 104. The acquisition unit 131 may acquire the high-energy image from an external device connected to the control device 103. The acquired radiation image may be any radiation image that corresponds to the input data of the learning data, and may be, for example, an energy image acquired by imaging using the voltage of a low-energy image or a virtual monochromatic X-ray image as the tube voltage during imaging.

[0107] Next, in step S1302, the processing unit 133 inputs the acquired high-energy image into the trained model and acquires a total thickness image output from the trained model. Here, FIG. 14 is a block diagram of inference processing using the trained model. When target data (input data) is input to the trained model, inference data according to the design of the trained model is output. At this time, the trained model outputs inference data that is likely to correspond to the target data, for example, according to a trend trained using the training data. The trained model according to this embodiment outputs a total thickness image that is likely to correspond to the input high-energy image, according to a trained trend.

[0108] The combination of the target data and the inference data corresponds to the combination of the input data and the output data of the training data. Therefore, the target data may be a radiation image such as a high-energy image, a low-energy image, or an energy image obtained by imaging using the voltage of a virtual monochromatic X-ray image as the tube voltage during imaging in accordance with the input data of the training data.

[0109] Here, with reference to FIGS. 15(a) and 15(b), the relationship between the input data and output data of the training data and the target data and inference data during inference will be described. FIG. 15(a) is a block diagram of a training process using exemplary training data, and FIG. 15(b) is a block diagram of an inference process using exemplary target data and inference data. As described above, in this embodiment, the input data and target data of the training data are high-energy images, and the output data of the training data and the inference data of the trained model are total thickness images. As shown in FIG. 15(a), the trained model trained using high-energy images and total thickness images as training data trains features such as regularity and regularity inherent in the high-energy images and total thickness images. Therefore, as shown in FIG. 15(b), when a high-energy image is input as target data, the trained model can output a total thickness image as inference data that is likely to correspond to the target data according to the training trend.

[0110] Here, the total thickness image will be described with reference to FIG. 16. FIG. 16 is a diagram for explaining the total thickness image, and schematically shows a cross section of a human body. In the human body, bones are surrounded by soft tissue, and the thickness of the soft tissue in the area where the bone is located is reduced by the thickness of the bone. Therefore, when a soft tissue image is subjected to gradation processing and displayed, the contrast of the bone appears in the displayed soft tissue image. On the other hand, when the soft tissue image and the bone image are added, the reduced thickness of the bone in the soft tissue image is filled in again. Therefore, the total thickness image obtained by adding the soft tissue image and the bone image is an image from which the contrast of the bone has been removed, as shown in FIG. 16, in other words, an image with less (simplified) structure of the depicted subject.

[0111] Machine learning models are trained on the features of training data. However, when training is performed using data containing complex features, such as bone images or soft tissue images, the inference data may contain unwanted artifacts due to the complex features. In contrast, training a machine learning model using images with fewer subject structures can reduce artifacts that may occur in the inference data due to the subject's structure (features). Therefore, when high-energy images captured as target data are input to a trained model that has been trained using high-energy images as input data and total thickness images as output data, a total thickness image with reduced artifacts can be output as inference data.

[0112] According to the process of step S1302, a total thickness image, which is an ES image, can be generated from a single radiation image such as a high-energy image or a low-energy image using an ES image generation model based on machine learning. Therefore, when the control device 103 that performs such a process acquires a total thickness image using a trained model, there is no need to capture two images, a high-energy image and a low-energy image.

[0113] In contrast, in general ES processing, bone images and soft tissue images are generated using high-energy images and low-energy images. In order to acquire high-energy images and low-energy images with little time lag, a detector equipped with a sample-and-hold circuit as shown in Fig. 2 and a radiation source capable of switching tube voltage as shown in Fig. 4 are required. This requires large-scale hardware modifications to the radiation imaging device 104 and the radiation generator 101.

[0114] In contrast, the processing in step S1302 can be performed using a single radiographic image, which eliminates the need for the hardware modifications described above to the radiographic imaging system used for inference, thereby reducing the cost of the radiographic imaging system. Furthermore, the processing in step S1302 can be performed using a single radiographic image, which can also reduce the radiation exposure of the subject associated with capturing the radiographic image. Furthermore, since processing can be performed using a single radiographic image, there is an advantage in that motion artifacts due to misalignment that occur when processing is performed using two radiographic images are eliminated.

[0115] In step S1303, the processing unit 133 performs filtering such as a low-pass filter on the total thickness image acquired in step S1302. Note that by applying a low-pass filter (LPF), it is possible to reduce artifacts of high-frequency components caused by the ES image generation model.

[0116] Here, we explain why applying a low-pass filter to the total thickness image can reduce the high-frequency artifacts generated by the ES image generation model. The surface of human bones, i.e., the cortical bone, is covered with high-density bone. On the other hand, the interior of the bone, i.e., the trabecular bone, is made up of spongy bone. Therefore, radiographic images and bone images contain high-frequency components resulting from the structure of the cortical bone and trabecular bone.

[0117] However, the spaces between the trabeculae are filled with bone marrow, which is classified as soft tissue. Therefore, in the total thickness image, which is the sum of the bone image and soft tissue, the structures of the bone trabeculae and bone marrow cancel each other out, reducing the high-frequency components of the image. Therefore, in the total thickness image, signal components are less likely to be lost even if the strength of the low-pass filter applied is increased. Increasing the strength of the low-pass filter processing can further reduce high-frequency component artifacts caused by the ES image generation model. Therefore, when applying a low-pass filter to the total thickness image, the strength of the low-pass filter processing can be increased, further reducing high-frequency component artifacts caused by the ES image generation model.

[0118] In step S1304, the display control unit 134 causes the display unit 120 to display the total thickness image. The processing unit 133 may further apply a time-direction filter such as a recursive filter or a space-direction filter such as a Gaussian filter to the total thickness image to reduce noise and then cause the display unit 120 to display the total thickness image that has been subjected to gradation processing. The display control unit 134 may also cause the display unit 120 to display an analysis result obtained by performing any analysis processing on the total thickness image by the processing unit 133. The control device 103 may also transmit the total thickness image and the analysis result of the total thickness image to an external device.

[0119] When images clipped by data augmentation are used as training data for a trained model, the processing unit 133 clips the radiographic image to be input to the ES image generation model to make the image size corresponding to the training data. After that, the processing unit 133 can connect the images together to make the total thickness image output from the ES image generation model by inputting the clipped radiographic image the same size as the image before clipping.

[0120] However, in this case, tile-like artifacts may occur due to differences in levels at the joints between the images. Therefore, the processing unit 133 may provide overlapping portions (overlapping portions) between the images when clipping the radiographic images, and join the overlapping portions of the total thickness image output from the ES image generation model by weighting and adding them together. The weighting may be performed, for example, using a method of weighting based on the coordinates of the pixels in the overlapping portions of the clipped low-resolution radiographic images.

[0121] As described above, the radiation imaging system according to this embodiment includes the radiation imaging device 104 that detects radiation irradiated from the radiation generation device 101, and the control device 103 that is communicatively connected to the radiation imaging device 104. The control device 103 can function as an example of an image processing device that performs image processing. The control device 103 includes an acquisition unit 131 and a processing unit 133. The acquisition unit 131 functions as an example of an acquisition unit that acquires a radiation image. The processing unit 133 functions as an example of a processing unit that inputs the acquired radiation image into a trained model obtained using training data including the radiation image and a total thickness image that indicates the sum of thicknesses of different materials in the subject, and thereby acquires a total thickness image output from the trained model.

[0122] With this configuration, the control device 103 can reduce artifacts caused by the structure of the subject when generating a total thickness image, which is an ES image, from a radiological image using the trained model. Therefore, the control device 103 according to this embodiment can appropriately support diagnosis and appropriately perform image analysis, etc., based on the total thickness image, which is an ES image acquired using the trained model.

[0123] Furthermore, in the processing using the trained model according to this embodiment, since processing can be performed using a single radiographic image, the above-described hardware modifications to the radiographic imaging system used for inference are not required, and the cost of the radiographic imaging system can be reduced. Furthermore, since processing can be performed using a single radiographic image, the amount of radiation exposure to the subject associated with capturing the radiographic image can also be reduced.

[0124] In this embodiment, the total thickness image included in the training data can be generated by adding together multiple material decomposition images that represent the thicknesses of different materials in the subject. Here, the multiple material decomposition images can include, for example, an image that represents the thickness of bone and an image that represents the thickness of soft tissue. Furthermore, the multiple material decomposition images can be generated by energy subtraction processing using radiation images of different energies.

[0125] Note that multiple material decomposition images can also be generated using forward projection based on 3D CT data. In this case, it is possible to easily prepare training data for a trained model. As a result, it is expected that artifacts will be reduced due to improved training accuracy, and that the number of regions that can be covered by the trained model will increase.

[0126] Furthermore, the processing unit 133 can apply a low-pass filter to the total thickness image acquired using the trained model. In this case, it is possible to reduce high-frequency component artifacts in the total thickness image output by the trained model. In particular, when applying a low-pass filter to the total thickness image, signal components are less likely to be lost even if the strength of the applied low-pass filter is increased. Therefore, it is possible to increase the strength of the low-pass filter processing, and it is possible to more accurately reduce high-frequency component artifacts caused by the trained model.

[0127] Furthermore, the control device 103 can further include a display control unit 134 that causes the display unit 120 to display the total thickness image acquired by the processing unit 133 .

[0128] (Second embodiment) In the first embodiment, a configuration for acquiring a total thickness image using a trained model has been described. In contrast, in the second embodiment of the present disclosure, a configuration for acquiring an ES image, such as a material decomposition image such as a bone image or a soft tissue image, based on a total thickness image output from a trained model using an approximate formula for ES processing will be described.

[0129] The radiation imaging system according to this embodiment will be described below with reference to Figs. 17 and 18. The configuration of the radiation imaging system according to this embodiment is similar to that of the radiation imaging system according to the first embodiment, and therefore the same reference numerals are used and a description thereof will be omitted. The learning process and learning data according to this embodiment are similar to those according to the first embodiment, and therefore a description thereof will be omitted. The radiation imaging system according to this embodiment will be described below, focusing on the differences from the radiation imaging system according to the first embodiment.

[0130] Fig. 17 is a flowchart showing an example of a series of processes according to this embodiment. Fig. 18 is a block diagram showing an example of a series of processes according to this embodiment. Note that steps S1701 and S1702 are similar to the processes of steps S1301 and S1302 according to the first embodiment, and therefore description thereof will be omitted. In step S1702, when the processing unit 133 acquires a total thickness image, the process proceeds to step S1703.

[0131] In step S1703, the processing unit 133 performs filtering such as a low-pass filter on the total thickness image acquired in step S1702. As described above, by applying a low-pass filter (LPF) to the total thickness image, it is possible to reduce artifacts of high-frequency components caused by the ES image generation model.

[0132] In step S1704, the processing unit 133 performs logarithmic conversion processing on the radiation image acquired in step S1701. The logarithmic conversion processing may be performed by any known method.

[0133] In step S1705, the processing unit 133 acquires an ES image such as a material decomposition image using an approximation formula for ES processing based on the total thickness image after filtering obtained in step S1703 and the logarithm of the high-energy image obtained in step S1704. More specifically, the processing unit 133 acquires an ES image such as a material decomposition image such as a bone image or a soft tissue image, or an ES image such as a radiation image of any energy, by weighted addition using the total thickness image after filtering and the logarithm of the high-energy image.

[0134] The process of generating an ES image using an approximate formula for ES processing will be explained below. The pixel value X of a radiographic image, the thickness B of the bone, and the thickness S of the soft tissue are related as shown in formula (4). For simplicity, the spectrum N(E) of radiation when a radiographic image is acquired at an arbitrary energy is expressed as the average energy E X Approximating this to a virtual monochromatic X-ray with a peak at , it is expressed by the following equation (11).

number

[0135] From equations (4) and (11), the pixel value X of a radiographic image acquired at any energy is calculated by the average energy E X The linear attenuation coefficient μ of bone in XB and the average energy E X The linear attenuation coefficient μ of soft tissue in XS Using this, it is expressed by the following equation (12).

number

[0136] Furthermore, the total thickness T is the sum of the bone thickness B and the soft tissue thickness S. Therefore, the bone thickness B and the soft tissue thickness S are expressed using the total thickness T by the following equation (13).

number

[0137] For simplicity, the spectrum N(E) of radiation when a radiological image is acquired at any energy is expressed as the average energy E Y In this case, the pixel value Y of a radiation image of any energy can be approximated by E Y The linear attenuation coefficient μ of bone in YB And, E Y The linear attenuation coefficient μ of soft tissue in YS Using this, it is expressed by the following equation (14).

number

[0138] Equation (13) is an equation in which the logarithm of the radiographic image and the total thickness image are multiplied by a coefficient determined by the linear attenuation coefficient and then added together. The same applies to equation (14). Therefore, as shown in Fig. 18, the processing unit 133 can obtain a material decomposition image such as a bone image or soft tissue image, or an ES image such as a radiographic image of any energy, by weighted addition of the logarithm of the high-energy image and the total thickness image, which are input data.

[0139] Note that both equations (13) and (14) are weighted additions of the logarithm of the radiation image and the total thickness image generated by the ES image generation model. Furthermore, the ES image acquired by these weighted additions is a different type of ES image from the total thickness image generated by the ES image generation model. Therefore, in step S1705, the processing unit 133 can generate a different type of ES image from the total thickness image generated by the ES image generation model by using the weighted addition of the logarithm of the high-energy image and the total thickness image generated by the ES image generation model.

[0140] In the above process, for simplicity, the radiation spectrum N(E) when a radiological image is acquired at any energy is expressed as the average energy E X The ES image was approximated using virtual monochromatic X-rays with a peak at 1000 MHz. However, the inventors of the present application have found that calculations using approximation using virtual monochromatic X-rays may not result in accurate ES images. Therefore, a method for generating an ES image of a different type from the ES image generated by the ES image generation model without approximation using virtual monochromatic X-rays will be described.

[0141] The pixel value X of a radiographic image is calculated by the linear attenuation coefficient μ of bone at energy E. B (E) and the linear attenuation coefficient μ of soft tissue at energy E S Using (E), this is expressed as the following equation (15).

number

[0142] By solving the nonlinear equation of Equation (15), the thickness S of the soft tissue can be calculated from the thickness B of the bone and the pixel value X of the radiographic image. The Newton-Raphson method or an iterative method such as the least squares method or bisection method can be used to solve Equation (15). When the Newton-Raphson method is used, for example, the value obtained by the approximation of Equation (15) can be used as the initial value. Alternatively, a table can be generated by calculating the thickness S of the soft tissue for various combinations of pixel value X of the radiographic image and bone thickness B, and the thickness S of the soft tissue can be calculated quickly by referring to the table. The thickness S of the bone can also be calculated from the thickness S of the soft tissue and the pixel value X of the radiographic image using a similar process.

[0143] Here, by using equation (15), it is also possible to acquire a bone image or a soft tissue image using the total thickness T and the pixel value X of the radiographic image. For example, by substituting (TS) for B in equation (15) or (TB) for S, it is possible to acquire a bone image or a soft tissue image using the total thickness image and the radiographic image. Therefore, the processing unit 133 can generate an ES image of a different type from the total thickness image generated by the ES image generation model by performing a calculation using a nonlinear equation between the high-energy image and the total thickness image generated by the ES image generation model. Furthermore, by performing the same processing as above, it is also possible to acquire a bone image or a soft tissue image using the total thickness image and the radiographic image using a table. Note that when performing the calculation according to equation (15), step S1704 may be omitted.

[0144] In step S1706, the display control unit 134 causes the display unit 120 to display the ES image acquired in step S1705. The display control unit 134 may also cause the display unit 120 to display the analysis results of any analysis process performed on the ES image by the processing unit 133. The control device 103 may also transmit the ES image and the analysis results of the ES image to an external device. As in step S1303 according to the first embodiment, the display control unit 134 can also cause the display control unit 134 to display the total thickness image, the analysis results of the total thickness image, and the like. As in the first embodiment, the control device 103 can also transmit the total thickness image, the analysis results of the total thickness image, and the like to an external device.

[0145] In this embodiment, a high-energy image is used as input data, and a total thickness image is output from the ES image generation model. However, as described above, the input data and the inferred data output from the ES image generation model are not limited to this. The input data and the inferred data may be data corresponding to the training data. For example, the input data may be a low-energy image, or a radiological image such as an energy image obtained by imaging using the voltage of the virtual monochromatic X-ray image used as input data for the training data as the tube voltage during imaging.

[0146] In this embodiment, the logarithmic conversion process in step S1704 is performed after the process in step S1703. However, the logarithmic conversion process in step S1704 may be performed between steps S1701 and S1705. The logarithmic conversion process in step S1704 may also be performed in parallel with the processes in steps S1701 to S1703.

[0147] As described above, the processing unit 133 according to this embodiment can generate a type of image different from the total thickness image by using a weighted addition of the logarithm of the radiation image acquired by the acquisition unit 131 and the total thickness image acquired using the trained model.

[0148] With this configuration, the control device 103 according to this embodiment can generate ES images other than the total thickness image using a single radiographic image. This eliminates the need for significant hardware modifications to the radiographic imaging system used for inference using a trained model in order to acquire ES images other than the total thickness image, thereby reducing the cost of the radiographic imaging system. Furthermore, since processing can be performed using a single radiographic image, it is also possible to reduce the amount of radiation exposure of the subject associated with capturing the radiographic image.

[0149] The weighted addition can be performed by multiplying the logarithm of the radiographic image acquired by the acquisition unit 131 and the total thickness image acquired using the trained model by a coefficient determined by the linear attenuation coefficient of the material of the subject, and adding up the results of these multiplications. Specifically, the processing unit 133 can perform weighted addition of the logarithm of the radiographic image acquired by the acquisition unit 131 and the total thickness image acquired using the trained model using Equation (13) or Equation (14) as an approximation equation for the ES processing.

[0150] The type of image different from the total thickness image may be a material decomposition image showing the thickness of the material of the subject, a virtual monochromatic X-ray image, or a radiation image of any energy. The material decomposition image may be at least one of an image showing the thickness of bone and an image showing the thickness of soft tissue.

[0151] Furthermore, the processing unit 133 can apply a low-pass filter to the total thickness image acquired using the trained model, and perform weighted addition using the total thickness image to which the low-pass filter has been applied. In this case, high-frequency component artifacts in the total thickness image output by the trained model can be reduced. In particular, when a low-pass filter is applied to the total thickness image, signal components are less likely to be lost even if the strength of the applied low-pass filter is increased. Therefore, the strength of the low-pass filter processing can be increased, and high-frequency component artifacts caused by the trained model can be more accurately reduced.

[0152] The control device 103 may further include a display control unit 134 that causes the display unit 120 to display at least one of the acquired total thickness image and a different type of image.

[0153] Furthermore, the processing unit 133 can also generate a type of image different from the total thickness image by performing a calculation using a nonlinear equation between the radiographic image acquired by the acquisition unit 131 and the total thickness image acquired using the trained model. In this case, the same effect as above can be achieved.

[0154] Note that the post-processing of the processing using the trained model according to the above embodiment is not limited to including only the above-described processing. For example, it may include tone conversion processing, any image quality improvement processing, any resolution improvement processing, any image conversion processing, etc. Furthermore, in the post-processing according to the above embodiment, filtering processing using a low-pass filter or the like is performed, but filtering may not be performed. Furthermore, the filter used in filtering processing may be a band-pass filter that passes signals in a desired frequency band.

[0155] In addition, in the trained model described in the above embodiment, it is considered that the magnitude of the brightness values ​​of the input data image, the order and gradient of the bright and dark areas, position, distribution, continuity, etc. are extracted as part of the features and used in the inference process.

[0156] (Other Examples) The present disclosure can also be realized by providing a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions. A computer may have one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0157] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0158] The above disclosure includes the following configurations, methods, and programs. (Configuration 1) an acquisition unit for acquiring a radiation image; a processing unit that inputs the acquired radiographic image into a trained model obtained using training data including the radiographic image and a total thickness image indicating the sum of thicknesses of different materials in the subject, and thereby acquires a total thickness image output from the trained model; An image processing device comprising: (Configuration 2) 2. The image processing device according to claim 1, wherein the total thickness image included in the training data is generated by adding together a plurality of material decomposition images that indicate thicknesses of different materials of the subject. (Configuration 3) 3. The image processing device according to configuration 2, wherein the plurality of material decomposition images include an image showing bone thickness and an image showing soft tissue thickness. (Configuration 4) 4. The image processing device according to configuration 2 or 3, wherein the plurality of material decomposition images are generated by energy subtraction processing using radiation images of different energies. (Configuration 5) 4. The image processing device according to configuration 2 or 3, wherein the plurality of material decomposition images are generated using forward projection based on three-dimensional CT data. (Configuration 6) 6. The image processing device according to any one of configurations 1 to 5, wherein the processing unit applies a low-pass filter to the acquired total thickness image. (Configuration 7) 7. The image processing device according to any one of configurations 1 to 6, further comprising a display control unit that causes a display unit to display the total thickness image acquired by the processing unit. (Configuration 8) 8. The image processing device according to any one of configurations 1 to 7, wherein the processing unit generates an image of a type different from the total thickness image using the acquired radiation image and the acquired total thickness image. (Configuration 9) The processing unit a weighted addition of the logarithm of the acquired radiographic image and the acquired total thickness image; or 9. The image processing device according to configuration 8, wherein the different types of images are generated by performing a calculation using a nonlinear equation between the acquired radiation image and the acquired total thickness image. (Configuration 10) The image processing device according to configuration 9, wherein the weighted addition is performed by multiplying each of the logarithm of the acquired radiographic image and the acquired total thickness image by a coefficient determined by a linear attenuation coefficient related to the material of the subject, and adding up the results of the multiplications. (Configuration 11) 11. The image processing device according to any one of configurations 8 to 10, wherein the different types of images are material decomposition images showing the thickness of a material of the subject, virtual monochromatic X-ray images, or radiation images of any energy. (Configuration 12) 12. The image processing device according to claim 11, wherein the material decomposition image is at least one of an image showing a bone thickness and an image showing a soft tissue thickness. (Configuration 13) The processing unit applying a low-pass filter to the acquired total thickness image; 13. The image processing device according to any one of configurations 8 to 12, wherein the different types of images are generated using the total thickness image to which the low-pass filter has been applied. (Configuration 14) 14. The image processing device according to claim 8, further comprising a display control unit that causes at least one of the acquired total thickness image and the different types of images to be displayed on a display unit. (Configuration 15) a radiation imaging device for detecting radiation; the image processing device according to any one of configurations 1 to 14, which is communicably connected to the radiation imaging device; A radiation imaging system comprising: (Method 1) obtaining a radiological image; inputting the acquired radiographic image into a trained model obtained using training data including the radiographic image and a total thickness image indicating the sum of thicknesses of different materials of the subject, thereby obtaining a total thickness image output from the trained model; An image processing method comprising: (Program 1) A program that, when executed by a computer, causes the computer to perform the image processing method described in Method 1.

[0159] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Inventions modified within the scope of the present disclosure and inventions equivalent to the present disclosure are also included in the present disclosure. Furthermore, the above-described embodiments can be combined as appropriate within the scope of the present disclosure. [Explanation of symbols]

[0160] 103: control device (image processing device), 131: acquisition unit, 133: processing unit

Claims

1. an acquisition unit for acquiring a radiation image; a processing unit that inputs the acquired radiographic image into a trained model obtained using training data including the radiographic image and a total thickness image indicating the sum of thicknesses of different materials in the subject, and thereby acquires a total thickness image output from the trained model; An image processing device comprising:

2. The image processing device according to claim 1 , wherein the total thickness image included in the training data is generated by adding together a plurality of material decomposition images that indicate thicknesses of different materials of the subject.

3. The image processing apparatus according to claim 2 , wherein the plurality of material decomposition images include an image showing a bone thickness and an image showing a soft tissue thickness.

4. The image processing apparatus according to claim 2 , wherein the plurality of material decomposition images are generated by energy subtraction processing using radiation images of different energies.

5. The image processing apparatus according to claim 2 , wherein the plurality of material decomposition images are generated using forward projection based on three-dimensional CT data.

6. The image processing device according to claim 1 , wherein the processing unit applies a low-pass filter to the acquired total thickness image.

7. The image processing device according to claim 1 , further comprising a display control unit that causes a display unit to display the total thickness image acquired by the processing unit.

8. The image processing device according to claim 1 , wherein the processing unit generates an image of a type different from the total thickness image by using the acquired radiographic image and the acquired total thickness image.

9. The processing unit a weighted addition of the logarithm of the acquired radiographic image and the acquired total thickness image; or The image processing apparatus according to claim 8 , wherein the different types of images are generated by performing a calculation using a nonlinear equation between the acquired radiation image and the acquired total thickness image.

10. 10. The image processing device according to claim 9, wherein the weighted addition is performed by multiplying each of the logarithm of the acquired radiographic image and the acquired total thickness image by a coefficient determined by a linear attenuation coefficient related to a material of the subject, and then adding up the results of these multiplications.

11. The image processing device according to claim 8 , wherein the different types of images are a material decomposition image showing the thickness of a material of the object, a virtual monochromatic X-ray image, or a radiation image of any energy.

12. The image processing apparatus according to claim 11 , wherein the material decomposition image is at least one of an image showing a bone thickness and an image showing a soft tissue thickness.

13. The processing unit applying a low-pass filter to the acquired total thickness image; The image processing apparatus according to claim 8 , wherein the different types of images are generated using the total thickness image to which the low-pass filter has been applied.

14. The image processing device according to claim 8 , further comprising a display control unit that causes at least one of the acquired total thickness image and the different types of images to be displayed on a display unit.

15. a radiation imaging device for detecting radiation; The image processing device according to claim 1 , which is communicably connected to the radiation imaging device; A radiation imaging system comprising:

16. obtaining a radiological image; inputting the acquired radiographic image into a trained model obtained using training data including the radiographic image and a total thickness image indicating the sum of thicknesses of different materials of the subject, thereby obtaining a total thickness image output from the trained model; An image processing method comprising:

17. A program that, when executed by a computer, causes the computer to execute the image processing method according to claim 16.

Images