Image processing apparatus, image processing method, and computer-readable medium

Abstract

An image processing apparatus is provided, comprising: an acquisition unit configured to acquire a first radiographic image of an object to be inspected; and a generation unit configured to generate a second radiographic image with reduced noise compared to the first radiographic image by inputting the first radiographic image acquired by the acquisition unit into a learned model, wherein the learned model is obtained by training using training data, the training data including radiographic images obtained by adding noise attenuated by adding high-frequency components.

Description

Technical Field

[0001] This disclosure relates to image processing apparatus, image processing method, training apparatus, training method, and computer-readable medium. Background Technology

[0002] In recent years, radiation detectors equipped with detection units for detecting radiation such as X-rays have been widely used in fields such as industry and medical treatment. In particular, digital radiography (DR) devices that use semiconductor sensors to acquire radiation images have become widely used.

[0003] In such digital radiography apparatuses, various image processing techniques are typically performed to improve the image quality of the captured images, and one such image processing technique is noise reduction processing to improve the graininess of the captured images and thereby enhance the visibility of the area to be diagnosed.

[0004] Various methods have been proposed as a means of noise reduction. For example, PTL1 discloses a rule-based noise reduction processing technique that creates rules based on the noise characteristics of a digital radiography device and switches the characteristics of a smoothing filter.

[0005] [List of Citations]

[0006] [Patent Literature]

[0007] PTL 1: Japanese Patent No. 4679710

[0008] [Non-patent literature]

[0009] NPL 1: "Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections", Xiao-Jiao Mao, Chunhua Shen, and Yu-Bin Yang, ArXiv:1606.08921, 2016 Summary of the Invention

[0010] [Technical Issues]

[0011] However, the aforementioned prior art sometimes presents the following problems. In the rule-based noise reduction processing proposed in PTL1, it is difficult to create appropriate rules for every case of various subject structures included in the captured image, and there are situations where the noise reduction performance cannot be obtained due to subject dependence.

[0012] In digital radiography, when radiation passing through a subject is detected as an electrical signal, a common configuration involves converting the radiation into visible light using a scintillator such as CsI or GOS (Gd₂O₂S) and then detecting it using a semiconductor sensor. Because the converted visible light is diffused before reaching the semiconductor sensor after passing through the scintillator, noise attributable to fluctuations in radiation quanta (hereinafter referred to as "quantum noise") is smoothed, resulting in high-frequency attenuation. On the other hand, noise generated from detectors and circuitry (hereinafter referred to as "system noise") is unaffected by the scintillator. Since the noise in a digital radiography device is primarily composed of the sum of system noise and quantum noise, this noise exhibits characteristics where the ratio of these two types of noise varies depending on the input signal, and the frequency characteristics differ depending on the signal strength.

[0013] NPL 1 proposes a technique to achieve noise reduction processing suitable for a larger number of subjects by increasing the number of images used for training. However, when considering the application of the discussed technique to digital radiography, it is difficult to achieve favorable noise reduction for training neural networks in some cases because the noisy characteristics of digital radiography, as mentioned above, are not taken into account.

[0014] In view of the above problems, the purpose of this disclosure is to provide an image processing apparatus, an image processing method, a training apparatus, a training method, and a computer-readable medium storing a program that can perform advantageous noise reduction processing in a digital radiography apparatus.

[0015] [Solution to the problem]

[0016] An image processing apparatus according to an embodiment of the present disclosure includes: an acquisition unit configured to acquire a first radiation image of an object to be inspected; and a generation unit configured to generate a second radiation image with reduced noise compared to the first radiation image by inputting the first radiation image acquired by the acquisition unit into a learned model, wherein the learned model is obtained by training using training data, the training data including radiation images obtained by adding noise attenuated by adding high-frequency components. Attached Figure Description

[0017] Figure 1A This is a schematic diagram illustrating the configuration of a radiation imaging system according to Embodiment 1.

[0018] Figure 1B This is a schematic diagram illustrating the configuration of a radiation detector according to Embodiment 1.

[0019] Figure 2AThis is a diagram illustrating a schematic configuration of the control unit according to Embodiment 1.

[0020] Figure 2B This is a schematic diagram illustrating the configuration of the noise reduction processing unit according to Embodiment 1.

[0021] Figure 3A This is a diagram illustrating an example of a schematic configuration of the learned model according to Embodiment 1.

[0022] Figure 3B This is a diagram illustrating an example of a schematic configuration of the learned model according to Embodiment 1.

[0023] Figure 3C This is a diagram used to describe an operational example of the training process according to Example 1.

[0024] Figure 3D This is a diagram used to describe an operational example of the training process according to Example 1.

[0025] Figure 3E This is a diagram used to describe an operational example of the training process according to Example 1.

[0026] Figure 4 This is a flowchart of the training process according to Example 1.

[0027] Figure 5A This is a diagram used to describe the region division process according to Example 1.

[0028] Figure 5B This is a diagram used to describe the region division process according to Example 1.

[0029] Figure 5C This is a diagram used to describe the region division process according to Example 1.

[0030] Figure 6 It is a graph used to describe the spatial frequency characteristics of noise that vary depending on the radiation dose.

[0031] Figure 7A This is a diagram used to describe noise decorrelation processing.

[0032] Figure 7B This is a diagram used to describe noise decorrelation processing.

[0033] Figure 8 This is a schematic diagram illustrating an example of adjusting the noise reduction processing intensity in the inference process according to Example 1.

[0034] Figure 9 This is a flowchart of the inference process based on Example 1.

[0035] Figure 10This is an example diagram illustrating images before and after image processing according to Embodiment 1.

[0036] Figure 11 This is a flowchart of the training process according to Example 2.

[0037] Figure 12 This is a diagram illustrating an example of the relationship between spatial frequency and MTF in a radiation detector. Detailed Implementation

[0038] In the following, exemplary examples for carrying out the invention will be described in detail with reference to the accompanying drawings. However, the dimensions, materials, shapes, and relative positions of the components described in the following embodiments are not deterministic and may vary depending on the configuration of the apparatus to which the invention is applied or various conditions. Furthermore, identical or functionally similar elements are represented by the same reference numerals in different drawings.

[0039] Note that in the embodiments of this disclosure described below, the term "radiation" includes alpha rays, beta rays, and gamma rays as beams generated by particles (including photons) emitted by radioactive decay, and also includes beams with similar or higher energy levels than these beams, such as X-rays, particle rays, and cosmic rays. Additionally, in the following text, dark current noise, electrical noise, etc., which are independent of the amplitude of the signal in a radiation image, are referred to as "system noise".

[0040] Furthermore, in the following text, the term "machine learning model" refers to a learning model learned according to a machine learning algorithm. Specific examples of algorithms used for machine learning include nearest neighbor, Naive Bayes, decision trees, and support vector machines. Additionally, neural networks or deep structured learning (deep learning) can be used. The algorithms available among those mentioned above can be appropriately used and applied to the embodiments and modifications described below. Furthermore, the term "training data" refers to the dataset used for training the machine learning model and consists of pairs of input data fed into the machine learning model and true values ​​representing the correct answers as the output of the machine learning model.

[0041] It is important to note that the term "learned model" refers to a model that has been pre-trained using appropriate training data on a machine learning model based on any machine learning algorithm, such as deep learning. However, although a learned model is a model obtained in advance using appropriate training data, it is not a model that does not perform additional learning; rather, it is a model that can also perform incremental learning. Incremental learning can also be performed after the device is installed at its intended use.

[0042] (Example 1)

[0043] (Configuration of the radiation system)

[0044] In the following text, see references Figures 1A to 10 A radiographic imaging system according to Embodiment 1 of this disclosure is described. Figure 1A This is a schematic diagram illustrating the configuration of the radiographic imaging system 1 according to this embodiment. Note that although the object O to be examined is described as a human body in the following description, the object O to be examined by the radiographic imaging system according to this disclosure is not limited to a human body, and may be another animal or plant, or an object to be examined non-destructively, etc.

[0045] The radiation imaging system 1 according to this embodiment provides a radiation detector 10, a control unit 20, a radiation generator 30, an input unit 40, and a display unit 50. Note that the radiation imaging system 1 may include an external storage device 70, such as a server, which is connected to the control unit 20 via a network 60, such as the Internet or an intranet.

[0046] The radiation generator 30 includes a radiation source, such as an X-ray tube, and is capable of emitting radiation. The radiation detector 10 can detect the radiation emitted by the radiation generator 30 and can generate a radiation image corresponding to the detected radiation. Therefore, by detecting radiation emitted by the radiation generator 30 and that has passed through the object O to be inspected, the radiation detector 10 can generate a radiation image of the object O to be inspected.

[0047] here, Figure 1B This is a schematic diagram illustrating the configuration of a radiation detector 10 according to this embodiment. A scintillator 11 and an imaging sensor 12 are provided in the radiation detector 10. The scintillator 11 converts radiation incident on the radiation detector 10 into light having a wavelength that can be detected by the imaging sensor 12. The scintillator 11 may include, for example, CsI or GOS (Gd₂O₂S). The imaging sensor 12 includes, for example, a photoelectric conversion element made of a-Si or crystalline Si, and can detect the light corresponding to the radiation converted by the scintillator 11, and output a signal corresponding to the detected light. The radiation detector 10 can generate a radiation image by performing A / D conversion, etc., on the signal output by the imaging sensor 12. Note that, although from... Figure 1B The schematic configuration is omitted, but the radiation detector 10 may include a computing unit and an A / D conversion unit.

[0048] The control unit 20 is connected to the radiation detector 10, the radiation generator 30, the input unit 40, and the display unit 50. The control unit 20 can acquire the radiation image output from the radiation detector 10 and perform image processing on the radiation image, and can control the driving of the radiation detector 10 and the radiation generator 30. Therefore, the control unit 20 can be used as an example of an image processing device. Additionally, the control unit 20 can be connected to an external storage device 70 via any network 60, such as the Internet or an intranet, and can acquire radiation images from the external storage device 70. Furthermore, the control unit 20 can be connected to another radiation detector or radiation generator via the network 60. Note that the control unit 20 can be connected to the external storage device 70, etc., either wired or wirelessly.

[0049] The input unit 40 includes input devices such as a mouse, keyboard, trackball, or touchpad, and can be operated by an operator to input commands to the control unit 20. The display unit 50 includes, for example, any monitor, and can display information or images output from the control unit 20, as well as information input from the input unit 40.

[0050] Note that although in this embodiment the control unit 20, input unit 40, display unit 50, etc., are each constituted as separate devices, two or more of these units can be integrated into a single device. For example, the input unit 40 and the display unit 50 can be constituted as a touchpad display. Furthermore, although in this embodiment the image processing device is constituted by the control unit 20, it is sufficient for the image processing device to acquire radiation images and perform image processing on the radiation images, and the image processing device does not need to control the driving of the radiation detector 10 and the radiation generator 30.

[0051] The control unit 20 can be connected to the radiation detector 10 and the radiation generator 30, etc., via wired or wireless connection. Furthermore, the external storage device 70 can be an image system such as a PACS system in a hospital, or it can be a server located outside the hospital.

[0052] (Control unit configuration)

[0053] Next, refer to Figure 2A and Figure 2B A more specific configuration of the control unit 20 is described. Figure 2A This is a diagram illustrating a schematic configuration of the control unit 20 according to this embodiment. Figure 2B This is a schematic diagram illustrating the configuration of the noise reduction processing unit 26 included in the control unit 20. Figure 2A As shown, the control unit 20 provides an acquisition unit 21, an image processing unit 22, a display control unit 23, a drive control unit 24, and a storage device 25.

[0054] The acquisition unit 21 can acquire radiation images output by the radiation detector 10, as well as various information input by the input unit 40. Additionally, the acquisition unit 21 can acquire radiation images and patient information from an external storage device 70, etc.

[0055] The image processing unit 22 provides a noise reduction processing unit 26 and a diagnostic image processing unit 27. The image processing unit 22 can perform image processing according to this disclosure on the radiographic image obtained by the acquisition unit 21. For example... Figure 2B As shown, the noise reduction processing unit 26 provides a training processing unit 261 and an inference processing unit 262. The inference processing unit 262 provides a partitioning unit 263, a decorrelation unit 264, a DC removal unit 265, and a computation processing unit 266. In addition to the configuration of the inference processing unit 262, the training processing unit 261 also provides an artificial noise addition unit 267 and a parameter update unit 268. According to this configuration, the noise reduction processing unit 26 can perform training of a machine learning model for performing noise reduction processing, and can use the machine learning model to apply beneficial noise reduction processing to radiographic images.

[0056] Additionally, the diagnostic image processing unit 27 can perform diagnostic image processing to transform an image that has already undergone noise reduction by the noise reduction processing unit 26 into an image suitable for diagnosis. Diagnostic image processing includes, for example, grayscale processing, enhancement processing, and grid stripe reduction processing.

[0057] Next, the configuration of the inference processing unit 262 will be described. The inference processing unit 262 can perform inference processing that applies the learned model to an input arbitrary radiation image and output the obtained result.

[0058] The segmentation unit 263 can divide a radiographic image into multiple regions. The segmentation unit 263 can also combine the images of multiple regions to generate a single image.

[0059] The decorrelation unit 264 performs the following process: for the amplitude of quantum noise caused by fluctuations in radiation included in the radiation image, this process eliminates the correlation (decorrelation) with the amplitude of the signal corresponding to the detected radiation. Here, it is known that the variance of the quantum noise (hereinafter referred to as the "noise amplitude") is proportional to the radiation dose. Therefore, the amplitude of the quantum noise is proportional to the signal quantity (hereinafter referred to as the "pixel value") of each pixel in the radiation image corresponding to the detected radiation dose. By performing decorrelation processing on the radiation image, the decorrelation unit 264 according to this embodiment can make the amplitude of the quantum noise approximately constant, regardless of the magnitude of the pixel value of each pixel in the radiation image. The specific process of decorrelation processing will be described later.

[0060] Furthermore, by performing an inverse transform on the decorrelation-processed radiation image, the decorrelation unit 264 can restore the correlation between the magnitude of quantum noise and the pixel value to the original correlation. Note that separate components can be provided to perform the noise decorrelation process and the inverse decorrelation process, respectively, and in this case, a functional block including the corresponding component can be provided as the decorrelation unit 264.

[0061] The DC removal unit 265 performs a process of subtracting the DC component of the image and equalizing the average signal level of the image. Specifically, the DC removal unit 265 removes the DC component of the radiation image by subtracting the average pixel value of the radiation image from each pixel value of the image, making the average pixel value of the radiation image zero (0). Furthermore, by adding the subtracted average pixel value to each pixel value of the image after the DC removal process, the DC removal unit 265 can restore the average signal level of the radiation image to its state before the transformation. Note that separate components can be provided to perform the DC removal process and the DC addition process, and in this case, a function block including the corresponding component can be provided as the DC removal unit 265. Additionally, the removal of the DC component can be performed on each image obtained when the radiation image is segmented by the segmentation unit 263, or it can be performed on the entire radiation image before segmentation by the segmentation unit 263.

[0062] The image, processed by the partitioning unit 263, the decorrelation unit 264, and the DC removal unit 265, is input to the arithmetic processing unit 266. The arithmetic processing unit 266 can perform processing on the image using a learned model, which will be described later, thereby generating an image with reduced noise already included in the input image. Note that in this embodiment, the noise reduced includes system noise such as dark current or electrical noise generated by the imaging sensor 12, as well as quantum noise caused by fluctuations in radiation.

[0063] Next, the configuration of the training processing unit 261 is described. The training processing unit 261 performs the training processing applied when training a machine learning model. In addition to the configuration of the inference processing unit 262, the training processing unit 261 also includes an artificial noise addition unit 267 and a parameter update unit 268.

[0064] Training data is required when performing training processing. The processes of partitioning unit 263, decorrelation unit 264, and DC removal unit 265 are applied to both the input data and the ground truth values ​​of the training data.

[0065] Here, the artificial noise addition unit 267 performs the process of adding artificial noise, created by simulating the characteristics of a radiation image, to the training data. The addition of artificial noise is performed only on the input data of the training data, or on the input data and the ground truth. Note that when adding artificial noise to both the input data and the ground truth, for example, the artificial noise addition unit 267 adds noise to both the input data and the ground truth in such a way that the amount of noise added to the input data is greater than the amount of noise added to the ground truth. However, the method for adding artificial noise to the input data and the ground truth is not limited to the methods described above, and it is sufficient to add artificial noise in a way that creates a difference between the input data and the ground truth. Furthermore, the timing of adding artificial noise can be any timing between the various processing operations performed in the inference processing unit 262. For example, the addition of artificial noise can be performed at any timing, such as by adding artificial noise to the radiation image to be input to the inference processing unit 262, or by adding artificial noise after decorrelation processing.

[0066] After the image with added artificial noise has been appropriately processed, it is input into the processing unit 266. Note that the details of the added artificial noise will be described later.

[0067] The parameter update unit 268 updates the parameters of the machine learning model owned by the computation processing unit 266 based on the computation result and truth value of the computation processing unit 266. The details of the operation of the parameter update unit 268 will be described later.

[0068] The training processing unit 261 does not necessarily need to be included in the radiographic imaging system 1. For example, it can be configured such that the training processing unit 261 is configured on hardware separate from the radiographic imaging system and is pre-trained using appropriate training data to create a learned model, while only the processing of the inference processing unit 262 is performed within the radiographic imaging system. Alternatively, it can be configured such that by including the training processing unit 261 in the radiographic imaging system 1, incremental learning can be performed using training data obtained after installation.

[0069] The display control unit 23 can control the display of the display unit 50, and can make the display unit 50 display the radiographic images before and after the image processing by the image processing unit 22, as well as patient information, etc. The drive control unit 24 can control the driving of the radiation detector 10 and the radiation generator 30, etc. Therefore, by controlling the driving of the radiation detector 10 and the radiation generator 30 by means of the drive control unit 24, the control unit 20 can control the imaging of the radiographic image.

[0070] Storage device 25 may store the operating system (OS), device drivers for peripheral devices, and programs for implementing various application software (including programs for performing processes described later). Storage device 25 may also store information obtained by acquisition unit 21, and radiographic images that have undergone image processing by image processing unit 22. For example, storage device 25 may store radiographic images obtained by acquisition unit 21, and may also store radiographic images that have undergone noise reduction processing described later.

[0071] Note that the control unit 20 can use a common computer configuration including a processor and memory, and can also be configured as a dedicated computer for the radiographic imaging system 1. Although the control unit 20 is used here as an example of an image processing apparatus according to this embodiment, the image processing apparatus according to this embodiment can be a computer as a separate device (external computer) communicatively connected to the control unit 20. Furthermore, the control unit 20 or the image processing apparatus can be, for example, a personal computer, and can be a desktop PC, a laptop PC, or a tablet PC (portable information terminal).

[0072] Note that the processor can be a CPU (Central Processing Unit). Alternatively, the processor can be, for example, an MPU (Microprocessor Unit), a GPU (Graphics Processing Unit), or an FPGA (Field Programmable Gate Array).

[0073] The various functions of the control unit 20 can be implemented by a processor such as a CPU or MPU executing software modules stored in the storage device 25. Alternatively, each function can be implemented by circuitry providing specific functions (such as an ASIC). For example, the image processing unit 22 can be implemented using dedicated hardware such as an ASIC, and the display control unit 23 can be implemented using a dedicated processor such as a GPU, which is different from a CPU. The storage device 25 can be made of any storage medium, such as an optical disk (e.g., a hard disk) or a memory.

[0074] (Configuration of machine learning models)

[0075] Next, refer to Figures 3A to 3C Examples of machine learning models constituting the learned model according to this embodiment are described. One example of a machine learning model used by the computation processing unit 266 according to this embodiment is a multi-layer neural network.

[0076] Figure 3A This is a diagram illustrating an example of a schematic configuration of a neural network model according to this embodiment. Figure 3AThe neural network model 33 shown is designed such that, for input data 31, the neural network model 33 outputs inferred data 32 in which noise has been reduced according to a pre-learned trend. The output radiographic image with reduced noise is an image based on training content in machine learning processing, and features learned by the neural network in this embodiment are used to distinguish between signals and noise included in the radiographic image.

[0077] Note that, for example, convolutional neural networks (hereinafter referred to as "CNNs") can be used in at least a portion of multi-layer neural networks. Additionally, techniques related to autoencoders can be used in at least a portion of multi-layer neural networks.

[0078] Here, we describe the case where CNN is used as a machine learning model for noise reduction processing of radiographic images. Figure 3B This is a diagram illustrating an example of a schematic configuration of a CNN constituting a neural network model 33 according to this embodiment. In the example of the learned model according to this embodiment, when a radiation image is input as input data 31, inference data 32 with reduced noise can be output.

[0079] Figure 3B The CNN shown includes multiple layers responsible for processing and outputting the input values. Note that, as types of layers included in a CNN configuration, there are convolutional layers, downsampling layers, upsampling layers, and merger layers.

[0080] A convolutional layer is a layer that performs convolution processing on input values ​​based on parameters such as the kernel size of the filters, the number of filters, the stride value, and the dilation value. Note that the dimension of the filter kernel size can also be changed according to the dimension of the input image.

[0081] A downsampling layer is a layer that performs processing by refining or combining input values ​​to make the number of output values ​​less than the number of input values. Specifically, for example, Max Pooling is such a process.

[0082] An upsampling layer is a layer that performs processing to make the number of output values ​​greater than the number of input values ​​by copying input values ​​or adding values ​​interpolated from input values. Specifically, for example, there is upsampling processing by deconvolution as such processing.

[0083] A merging layer is a layer that combines multiple source inputs, such as the output values ​​of a certain layer and the pixel values ​​that make up the image, by cascading or adding them together.

[0084] Additionally, the neural network can have an additional layer 34 that adds input data to the data output from the convolutional layer in the first layer on the decoder side. In this case, a shortcut for adding input data before the output can be configured. Depending on this configuration, the CNN can employ a setup for learning the difference between the input data and the ground truth, and can handle noise as a target appropriately.

[0085] It is important to note that the degree to which the trends trained on the training data can be reproduced at inference may differ when the parameters of the layers and nodes constituting the neural network are set differently. In other words, in many cases, since the appropriate parameters vary depending on the implementation mode, the parameters can be changed to preferred values ​​as needed.

[0086] Furthermore, CNNs can achieve better characteristics not only by changing the parameters as mentioned above, but also by changing the CNN configuration. These improved characteristics include, for example, higher accuracy in reducing noise in the output radiation image, shorter processing time, and shorter training time for the machine learning model.

[0087] Note that the CNN configuration used in this embodiment is a U-net type machine learning model, which includes the functionality of an encoder with multiple layers (including multiple downsampling layers) and the functionality of a decoder with multiple layers (including multiple upsampling layers). In other words, the CNN configuration includes a U-shaped configuration with encoder and decoder functions. The U-net type machine learning model is configured (e.g., by using skip connections) such that the geometric information (spatial information) that is blurred in the multiple layers configured as encoders can be used in the same dimension layers (corresponding layers) in the multiple layers configured as decoders.

[0088] Although not shown, as an example of changing the configuration of a CNN, for example, layers with activation functions (e.g., rectified linear units (ReLU)) can be incorporated before or after convolutional layers, and layers that perform various normalization processes (such as batch normalization) can be incorporated before or after such layers.

[0089] These steps of a CNN can extract noise features from an input radiation image.

[0090] Here, the training processing unit 261 includes a parameter update unit 268. For example... Figure 3CAs shown, the parameter update unit 268 performs a loss function calculation based on the inferred data 32 obtained by applying the neural network model 33 of the computation processing unit 266 to the input data 31 in the training data and the true value 35 in the training data. Furthermore, the parameter update unit 268 performs a parameter update process based on the calculated loss function. Here, the loss function is a function that represents the error between the inferred data 32 and the true value 35.

[0091] More specifically, parameter update unit 268 uses, for example, error backpropagation to update the filter coefficients of the convolutional layer, thereby reducing the error between the inferred data 32 represented by the loss function and the true value 35. Error backpropagation is a method of adjusting parameters between each node of the neural network to reduce the aforementioned error. Note that the technique of randomly deactivating the units constituting the CNN (individual neurons or individual nodes) (dropout) can be used for training.

[0092] Furthermore, the learned model used by the processing unit 266 can be a learned model generated using transfer learning. In this case, for example, a learned model for noise reduction processing can be generated by performing transfer learning on a machine learning model learned from radiographic images of objects O to be examined using different species, etc. By performing such transfer learning, a learned model can be efficiently generated even for objects O to be examined where large amounts of training data are difficult to obtain. The different species, etc., of objects O to be examined mentioned here can be, for example, animals, plants, or objects to be examined non-destructively.

[0093] Here, the GPU can perform efficient arithmetic operations by performing parallel processing of a larger amount of data. Therefore, it is effective to use the GPU to perform processing when training is performed multiple times using the CNN learning model as described above. Therefore, in the training processing unit 261 according to this embodiment, a GPU is used in addition to the CPU. Specifically, when a training program including the learning model is executed, training is performed by the cooperation of the CPU and GPU to perform arithmetic operations. Note that in the training process, arithmetic operations can be performed by either the CPU or the GPU alone. In addition, similar to the training processing unit 261, the various processes performed by the inference processing unit 262 can be implemented using the GPU.

[0094] While the configuration of the machine learning model has been described above, this disclosure is not limited to using the CNN model described above. The learning performed by the machine learning model is similar to that of machine learning using a model capable of extracting (representing) features from training data such as images by learning from itself.

[0095] (Training Processing)

[0096] Next, the processing flow of the training processing unit 261 according to this embodiment will be described. Here, as the training data used by the training processing unit 261, it is ideal to use a pair including a radiographic image containing noise as input data and a radiographic image that is substantially free of noise as ground truth. In order to obtain a radiographic image that is substantially free of noise, for example, methods such as creating an average image by performing imaging of the same object to be examined multiple times can be mentioned to eliminate noise components that are not related to the signal. However, especially in the medical field, when considering the invasiveness of radiography, it is impractical to perform the aforementioned methods on actual human bodies, and it is difficult to prepare a pair including a radiographic image containing noise and a radiographic image that is substantially free of noise as training data.

[0097] For this reason, in this embodiment, a radiographic image obtained by imaging a human body for medical purposes is used as the ground truth of the training data, and an image obtained by adding artificial noise corresponding to the noise appearing in the radiographic image to the aforementioned radiographic image is used as the input data for the training data. By training with such training data, the image features of the radiographic image and the characteristics of the artificial noise can be learned. Note that the added noise is not limited to artificial noise. For example, in cases where imaging of the same object to be examined can be performed multiple times, a difference processing can be performed using the radiographic image obtained by imaging the object to be examined and the average image obtained by imaging the same object multiple times to obtain the noise component to be added to the radiographic image.

[0098] The inventors have discovered that when artificial noise is added in the form of noise that simulates the characteristics of an actual radiation image through a pattern such as that described later, a machine learning model can be generated that can output a radiation image in which noise is appropriately reduced even when an actual radiation image is input.

[0099] Considering the above, we will refer to Figures 3D to 7B as well as Figure 12 The processing flow of the training processing unit 261 according to this embodiment is described. Figure 4 This is a flowchart illustrating the processing flow of the training processing unit 261 according to this embodiment. In the method for generating training data according to this embodiment, processing for adding artificial noise is performed before preprocessing including noise decorrelation processing and DC removal processing.

[0100] In step S401, training data is input to the training processing unit 261. The acquisition of the input training data is performed by the acquisition unit 21, and in this embodiment, an appropriate radiation image is acquired as the training data. Note that the acquisition unit 21 can acquire the radiation image stored in the storage device 25, or it can acquire the radiation image from an external storage device 70, etc. Alternatively, the acquisition unit 21 can acquire the radiation image already output by the radiation detector 10. Note that in this embodiment, the training data is the aforementioned radiation image used as both input data and ground truth. After the training data is input to the training processing unit 261 as the same data, appropriate artificial noise is added internally to the input data or the input data and ground truth, resulting in a state where there is a difference between the input data and the ground truth.

[0101] In step S402, the segmentation unit 263 performs region segmentation processing on the input radiation image. Figures 5A to 5C This is an image used to describe the region segmentation process according to this embodiment. Figure 5A In the example shown, the partitioning unit 263 partitions the radiation image 501 into regions to create an appropriately partitioned image 502. Note that the training data input to the training processing unit 261 consists of input data and ground truth values, and the partitioning unit 263 performs operations to partition the regions of the two images at the same locations.

[0102] Here, the number of pixels in the segmented image 502 can be set to any number, as long as it is not greater than the number of pixels within the range of a complete processing operation performed at once in the arithmetic unit operated by the arithmetic processing unit 266, and is large enough to enable the extraction of features related to noise within the image. Note that in the case of radiographic images used for medical purposes, the radiation detector 10 is often a high-resolution detector, and in some cases, the radiographic image 501 may have more than, for example, 2688 pixels × 2688 pixels, exceeding the number of pixels for which the arithmetic unit operated by the arithmetic processing unit 266 can perform arithmetic operations. For example, the number of pixels in the segmented image 502 can be set to approximately 256 pixels × 256 pixels.

[0103] Note that in the operation of training processing unit 261, the processing from steps S401 to S407 is repeated, and in the region segmentation processing in step S402, the extent of the region image can be set to include portions overlapping with the region images created during the loop. In this case, a larger amount of training data can be generated from a single radiation image. Additionally, various data expansions can be performed. Examples of such data expansions include rotation, scaling, scaling, tilting, changes in semaphore, and the addition of offsets. Furthermore, generative adversarial networks (GANs) can be used to perform training data expansion.

[0104] In step S403, artificial noise is added to the segmented image 502 by the artificial noise addition unit 267. As described above, noise that closely approximates the noise actually generated in the radiographic image can be used as artificial noise. An example of the artificial noise added in this embodiment is described below.

[0105] As mentioned earlier, the noise included in radiographic images is broadly divided into quantum noise, which is mainly generated by the fluctuation of radiographic quanta, and system noise generated from detectors and circuits.

[0106] In the radiation detector 10, since a configuration is employed to convert radiation into visible light via a scintillator 11, it is known that high-frequency attenuation occurs in the radiation signal according to the modulation transfer function (MTF) of the scintillator 11. For example, Figure 12 The illustration shows an example of the relationship between MTF and spatial frequency in a radiation detector 10 with a pixel size of 125 μm. In this example, the MTF decreases monotonically towards the Nyquist frequency. The MTF at the Nyquist frequency of 4 lp / mm is, for example, approximately 0.2, and the state is one in which the high-frequency signal is attenuated.

[0107] Similar high-frequency attenuation, as in modulation transfer functions, also appears in quantum noise generated by fluctuations in radiation quanta. On the other hand, system noise generated from detectors and circuits is unaffected by scintillators. The noise included in a radiographic image consists of system noise independent of the radiation dose during imaging and quantum noise that changes depending on the radiation dose during imaging. For this reason, the noise in a digital radiography apparatus has characteristics that make its frequency response (NPS: noise power spectrum) different depending on the intensity of the input radiation. The relationship between the NPS of the correlated noise and the intensity (radiation dose) of the input signal is discussed in... Figure 6 As shown in the image. Note that, Figure 6The vertical axis represents the intensity of the input signal, and the horizontal axis represents the spatial frequency of the noise. Figure 601 illustrates an example of the spatial frequency characteristics of system noise alone. Figure 604 illustrates an example of the spatial frequency characteristics of quantum noise according to the MTF of scintillator 11. Figures 602 and 603 each illustrate examples of the combined spatial frequency characteristics of system noise and quantum noise.

[0108] In the absence of an input signal, as shown in Figure 601, the noise has an NPS that exhibits characteristics similar to white noise derived from system noise. When the input signal is small, the noise has a low NPS with low high-frequency attenuation, as shown in Figure 602, and when the input signal is larger than in Figure 602, the noise has an NPS with even greater high-frequency attenuation, as shown in Figure 603. Furthermore, when the input signal is sufficiently large and quantum noise dominates, the noise has an NPS based on the MTF of the scintillator 11, as shown in Figure 604.

[0109] In view of the above, in this embodiment, the noise obtained by synthesizing white noise corresponding to the system noise of the radiation detector 10 and quantum noise affected by the MTF of the scintillator 11 is treated as artificial noise.

[0110] Here, regarding the synthesis ratio between quantum noise and system noise, for example, the characteristics of the radiation detector 10 can be simulated using the following relationships (Equations 1 to 4). Here, when the standard deviation of the quantum noise is σ q The standard deviation of the system noise generated by the radiation detector 10 is represented by σ. s When represented, the variance σ of the noise in the radiographic image all 2 The following relationship must be satisfied:

[0111] σ all 2 =σ q 2 +σ s 2 (Equation 1).

[0112] Additionally, when the signal of the input image is from I sig When expressed, because of the standard deviation σ of quantum noise q With signal I sig Since they are proportional, the following relationship can be written using the quantum noise coefficient q:

[0113] σ q 2 =q 2 ×I sig (Equation 2).

[0114] In addition, because the standard deviation σ of the system noise s It is related to signal I sig Since the constant value is disproportionate, the following relationship can be written using the system noise coefficient s:

[0115] σ s 2 =s 2 (Equation 3).

[0116] Based on the above, artificial noise (addNoise) can be obtained in the following way. First, the artificial system noise (sNoise) of the simulated system noise is defined as having a standard deviation σ. s Additive white Gaussian noise (AWGN). Next, the artificial quantum noise qNoise, which simulates quantum noise, is defined as having a variance of σ. q 2 The noise is a Poisson distribution and is based on the NPS of the MTF of the scintillator 11. For example, when the value obtained by approximating the MTF of the scintillator 11 using a two-dimensional filter is defined as fMTF, the artificial quantum noise qNoise can be created by convolving fMTF in a noise image that follows a Poisson distribution. Note that since the Poisson distribution can approximate a normal distribution when the variance is large enough, the artificial quantum noise qNoise can be treated as noise that follows a normal distribution. In this case, the artificial noise (addNoise) can be expressed as follows:

[0117] addNoise = sNoise + qNoise (Equation 4).

[0118] It is known that the distribution of noise generated in radiographic images is experimentally approximated by a normal distribution with a mean and median of approximately 0, and that a distribution with a mean and median of approximately 0 can be achieved in the case of artificial noise obtained as described above.

[0119] Note that the coefficients q and s are coefficients representing the performance of the radiation detector 10 and can be obtained by performing measurements in advance. For example, measurements obtained by performing imaging multiple times while varying the radiation dose can be plotted, and the coefficients q for quantum noise and s for system noise can be obtained by approximating a series of measurements using (Equation 1). In this way, arbitrary signal I can be obtained. sig The standard deviation σ of quantum noise in q and the standard deviation of system noise σ s However, the standard deviation σ used to obtain quantum noise... q and the standard deviation of system noise σ s The methods are not limited to those mentioned above, and any known method can be used. The standard deviation σ of quantum noise.q and the standard deviation of system noise σ s The images can be pre-stored in the storage device 25 for each radiation detector 10 used to obtain radiation images, or they can be obtained through measurements performed before imaging.

[0120] Furthermore, although the radiation detector 10 illustrated here as an example uses a method called the "indirect conversion method" (which converts radiation into visible light by means of a scintillator 11), there are also radiation detectors 10 that use a method called the "direct conversion method" (which directly converts radiation into an electrical signal). Regarding such a radiation detector 10, the quantum noise is equivalent in that it has an NPS affected by the MTF, and suitable training can be performed by adding artificial noise, as expressed above (Equation 4), to the input data or the input data and the true value.

[0121] In step S404, the decorrelation unit 264 performs noise decorrelation processing on the segmented image with added artificial noise. Here, reference will be made to... Figure 7A and Figure 7B Describe noise decorrelation processing. Figure 7A This is a graph illustrating the relationship between the variance of noise in a radiographic image and the input signal of the image. Figure 7B This is a graph illustrating the relationship between the noise and the transformed signal value after noise decorrelation processing. The term "noise decorrelation processing" refers to the process of eliminating the correlation between the variance of the noise and the signal.

[0122] As mentioned above, quantum noise proportional to the signal amplitude and system noise disproportionate to the signal amplitude are included in the radiographic image, and as shown in (Equations 1) to (Equations 3) and Figure 7A As shown, there exists a variance σ of noise in the radiographic image. all 2 The characteristic of being proportional to signal I. To more appropriately perform training for noise reduction, rather than dealing with multiplicative noise whose amplitude varies with signal I, such as... Figure 7B As shown, the goal is to transform the noise into a format that is substantially constant regardless of the signal intensity, and to address additive noise. Because quantum noise follows a Poisson distribution, it can be made approximately constant regardless of the intensity of the input radiation by performing a square root transformation or a logarithmic transformation.

[0123] In this embodiment, as a noise decorrelation process, based on the above (Equations 1) to (Equations 3), when the system noise component I0 is defined as...

[0124] [Mathematical Expression 1]

[0125]

[0126] Furthermore, when the input image is represented by Im, the noise-decorrelated image Im' is obtained by performing a square root transformation on the input image Im (Equation 6).

[0127] [Mathematical Expression 2]

[0128]

[0129] Note that the decorrelation unit 264 can perform decorrelation processing according to the following (Equation 7) so that the standard deviation of the transformed noise is 1 in the square root transform.

[0130] [Mathematical Expression 3]

[0131]

[0132] With this processing, the amplitude of noise in a radiographic image can be correlated with the amplitude of the signal.

[0133] In step S405, the DC removal unit 265 performs DC removal processing on each decorrelated segment of the image, thereby removing the DC component (average signal level of the image) from each segment. Especially in radiographic images for medical applications, the dynamic range of the signal of the object being examined may be wide, and the average signal level within the image may vary significantly depending on the segmented region. In some cases, this affects the loss function and hinders training. Therefore, in this embodiment, the DC component is removed from the pixel values ​​of the region image by the DC removal unit 265. Specifically, the DC component is removed from each segment by subtracting the average pixel value of the segmented image from each pixel value of the correlated image, making the average pixel value of the segmented image zero. Through this operation, the differences in the average signal levels between each segmented image can be balanced without affecting the amplitude or frequency characteristics of the noise, and training can proceed stably.

[0134] In step S406, inference processing is performed by the computation processing unit 266 using a machine learning model it possesses. In this process, the image created by the processing up to step S405 is used as input, and inference processing is performed using a machine learning model such as the aforementioned CNN. For example, as... Figure 3C As shown, a configuration can be adopted, for example, using the signal that has undergone DC removal in step S405 as input data 31 and outputting inferred data 32.

[0135] Alternatively, such as Figure 3DAs shown, a configuration with multiple channels for input data (here, an example with two channels is illustrated) and multiple channels for inference data can also be used.

[0136] Here, considering the characteristics of the noise being learned, the noise variance (also referred to as "amplitude") becomes substantially constant regardless of the signal value after performing noise decorrelation processing in step S404. However, as mentioned above, considering the way noise appears in radiographic images, the state is one where the NPS of the noise is still correlated with the input signal. Therefore, although it is desirable to input information about the signal as training information, on the other hand, by performing DC removal processing in step S405 to make the training consistent, the state is entered where information about the signal has been lost.

[0137] In view of the above, the image that has undergone DC removal is input as input data 36 for the first channel, and in addition, an image including information about the semaphore is input as input data 37 for the second channel. For example, for the image that performs noise decorrelation in step S404, a normalized image that has been normalized in a state where information about the semaphore has been stored in the image can be used. Specifically, a value that has been normalized by dividing the value obtained by performing the square root transformation of (Equation 6) or (Equation 7) by using the assumed maximum value max(Im) (e.g., 65535) in Im, so that the maximum value becomes 0 to 1, can be used, as shown in (Equation 8) below.

[0138] [Mathematical Expression 4]

[0139]

[0140] Therefore, training, including information about the semaphore lost due to DC removal, can be performed stably while training is being conducted, thus allowing the characteristics of noise to be learned in a more appropriate manner. As inference data, a configuration is used where inference data 38, with noise reduced from input data 36 of the first channel having DC removed, and inference data 39, with noise reduced from input data 37 of the second channel including information about the semaphore.

[0141] Alternatively, such as Figure 3E As shown, a configuration can be employed that has multiple channels for input data (here, an example with two channels is illustrated) and one channel for inference data. In this case, as the inference data, a configuration can be employed where the noise of the input data 36 with DC removed is reduced, resulting in inference data 43.

[0142] In step S407, parameter update unit 268 compares the inferred data with the true value and calculates a loss function that quantifies the error between the two. Additionally, the parameters of the machine learning model (e.g., the filter coefficients of a convolutional layer in a CNN) are updated based on the loss function. Examples of loss functions that can be mentioned include mean absolute error (L1 loss) and mean squared error (L2 loss). Note that when the inferred data is as follows... Figure 3C In the case of single-channel data shown, a pair of inferred data 32 and true value 35 are compared (and similarly in...). Figure 3E In the example shown, a pair of inferred data 43 and true value 44). The inferred data is as follows: Figure 3D In the case of multiple channels of data shown, parameter update unit 268 compares each corresponding pair. Specifically, for example, parameter update unit 268 calculates the loss function between the DC-removed inferred data 38 of the first channel and the DC-removed true value 41 corresponding to the inferred data 38, and calculates the loss function between the normalized inferred data 39 of the second channel and the normalized true value 42 corresponding to the inferred data 39.

[0143] Here, the distribution of the noise generated in the radiographic image and the artificial noise added in step S403 can approximate a normal distribution with a mean and median of 0. In this case, if the loss function is set to the mean absolute error (L1 loss) or mean squared error (L2 loss) as described above, the components derived from the noise are approximately canceled out by averaging in the loss function calculation. Therefore, the main factor affecting the loss function is the error of the noise-free signal components included in the data. That is, the machine learning model possessed by the processing unit 266 can be trained in a configuration to reproduce the noise-free signal from the input data including noise.

[0144] In step S408, a determination is made regarding whether training has ended. If training has ended, the process terminates; otherwise, the process returns to step S401 to repeat the steps from S401 to S407 using different data. As criteria for determining whether training has ended, appropriate criteria commonly used in common machine learning can be used. For example, it can be determined whether a specific number of iterations have been performed, whether the loss function is equal to or less than a certain level, whether overfitting is occurring, or PSNR or SSIM, as indicators of noise reduction performance in the inference data, can be evaluated to determine whether the performance has reached a sufficient level.

[0145] As described above, by adding artificial noise that simulates the characteristics of noise in an actual radiation image to the input data or the input data and the ground truth, the training processing unit 261 in this embodiment can generate a machine learning model that can output a radiation image with the noise advantageously reduced.

[0146] (Inference Processing Unit)

[0147] Next, refer to Figures 3C to 3E , Figure 5B , Figure 5C and Figure 9 The process flow of the inference processing unit 262 according to this embodiment is described. Figure 9 This is a flowchart illustrating a series of image processing steps performed by the inference processing unit 262 according to this embodiment.

[0148] In a series of image processing steps according to this embodiment, a radiation image, which is the object of processing, is divided into regions and preprocessed, and used as input to a learned model. Subsequently, the output from the learned model undergoes postprocessing corresponding to the preprocessing, and finally, a combination processing is performed on the divided images to generate a radiation image with reduced noise corresponding to the original radiation image. Note that in this embodiment, image processing performed on the image before it undergoes inference processing is referred to as "preprocessing," and image processing performed on the image after it has undergone inference processing and is output is referred to as "postprocessing." In this embodiment, preprocessing includes noise decorrelation processing and DC removal processing, and postprocessing includes DC addition processing corresponding to DC removal processing and inverse transform processing corresponding to noise decorrelation processing.

[0149] When a series of image processing steps according to this embodiment begins, in step S901, the obtaining unit 21 obtains a radiation image. The obtaining unit 21 may obtain a radiation image generated by the radiation detector 10, or it may obtain a radiation image from the storage device 25 or the external storage device 70, etc.

[0150] In step S902, similar to step S402, the segmentation unit 263 performs region segmentation processing on the obtained radiation image to generate multiple segmented images (multiple radiation images). Note that, as Figure 5B As shown, in step S902, the partitioning unit 263 partitions the radiation image 501 into multiple partitioned images R11 to Rij. Note that the number of pixels in each partitioned image is no greater than the number of pixels that the machine learning model used by the processing unit 266 can process completely at one time (e.g., 256 pixels × 256 pixels), and can be arbitrarily set as long as it is large enough to allow the extraction of a number of features related to noise within the image.

[0151] Note that, as Figure 5B As shown, the partitioning unit 263 can be configured to partition regions in a manner that prevents regions from overlapping. Additionally, as... Figure 5C As shown, the segmentation unit 263 can be configured to set the segmented regions in such a way that adjacent regions partially overlap, and to synthesize the overlapping parts when combining the segmented images. In this case, if slight differences in the arithmetic operation results occur at the boundaries of the combined regions in the combined radiographic image, the impact of such differences can be mitigated. Furthermore, convolution operations are performed in the inference process, and to facilitate edge processing of the image at this time, the following configuration can be used: Figure 5C As shown, an appropriate filling region 503 is provided in the radiographic image 501, and subsequent processing can also be performed in the segmented region after the appropriate filling region is provided.

[0152] In step S903, similar to step S404, the decorrelation unit 264 performs noise decorrelation processing on each segmented image generated in step S902. Here, the decorrelation processing is performed using the same parameters as those used in the decorrelation processing performed in the training process. For example, if the standard deviation of the transformed noise has been made to 1 in the training process as shown in Equation 7, then decorrelation processing can be performed similarly using Equation 7.

[0153] Furthermore, by altering the standard deviation of the transformed noise, the effectiveness of the noise reduction processing in the inference process of step S905 can be changed. For example, the transformation can be performed to make the noise larger than it was in the decorrelation processing during training, such as by making the standard deviation of the transformed noise 1.2. Because the learned model operates to reduce noise under the assumption that the noise has a standard deviation of 1, a mitigated noise reduction effect can be achieved. Based on the above configuration, by making the standard deviation in the transformation processing different between training and inference processing, for example, the magnitude of the noise reduction effect can be adjusted according to the request of a diagnostician performing a diagnosis using an image that has undergone noise reduction processing.

[0154] In step S904, DC removal unit 265 performs DC removal processing on each segment of the image that has undergone noise decorrelation processing in step S903 in a manner similar to that in step S405.

[0155] Next, in step S905, based on the segmented images for which DC removal processing was performed in step S904, the computation processing unit 266 performs inference processing using the previously mentioned learned model to generate segmented images with reduced noise. Note that the learned model is sufficient if it is a pre-trained machine learning model, and it is not necessary to perform training for each relevant series of image processing steps. Note that, as in steps S406 and... Figures 3C to 3E As shown, the input applied to the computation processing unit 266 is based on the learning model.

[0156] Furthermore, although this embodiment employs a configuration where the learned model is provided in the control unit 20, the learned model can also be provided in an external storage device 70 or the like connected to the control unit 20. Note that the processing for generating a noise-reduced segmented image by the arithmetic processing unit 266 may include, for example, obtaining a noise-reduced radiation image generated based on the segmented image sent from the control unit 20 using a learned model from the external storage device 70 or the like.

[0157] The following steps are based on Figure 3E The system shown is described as an example. In the relevant system, an image that has undergone DC removal processing is input to a first channel, and an image that has undergone decorrelation processing and has been normalized so that the maximum value is between 0 and 1 is input to a second channel. Furthermore, in this system, the output is an image obtained by performing noise reduction on the image that has undergone DC removal processing (the noise-reduced image of the first channel).

[0158] Note that the unit described below can be used to perform adjustments to the intensity of noise reduction processing in the inference process. Figure 8 This is a schematic diagram illustrating an example of a unit used to adjust the effect of noise reduction processing. As a unit for adjusting the intensity of the noise reduction processing, image 84 is used, which is obtained by mixing the input image 81 with the output image 83 of inference processing using a machine learning model 82 that has been learned, according to a specific coefficient α. The magnitude of α (α is a real number from 0 to 1) can be changed according to user preference. Note that the adjustment processing can be performed, for example, by inference processing unit 262 or arithmetic processing unit 266.

[0159] In step S906, the DC removal unit 265 processes the noise-reduced segmented image generated in step S905 to add the DC component of the relevant segmented image that was removed in step S904.

[0160] In step S907, the decorrelation unit 264 performs an inverse transform on the image segmented by which the DC addition processing was performed in step S906, as performed in step S903. Specifically, if the square root transform according to (Equation 6) has been performed in step S903, then when the segmented image is represented by Im2, the decorrelation unit 264 performs an inverse transform to give (Equation 9).

[0161] [Mathematical Expression 5]

[0162] Im2′=Im2 2 -I0 (Equation 9)

[0163] Additionally, if the square root transformation according to (Equation 7) has been performed in step S903, then when the segmented image is represented by Im2, the inverse transformation is performed in the relevant unit 264 to give (Equation 10).

[0164] [Mathematical Expression 6]

[0165]

[0166] By performing the above processing, the pixel values ​​in each segmented image can be returned to the pixel values ​​corresponding to the pixel values ​​in each segmented image before the noise decorrelation processing. However, since the correlated segmented images have already undergone noise reduction processing in step S905, each pixel value is a pixel value where the noise has been reduced.

[0167] In step S908, the end determination is performed, and if it is determined that the post-processing of all segmented images has been completed, the processing proceeds to step S909; otherwise, the processing flow from step S903 to step S907 is repeated.

[0168] In step S909, the partitioning unit 263 arranges the partitioned images, which underwent inverse transformation processing in step S907, at the respective positions of the partitioned images when the radiation image was partitioned during region partitioning processing in step S902. It then performs a combination processing on the partitioned images to combine multiple partitioned images and generate the entire radiation image. Note that in step S902, when partitioned images are obtained by partially overlapping adjacent regions through partitioning, it is sufficient to perform arithmetic averaging processing on the overlapping portions, where the distance from each original image is used as a weight, and to perform image combination processing such that no horizontal differences in pixel values ​​occur at the overlapping portions.

[0169] After the combination process is performed in step S909, the series of image processing according to this embodiment ends. Figure 10An example of a radiographic image 1001 before a series of image processing steps according to this embodiment and a radiographic image 1002 after a series of image processing steps are illustrated. Comparing radiographic image 1001 and radiographic image 1002, it can be seen that in radiographic image 1002, which has undergone a series of image processing steps according to this embodiment, the noise included in radiographic image 1001 is reduced, and the object O to be inspected included in the radiographic image is clearly shown in radiographic image 1002.

[0170] Note that in this embodiment, the decorrelation unit 264 performs a square root transformation as noise decorrelation processing in steps S404 and S903. However, the noise decorrelation processing is not limited to the aforementioned processing, and other processing can be used, as long as it is a process that can stabilize the variance of the Poisson distribution. For example, the decorrelation unit 264 can perform noise decorrelation processing by performing a logarithmic transformation of the radiation image. Specifically, when the radiation image (input image) is represented by Im, the decorrelation unit 264 performs a logarithmic transformation of the radiation image to give (Equation 11).

[0171] [Mathematical Expression 7]

[0172] Im′=log(Im+I0) (Equation 11)

[0173] In this case, during the inverse transformation process in step S907, when the radiation image is represented by Im2, the decorrelation unit 264 performs an inverse transformation on the radiation image to give (Equation 12).

[0174] [Mathematical Expression 8]

[0175]

[0176] In this case, the correlation between noise and signal amplitude in radiation images can also be eliminated, and the training of machine learning models using training data based on such images can be simplified.

[0177] Therefore, in a series of image processing according to this embodiment, by using the learned model created by the training processing unit 261 to perform the processing of the inference processing unit 262, a radiographic image that has undergone advantageous noise reduction in the digital radiography apparatus can be generated.

[0178] As described above, the control unit 20 according to this embodiment includes an acquisition unit 21 and a processing unit 266. The acquisition unit 21 acquires a first radiation image of the object O to be inspected. The processing unit 266 serves as an example of a generation unit that generates a second radiation image with reduced noise compared to the first radiation image by inputting the first radiation image acquired by the acquisition unit 21 into a learned model obtained by training data including a radiation image obtained by adding noise attenuating high-frequency components. Here, the noise includes noise attenuating high-frequency components. Specifically, the noise includes noise based on the modulation transfer function (MTF) of the scintillator 11 included in the radiation detector 10.

[0179] In this embodiment, artificial noise includes, for example, noise obtained by synthesizing the system noise of the simulated radiation detector 10 (such as additive white noise) and quantum noise (noise with attenuated high-frequency components) that takes into account the effect of the MTF on noise following a Poisson distribution. For example, artificial noise may include noise obtained by synthesizing the system noise of the simulated radiation detector 10 and the noise with attenuated high-frequency components at a predetermined synthesis ratio. Alternatively, for example, artificial noise may include noise obtained by synthesizing the system noise of the simulated radiation detector 10 and the noise with attenuated high-frequency components at a synthesis ratio determined using the relationship between the variance of the overall noise included in the radiation image of the object O to be examined for training data, the variance of the system noise, the variance of the quantum noise, and the signal values ​​of the radiation image. Furthermore, artificial noise may include noise obtained by synthesizing the system noise of the simulated radiation detector 10 and the noise with attenuated high-frequency components at a first synthesis ratio, and noise obtained by synthesizing the two aforementioned noises at a second synthesis ratio different from the first synthesis ratio. Additionally, artificial noise may include random noise and noise different from random noise.

[0180] Using the above configuration, the control unit 20 according to this embodiment can use a learned model to generate radiographic images, including noise reduction from the radiographic images caused by noise attributable to scintillator blurring (noise with attenuated high-frequency components). By using such radiographic images, more appropriate image analysis can be supported, and appropriate image diagnosis by physicians can be facilitated.

[0181] The control unit 20 also includes a decorrelation unit 264. The decorrelation unit 264 serves as an example of a transformation unit that performs a transformation process on the radiographic image of the object O to be inspected in order to stabilize (make the variance approximately constant) the variance of the noise following a Poisson distribution included in the radiographic image of the object O. The decorrelation unit 264 performs the transformation process on the first radiographic image, and the arithmetic processing unit 266 generates a second radiographic image based on the first radiographic image that has undergone the transformation process. Additionally, the decorrelation unit 264 performs an inverse transformation process on the second radiographic image.

[0182] More specifically, the relevant unit 264 performs a square root transform on the first radiation image and an inverse square root transform on the second radiation image. This configuration further simplifies the training of the machine learning model and improves the accuracy of noise reduction processing using the learned model.

[0183] The control unit 20 also includes a DC removal unit 265. The DC removal unit 265 serves as an example of an adjustment unit that adjusts the average pixel values ​​included in the radiation image of the object O to be inspected. The DC removal unit 265 subtracts the average pixel values ​​from each pixel value included in the first radiation image, which has undergone transformation processing by the decorrelation unit 264. The arithmetic processing unit 266 generates a second radiation image based on the first radiation image with the subtracted average. The DC removal unit 265 adds the subtracted average to each pixel value included in the second radiation image. The decorrelation unit 264 performs an inverse transformation on the second radiation image with the added average. With this configuration, variations in brightness of each input image (radiation image) can be suppressed, and the machine learning model can easily learn the characteristics of the shape (distribution) of noise in the image, thus improving the accuracy of noise reduction processing using the learned model.

[0184] Note that the training data may include a radiation image obtained by performing a transformation process related to noise decorrelation on the radiation image of the object O to be examined. Alternatively, the training data may include a radiation image obtained by subtracting the average of the pixel values ​​from each pixel value included in the radiation image for which the transformation process has been performed. Furthermore, the training data may include a radiation image obtained by performing a square root transformation such that the standard deviation of the noise becomes greater than the standard deviation of the noise in the square root transformation performed on the first radiation image.

[0185] Note that the training data may include the following: In this data, a radiation image obtained by adding artificial noise to the radiation image of the object O to be examined is set as input data, and the radiation image of the object O to be examined is set as the ground truth. Alternatively, the training data may include the following: In this data, a radiation image obtained by adding artificial noise to a radiation image that has undergone transformation processing related to noise decorrelation processing is set as input data, and the radiation image that has undergone transformation processing is set as the ground truth. Furthermore, the training data may include the following: In this data, a radiation image obtained by adding artificial noise to a radiation image that has had its pixel values ​​subtracted from its average value is set as input data, and the radiation image that has had its pixel values ​​subtracted from its average value is set as the ground truth.

[0186] The control unit 20 also includes a partitioning unit 263. Partitioning unit 263 serves as an example of a partitioning unit that divides a radiation image into multiple radiation images. Partitioning unit 263 partitions the radiation image of the object O to be inspected into multiple first radiation images. The processing unit 266 generates multiple second radiation images based on the multiple first radiation images. Partitioning unit 263 combines the multiple second radiation images to generate a noise-reduced third radiation image. In this case, the radiation images used in the training data may include multiple radiation images obtained by partitioning the radiation image of the object O to be inspected. According to this configuration, even if the number of pixels in the radiation image exceeds the number of pixels that the learned model can process at one time, a radiation image with noise reduction processing can still be generated.

[0187] Note that in this embodiment, in steps S402 and S902, the segmentation unit 263 divides the obtained radiation image into multiple segmented images. However, if the number of pixels in the radiation image to which noise reduction processing is intended is equal to or less than the number of pixels that the machine learning model can process at one time, it is not necessary to perform the aforementioned segmentation processing. In this case, it is sufficient to perform the processing from step S402 forward and the processing from step S902 forward on the entire obtained radiation image.

[0188] Note that even when processing the entire radiographic image, the DC removal processing in steps S405 and S904 and the DC addition processing in step S906 can be performed. In this case, the brightness variations of individual radiographic images can be suppressed, and the occurrence of situations where the brightness variations of individual radiographic images output from the learned model are emphasized can be prevented.

[0189] Furthermore, in this embodiment, the decorrelation unit 264 performs a square root transformation as noise decorrelation processing in steps S404 and S903. However, the noise decorrelation processing is not limited to the aforementioned processing, and other processing can be used, as long as it is a process that can stabilize the variance of the Poisson distribution. For example, the decorrelation unit 264 can perform noise decorrelation processing by performing a logarithmic transformation of the radiation image. Specifically, when the radiation image (input image) is represented by Im, the decorrelation unit 264 performs a logarithmic transformation of the radiation image to give (Equation 11).

[0190] [Mathematical Expression 9]

[0191] Im′=log(Im+I0) (Equation 11)

[0192] In this case, during the inverse transformation process in step S907, when the radiation image is represented by Im2, the decorrelation unit 264 performs an inverse transformation on the radiation image to give (Equation 12).

[0193] [Mathematical Expression 10]

[0194]

[0195] In this case, the correlation between noise and signal amplitude in radiation images can also be eliminated, and the training of machine learning models using training data based on such images can be simplified.

[0196] Furthermore, as described above, a training device for training the machine learning model or generating training data can be provided separately from the control unit 20, and it is sufficient for the training device to have a configuration corresponding to an image processing unit 22 used as an example of an acquisition unit 21 and a training unit. Additionally, it is sufficient that the learned model is a pre-trained machine learning model, and it is not necessary to perform training every time image processing, including noise reduction processing using the learned model, is performed. Furthermore, the learned model can be provided in an external storage device 70 connected to the control unit 20, and for example, in a cloud server, FOG server, edge server, etc., connected to the control unit 20 via a network 60 (such as the Internet).

[0197] (Example 2)

[0198] Next, refer to Figure 11A method for generating training data according to Embodiment 2 of this disclosure will be described. As described above, in the training processing unit, the timing of adding artificial noise can be any timing between the various processing operations performed in the inference processing unit. In the method for generating training data according to this embodiment, the processing for adding artificial noise is performed after preprocessing including noise decorrelation processing and DC removal processing. Note that the training processing unit for generating training data has the same configuration as the training processing unit 261 described in Embodiment 1, therefore, descriptions using reference numerals associated with the training processing unit 261 according to Embodiment 1 are omitted here.

[0199] Steps S1101, S1102, S1103, and S1104 are identical to steps S401, S402, S404, and S405 of the method for generating training data according to Embodiment 1. In step S1105, the artificial noise addition unit 267 adds artificial noise to the image after the DC removal process in step S1104. Therefore, in the artificial noise addition process (step S1105) according to this embodiment, the artificial noise addition unit 267 adds artificial noise addNoise2, which is different from the artificial noise addNoise according to Embodiment 1.

[0200] The artificial noise addNoise2 is described in more detail below. First, the artificial system noise sNoise2, which simulates system noise, is defined as AWGN. Additionally, the artificial quantum noise qNoise2, which simulates quantum noise, is defined as noise that follows a Poisson distribution and whose spatial frequency characteristics are based on the MTF of the scintillator 11. In this embodiment, when the value obtained by approximating the MTF of the scintillator 11 using a two-dimensional filter is defined as fMTF, fMTF is convolved in the noise image pNoise, which follows a Poisson distribution, to give the following (Equation 13):

[0201] qNoise2 = pNoise × fMTF (Equation 13).

[0202] In this case, when α represents the synthesis ratio, the artificial noise added according to this embodiment can be the artificial noise given by the following (Equation 14).

[0203] [Mathematical Expression 11]

[0204]

[0205] Here, the synthesis ratio α takes a value from 0 to 1 and can be applied as a suitable fixed value or a randomly changed value (arbitrary value). However, it is necessary to make the standard deviations of the artificial system noise sNoise2, the artificial quantum noise qNoise2, and the added artificial noise addNoise2 the same. Here, in step S1103, for example, if the standard deviations are normalized to 1 according to (Equation 7), it is sufficient to adopt a configuration that makes the standard deviations of the artificial system noise sNoise2, the artificial quantum noise qNoise2, and the added artificial noise addNoise2 equal to 1.

[0206] Additionally, the synthesis ratio α can be used with respect to the variance σ of the noise in the radiographic image. all 2 The relationship between noise and signal, expressed by Equation 1, is based on the signal I of the radiographic image. sig To determine. For example, when given the following (Equation 15),

[0207] [Mathematical Expression 12]

[0208]

[0209] The following can be given (Equation 16).

[0210] [Mathematical Expression 13]

[0211]

[0212] Note that the method described above for determining the synthesis ratio α using the relationship between noise and signal expressed in Equation 1 is an example. Besides this method, the synthesis ratio α can also be determined based on the relationship between signal and noise expressed in Equation 1 and the signal I. sig To determine.

[0213] The artificial noise addition unit 267 adds the artificial noise addNoise2 determined in this way to the image Ime' of the segmentation that has had DC removed in step S1104 to obtain the segmentation image represented by (Equation 17).

[0214] Im add =Im e +addNoise2 (Equation 17)

[0215] In this embodiment, the image Ime', whose DC has been removed in step S1104, is used as the ground truth of the training data, and the image Im, whose artificial noise addNoise2 has been added in step S1105, is used. addThis serves as input data for training. The learned model, obtained by training on such data, can learn the trends in the characteristics of artificial noise corresponding to the noise generated in the radiographic image, and can output a radiographic image with noise reduced from the original radiographic image.

[0216] The subsequent steps S1106, S1107, and S1108 are the same as steps S406, S407, and S408 according to Embodiment 1.

[0217] As described above, in this embodiment, for the input data of the training data, artificial noise addNoise2, obtained by adding (synthesizing) artificial system noise sNoise2 and artificial quantum noise qNoise2 simulating quantum noise, is added to the radiographic image after noise decorrelation processing. When training is performed according to this sequence of processing, similar to Embodiment 1, the training processing unit 261 can also create a suitable learned model and can generate a radiographic image that has undergone advantageous noise reduction in the digital radiography apparatus by performing the processing of the inference processing unit 262. Note that although an example of adding artificial noise after DC removal is described here, similar effects can be obtained even when adding artificial noise after noise decorrelation.

[0218] (Modification 1)

[0219] A learned model for noise reduction processing can be prepared for each object O to be examined in the radiographic image undergoing noise reduction processing. For example, a learned model can be prepared where a radiographic image of a human chest and abdomen has been set as an object O to be examined and has been used as training data, and a learned model can be prepared where a radiographic image of an object related to non-destructive inspection has been set as an object O to be examined and has been used as training data. In this case, the computation processing unit 266 can select the learned model to be used for noise reduction processing according to each object O to be examined in the radiographic image to be processed.

[0220] Because the feature portions in the image will differ for each object O to be examined, by preparing a learned model for each object O in this way, the computational processing unit 266 can generate radiographic images with noise reduced more appropriately according to the type of object O to be examined. Similarly, a learned model can be prepared for each object O to be examined (e.g., for each patient, for each imaging site, or for each object in a non-destructive examination). Note that, similar to Modification 2 described later, these learned models can be generated by performing transfer learning, or by training each learned model using a separate training dataset.

[0221] (Revised 2)

[0222] For the learned model to be used for noise reduction processing, training can be performed to adjust (tune) each radiation detector to generate a dedicated learned model for the relevant radiation detector. For example, transfer learning using a radiation image obtained by a particular radiation detector can be performed using a general learned model for generating a noise-reduced radiation image, thereby generating a dedicated learned model for the radiation detector discussed in this discussion. Alternatively, the dedicated learned models for each radiation detector can be stored in storage device 25 or external storage device 70, such as a server, associated with the ID of the corresponding radiation detector. In this case, when noise reduction processing is performed on a radiation image obtained using a radiation detector, the control unit 20 can identify and utilize the dedicated learned model for the relevant radiation detector from multiple learned models based on the radiation detector ID. By using such a dedicated learned model for the radiation detector, the accuracy of noise reduction processing can be improved. It should be noted that the methods for generating dedicated learned models for each radiation detector are not limited to transfer learning, and each learned model can be generated by training with training data obtained using radiation images obtained from each radiation detector.

[0223] Furthermore, for radiation images that undergo noise reduction processing using a learned model, the computation processing unit 266 can input the radiation image into the learned model after transforming the radiation image based on the MTF of the scintillator in the radiation detector used to obtain the radiation image. If the MTF of the scintillator in the radiation detector used to obtain the radiation image for noise reduction differs from the MTF of the scintillator in the radiation detector used to obtain the radiation image used in the training data, there is a possibility that the noise reduction effect will not be advantageous. Therefore, the computation processing unit 266 can transform the radiation image based on the ratio of the MTF of the scintillator included in the radiation detector used to obtain the radiation image for noise reduction to the MTF of the scintillator included in the radiation detector used to obtain the radiation image used in the training data.

[0224] In this case, the processing unit 266 inputs the transformed radiation image into the learned model. Furthermore, the processing unit 266 uses the reciprocal of the ratio of the MTF of the associated scintillator to transform the noise-reduced radiation image output from the learned model. By this means, when the MTF of the scintillator used to obtain the radiation image for noise reduction differs from the MTF of the scintillator used to obtain the radiation image used in the training data, the weakening of the noise reduction effect can be suppressed. Note that it is sufficient to perform the correlation transformation processing of the radiation image before the noise reduction processing (step S905). Furthermore, at the timing after the noise reduction processing (step S905), it is sufficient to perform the inverse correlation transformation processing of the radiation image in the sequence corresponding to the sequence of processing that performs the correlation transformation of the radiation image. For example, if the correlation transformation processing is performed before the noise decorrelation processing (step S903), it is sufficient to perform the inverse correlation transformation processing after the inverse transformation processing related to the noise decorrelation processing (step S907). Note that separate components can be provided for performing noise reduction processing, transformation processing of the relevant radiation image, and corresponding inverse transformation processing, and in this case, a function block including the corresponding component can be used as the arithmetic processing unit 266.

[0225] (Revised 3)

[0226] For the machine learning model used by the processing unit 266, any layer configuration such as a variational autoencoder (VAE), a fully convolutional network (FCN), SegNet, or DenseNet can be combined and used as a CNN configuration. In common neural networks, by configuring each unit (each neuron or each node) to output a scalar value, the neural network is configured such that spatial information, for example, related to the spatial positional relationships (relative positions) between features in an image, is reduced. In this way, for example, training can be performed to reduce the effects of local distortions or parallel shifts in the image. On the other hand, in capsule networks, for example, a configuration is adopted such that spatial information is preserved by configuring each unit (each capsule) to output spatial information as a vector. In this way, for example, training can be performed considering the spatial positional relationships between features in an image.

[0227] (Revised 4)

[0228] Furthermore, the training data for various learned models is not limited to data obtained using the radiation detector that performs the actual imaging, and depending on the desired configuration, the training data can be data obtained using the same type of radiation detector, or data obtained using the same type of radiation detector, etc. It should be noted that in the learned models for noise reduction processing according to the various embodiments and modifications described above, for example, it is conceivable that the magnitude of the luminance value of the radiation image, as well as the order and tilt, position, distribution, and continuity of the bright and dark parts of the radiation image, will be extracted as part of the feature quantities and used for estimation processing related to the generation of the noise-reduced radiation image.

[0229] Furthermore, the learned models according to the various embodiments and modifications described above can be provided in the control unit 20. These learned models may be comprised, for example, software modules executed by processors such as CPUs, MPUs, GPUs, or FPGAs, or circuitry providing specific functions such as ASICs. Additionally, these learned models may be provided in different devices, such as servers, connected to the control unit 20. In this case, the control unit 20 can use the learned models by connecting to a server containing the learned models via any network (such as the Internet). Here, the server containing the learned models may be, for example, a cloud server, a FOG server, or an edge server.

[0230] According to the above embodiments and modifications of this disclosure, advantageous noise reduction processing can be performed in digital radiography apparatus.

[0231] (Other examples)

[0232] The present invention can also be implemented by having software (programs) implementing one or more functions according to the above embodiments and modifications supplied to a system or device via a network or storage medium, and the computer of the system or device reading and executing the processing of the program. The computer has one or more processors or circuits, and in order to read and execute computer-executable instructions, it may include a network of multiple computers, multiple processors, or circuits that are separate from each other.

[0233] In this context, the processor or circuitry may include a central processing unit (CPU), a microprocessor unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or a field-programmable gateway (FPGA). Alternatively, the processor or circuitry may include a digital signal processor (DSP), a data stream processor (DFP), or a neural processing unit (NPU).

[0234] This invention is not limited to the above embodiments and modifications, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the appended claims are attached to inform the public of the scope of the invention.

[0235] This application claims priority to Japanese Patent Application No. 2020-143037, filed on August 26, 2020, and Japanese Patent Application No. 2021-123329, filed on July 28, 2021, which are hereby incorporated herein by reference in their entirety.

[0236] This disclosure includes the following configurations, methods, and procedures.

[0237] (Configuration 1)

[0238] An image processing apparatus, comprising:

[0239] The obtaining unit is configured to obtain a first radiographic image of the object to be inspected; and

[0240] A generation unit is configured to generate a second radiation image with reduced noise compared to the first radiation image by inputting a first radiation image obtained by the obtaining unit into a learned model, wherein the learned model is obtained by training using training data, the training data including radiation images obtained by adding noise attenuated by adding high-frequency components.

[0241] (Configuration 2)

[0242] According to the image processing apparatus of configuration 1, the generation unit is configured to generate a second radiation image by inputting the first radiation image into a learned model, the learned model being obtained by training using training data, the training data including radiation images obtained by adding noise with high-frequency components attenuated according to the modulation transfer function of a scintillator included in the radiation detector.

[0243] (Configuration 3)

[0244] According to the image processing apparatus of configuration 1 or 2, wherein the generation unit is configured to generate the second radiation image by inputting the first radiation image into a learned model, the learned model being obtained by training using training data, the training data including radiation images obtained by adding artificial noise, the artificial noise including noise simulating system noise of a radiation detector and noise attenuating high-frequency components.

[0245] (Configuration 4)

[0246] According to the image processing apparatus of configuration 3, the artificial noise includes noise obtained by synthesizing the noise of the system noise of the simulated radiation detector and the noise attenuated by the high-frequency components at a predetermined synthesis ratio.

[0247] (Configuration 5)

[0248] According to the image processing apparatus of configuration 4, the generation unit is configured to generate the second radiation image by inputting the first radiation image into a learned model, the learned model being obtained by training using training data, the training data including a radiation image obtained by adding artificial noise obtained by synthesizing the noise of the system noise of the simulated radiation detector and the noise attenuated by high-frequency components at a first synthesis ratio, and a radiation image obtained by adding artificial noise obtained by synthesizing the noise of the system noise of the simulated radiation detector and the noise attenuated by high-frequency components at a second synthesis ratio, the second synthesis ratio being different from the first synthesis ratio.

[0249] (Configuration 6)

[0250] According to any one of configurations 3 to 5, the image processing apparatus wherein the mean or median of the artificial noise is 0.

[0251] (Configuration 7)

[0252] The image processing apparatus according to any one of configurations 1 to 6 further includes:

[0253] A transformation unit is configured to perform transformation processing on a radiographic image of an object to be inspected, so as to stabilize the variance of the noise in the radiographic image of the object, which follows a Poisson distribution.

[0254] in:

[0255] The transformation unit is configured to perform transformation processing on the first radiation image;

[0256] The generation unit is configured to generate a second radiation image based on the first radiation image that has undergone transformation processing; and

[0257] The transformation unit is configured to perform an inverse transformation process on the second radiation image.

[0258] (Configuration 8)

[0259] According to the image processing apparatus described in configuration 7, the transformation unit is configured to:

[0260] Perform square root transformation on the first radiation image; and

[0261] The second radiation image is subjected to the inverse transformation of the square root transformation.

[0262] (Configuration 9)

[0263] The image processing apparatus according to configuration 7 or 8 further includes:

[0264] An adjustment unit is configured to adjust the average value of pixel values ​​included in the radiographic image of the object to be inspected.

[0265] in:

[0266] For each of the pixel values ​​included in the first radiation image from which the transformation process is performed, the adjustment unit is configured to subtract the average value of the pixel values;

[0267] The generation unit is configured to generate a second radiation image based on the first radiation image minus the average value;

[0268] The adjustment unit is configured to add the average value to each of the pixel values ​​included in the second radiation image; and

[0269] The transformation unit performs the inverse transformation process on the second radiation image with the average value added.

[0270] (Configuration 10)

[0271] According to any one of configurations 7 to 9, the image processing apparatus, wherein the training data includes a radiation image obtained by performing a transformation process on a radiation image of an object to be examined.

[0272] (Configuration 11)

[0273] According to the image processing apparatus of configuration 10, the training data includes a radiation image for which a square root transformation has been performed such that the standard deviation of the noise becomes greater than the standard deviation of the noise relative to the square root transformation performed on the first radiation image.

[0274] (Configuration 12)

[0275] According to any one of configurations 7 to 11, the image processing apparatus wherein the training data comprises a radiation image obtained by subtracting the average of the pixel values ​​from each of the pixel values ​​included in the radiation image from which the transformation processing is performed.

[0276] (Configuration 13)

[0277] According to the image processing apparatus of configuration 12, the training data includes a radiation image obtained by performing normalization processing on a radiation image to which transformation processing is performed, and a radiation image obtained by subtracting the average value of each pixel value included in the radiation image to which transformation processing is performed, as input data.

[0278] (Configuration 14)

[0279] According to any one of configurations 1 to 13, the image processing apparatus, wherein the training data comprises data in which a radiographic image obtained by adding noise to a radiographic image of an object to be inspected is set as input data, and a radiographic image of the object to be inspected is set as ground truth.

[0280] (Configuration 15)

[0281] According to any one of configurations 7 to 13, the image processing apparatus, wherein the training data includes data in which a radiation image obtained by performing a transformation process on a radiation image with added noise is set as input data, and a radiation image for which the transformation process is performed is set as the true value.

[0282] (Configuration 16)

[0283] According to the image processing apparatus described in configuration 9 or 12, the training data includes data in which a radiation image obtained by subtracting the average pixel value from a radiation image with added noise is set as input data, and a radiation image obtained by subtracting the average pixel value is set as the true value.

[0284] (Configuration 17)

[0285] According to any one of configurations 7 to 13, the image processing apparatus, wherein the training data includes data in which a radiographic image obtained by performing a transformation process on a radiographic image of an object to be inspected with noise added is set as input data, and a radiographic image obtained by performing a transformation process on a radiographic image of an object to be inspected is set as the true value.

[0286] (Configuration 18)

[0287] According to the image processing apparatus of configuration 9 or 12, the training data includes data in which a radiation image obtained by subtracting the average of the pixel values ​​from each of the pixel values ​​included in the radiation image obtained by performing a transformation process on the radiation image of the object to be inspected with added noise is set as input data, and a radiation image obtained by subtracting the average of the pixel values ​​from each of the pixel values ​​included in the radiation image obtained by performing a transformation process on the radiation image of the object to be inspected is set as true data.

[0288] (Configuration 19)

[0289] The image processing apparatus according to any one of configurations 1 to 18 further includes:

[0290] A segmentation unit, configured to divide a radiation image into multiple radiation images.

[0291] in:

[0292] The segmentation unit is configured to divide the radiographic image of the object to be inspected into a plurality of first radiographic images;

[0293] The generation unit is configured to generate a plurality of second radiation images based on the plurality of first radiation images; and

[0294] The segmentation unit is configured to combine the plurality of second radiation images to generate a third radiation image with reduced noise.

[0295] (Configuration 20)

[0296] According to any one of configurations 1 to 19, the image processing apparatus wherein the radiation images used in the training data include a plurality of radiation images obtained by dividing the radiation images of the object to be examined.

[0297] (Configuration 21)

[0298] According to any one of configurations 1 to 20, the image processing apparatus, wherein the generating unit is configured to:

[0299] The first radiation image is transformed according to the ratio of the modulation transfer function of the scintillator included in the radiation detector used to obtain the first radiation image to the modulation transfer function of the scintillator included in the radiation detector used to obtain the radiation image of the object to be examined used in the training data.

[0300] The learned model is used to generate the second radiation image from the transformed first radiation image; and

[0301] The second radiation image is transformed according to the reciprocal of the ratio.

[0302] (Configuration 22)

[0303] The image processing apparatus according to any one of configurations 1 to 21, wherein the learned model includes a neural network, the neural network including a U-shaped configuration having encoder and decoder functions, and the neural network having an additive layer configured to add input data to data output from a first convolutional layer on the decoder side.

[0304] (Configuration 23)

[0305] An image processing apparatus, comprising:

[0306] A generation unit is configured to generate a second radiation image with reduced noise compared to the first radiation image by inputting a first radiation image into a learned model, wherein the learned model is obtained by training using training data including radiation images of the object to be examined and radiation images obtained by adding random noise and noise different from the random noise to the radiation images.

[0307] (Configuration 24)

[0308] A training device, comprising:

[0309] A training unit configured to train a machine learning model using training data, the training data including radiographic images obtained by adding noise attenuated by adding high-frequency components.

[0310] (Configuration 25)

[0311] The training apparatus according to configuration 24 further includes:

[0312] A transformation unit is configured to perform transformation processing on a radiographic image of an object to be inspected, so as to stabilize the variance of the noise in the radiographic image of the object, which follows a Poisson distribution.

[0313] in:

[0314] The training unit is configured to train the machine learning model using training data, which is a radiographic image of the object to be examined, for which the transformation unit performs transformation processing.

[0315] (Configuration 26)

[0316] The training apparatus according to configuration 25 further includes:

[0317] An adjustment unit is configured to adjust the average value of pixel values ​​included in the radiographic image of the object to be inspected.

[0318] in:

[0319] From each of the pixel values ​​included in the radiographic image of the object to be examined, from which the transformation processing is performed, the adjustment unit is configured to subtract the average value of the pixel values; and

[0320] The training unit is configured to train the machine learning model using training data, which consists of radiographic images of the object to be examined, minus the average value.

[0321] (Configuration 27)

[0322] The training apparatus according to any one of configurations 24 to 26 further includes:

[0323] A segmentation unit, configured to divide the radiographic image of the object to be examined into multiple radiographic images.

[0324] in:

[0325] The training unit is configured to train a machine learning model using training data, which uses multiple radiation images divided by the partitioning unit.

[0326] (Method 1)

[0327] An image processing method, comprising:

[0328] Obtain the first radiographic image of the object to be examined; and

[0329] A second radiation image with reduced noise compared to the first radiation image is generated by inputting the obtained first radiation image into a learned model, wherein the learned model is obtained by training using training data, which includes radiation images obtained by adding noise attenuated by adding high-frequency components.

[0330] (Method 2)

[0331] The image processing method according to method 1 further includes:

[0332] A transformation process is performed on the first radiation image to stabilize the variance of the noise in the first radiation image that follows a Poisson distribution; and

[0333] Perform an inverse transform on the second radiation image.

[0334] in:

[0335] Generating the second radiation image involves generating the second radiation image based on the first radiation image after performing a transformation process.

[0336] (Method 3)

[0337] The image processing method according to method 2 further includes:

[0338] Subtract the average of the pixel values ​​from each of the pixel values ​​included in the first radiation image from which the transformation process is performed; and

[0339] The average value is added to each pixel value included in the second radiation image.

[0340] in:

[0341] Generating the second radiation image includes generating the second radiation image based on the first radiation image minus the average value; and

[0342] Performing the inverse transformation process includes performing the inverse transformation process on the second radiation image with the average value added.

[0343] (Method 4)

[0344] The image processing method according to any one of methods 1 to 3 further includes:

[0345] The radiographic image of the object to be examined is divided into multiple first radiographic images; and

[0346] Multiple second-radiation images are combined to generate a noise-reduced third-radiation image.

[0347] in:

[0348] Generating the second radiation image includes generating a plurality of second radiation images based on the plurality of first radiation images.

[0349] (Method 5)

[0350] A training method, comprising:

[0351] The machine learning model is trained using training data, which includes radiographic images obtained by adding noise attenuated by adding high-frequency components.

[0352] (Method 6)

[0353] The training method according to method 5 further includes:

[0354] A transformation process is performed on the radiographic image of the object to be inspected to stabilize the variance of the noise, which follows a Poisson distribution, included in the radiographic image of the object.

[0355] in:

[0356] The training includes training the machine learning model using training data, which is a radiographic image of the object to be examined that undergoes transformation processing.

[0357] (Method 7)

[0358] The training method according to method 6 further includes:

[0359] The average of the pixel values ​​is subtracted from each of the pixel values ​​included in the radiographic image of the object to be examined from the image to which the transformation process is performed.

[0360] in:

[0361] The training includes training the machine learning model using training data, which is a radiographic image of the object to be examined, minus the average value.

[0362] (Method 8)

[0363] The training method according to any one of methods 5 to 7 further includes:

[0364] The radiographic image of the object to be examined is divided into multiple radiographic images.

[0365] in:

[0366] The training includes training a machine learning model using training data, which uses multiple partitioned radiation images.

[0367] (Program 1)

[0368] A program, when executed by a computer, causes the computer to perform the steps of the image processing method according to any one of methods 1 to 4.

[0369] (Program 2)

[0370] A program, when executed by a computer, causes the computer to perform the steps of the training method according to any one of methods 5 to 8.

[0371] [List of Reference Symbols]

[0372] 20: Control unit (image processing device), 21: Acquisition unit, 266: Calculation and processing unit (generation unit).

Claims

1. An image processing apparatus, comprising: An acquisition unit is configured to acquire a first radiation image of an object to be inspected from a radiation detector including a scintillator; as well as A generation unit is configured to generate a second radiation image with reduced noise compared to the first radiation image by inputting the first radiation image obtained by the obtaining unit into a learned model. The learned model is obtained by training using training data, which includes a third radiation image and a fourth radiation image. The third radiation image is obtained by adding a first noise that attenuates the high-frequency components compared to the low-frequency components, and the fourth radiation image is obtained by adding a second noise that attenuates the high-frequency components more than the first noise.

2. The image processing apparatus of claim 1, wherein the generation unit is configured to generate the second radiation image by inputting the first radiation image into a learned model, the learned model being obtained by training using training data including the third radiation image and the fourth radiation image, the third radiation image being obtained by adding the first noise, which is attenuated compared to the low-frequency components, according to the modulation transfer function of the scintillator included in the radiation detector, and the fourth radiation image being obtained by adding the second noise, which is attenuated compared to the low-frequency components, according to the modulation transfer function of the scintillator included in the radiation detector.

3. The image processing apparatus of claim 1, wherein the generation unit is configured to generate the second radiation image by inputting the first radiation image into a learned model, the learned model being obtained by training using training data, the training data including radiation images obtained by adding artificial noise, the artificial noise including noise simulating system noise of a radiation detector and noise attenuating high-frequency components compared to low-frequency components.

4. The image processing apparatus of claim 3, wherein the artificial noise comprises noise obtained by synthesizing noise of a simulated radiation detector system noise and noise attenuated by a high-frequency component compared to a low-frequency component at a predetermined synthesis ratio.

5. The image processing apparatus of claim 4, wherein the generation unit is configured to generate the second radiation image by inputting the first radiation image into a learned model, the learned model being obtained by training using training data including the third radiation image and the fourth radiation image, the third radiation image being obtained by adding artificial noise obtained by synthesizing noise from the system noise of a simulated radiation detector and noise attenuated compared to low-frequency components at a first synthesis ratio, and the fourth radiation image being obtained by adding artificial noise obtained by synthesizing noise from the system noise of a simulated radiation detector and noise attenuated compared to low-frequency components at a second synthesis ratio, the second synthesis ratio being different from the first synthesis ratio.

6. The image processing apparatus according to claim 3, wherein the average or median of the artificial noise is 0.

7. The image processing apparatus according to any one of claims 1 to 6, further comprising: A transformation unit is configured to perform transformation processing on a radiographic image of an object to be inspected, so as to stabilize the variance of the noise in the radiographic image of the object, which follows a Poisson distribution. in: The transformation unit is configured to perform transformation processing on the first radiation image; The generation unit is configured to generate a second radiation image based on the first radiation image that has undergone transformation processing; and The transformation unit is configured to perform an inverse transformation process on the second radiation image.

8. The image processing apparatus of claim 7, wherein the training data comprises a radiation image obtained by performing a transformation process on a radiation image of an object to be inspected.

9. The image processing apparatus of claim 1, wherein the training data includes data in which the third radiation image obtained by adding the first noise to the radiation image of the object to be inspected is set as input data and the radiation image of the object to be inspected is set as true value, and data in which the fourth radiation image obtained by adding the second noise to the radiation image of the object to be inspected is set as input data and the radiation image of the object to be inspected is set as true value.

10. The image processing apparatus of claim 7, wherein the training data includes data in which a radiation image obtained by performing a transformation process on the third radiation image of the object to be inspected with the first noise added is set as input data and a radiation image obtained by performing a transformation process on a radiation image of the object to be inspected is set as a true value, and data in which a radiation image obtained by performing a transformation process on the fourth radiation image of the object to be inspected with the second noise added is set as input data and a radiation image of the object to be inspected is set as a true value.

11. The image processing apparatus according to claim 1, further comprising: A segmentation unit, configured to divide a radiation image into multiple radiation images. in: The segmentation unit is configured to divide the radiographic image of the object to be inspected into a plurality of first radiographic images; The generation unit is configured to generate a plurality of second radiation images based on the plurality of first radiation images; and The segmentation unit is configured to combine the plurality of second radiation images to generate a third radiation image with reduced noise.

12. The image processing apparatus of claim 1, wherein the radiation images used in the training data comprise a plurality of radiation images obtained by dividing the radiation images of the object to be examined.

13. The image processing apparatus according to claim 1, wherein the generating unit is configured to: The first radiation image is transformed according to the ratio of the modulation transfer function of the scintillator included in the radiation detector used to obtain the first radiation image to the modulation transfer function of the scintillator included in the radiation detector used to obtain the radiation image of the object to be examined used in the training data. The learned model is used to generate the second radiation image from the transformed first radiation image; as well as The second radiation image is transformed according to the reciprocal of the ratio.

14. The image processing apparatus of claim 1, wherein the learned model comprises a neural network having a U-shaped configuration having encoder and decoder functions, and the neural network having an addition layer configured to add input data to data output from a first convolutional layer on the decoder side.

15. An image processing method, comprising: A first radiation image of the object to be inspected is obtained from a radiation detector including a scintillator; as well as A second radiation image with reduced noise compared to the first radiation image is generated by inputting the obtained first radiation image into a learned model. The learned model is obtained by training using training data, which includes a third radiation image and a fourth radiation image. The third radiation image is obtained by adding a first noise that attenuates the high-frequency components compared to the low-frequency components, and the fourth radiation image is obtained by adding a second noise that attenuates the high-frequency components more than the first noise.

16. A non-transitory computer-readable medium having a program stored thereon, which, when executed by a computer, causes the computer to perform the steps of the image processing method according to claim 15.

Images