Image processing device, radiography system, image processing method, and program

Abstract

To provide an image processing technique suitable for reducing noise in radiographic images containing signals indicating microcalcification. [Solution] The image processing device includes an inference processing unit that generates a second radiographic image with reduced noise compared to the first radiographic image by inputting a first radiographic image to a learning model obtained by learning using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and ground truth data.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a radiation imaging system, an image processing method, and a program.

Background Art

[0002] In recent years, digital radiography devices equipped with a detection unit for detecting radiation such as X-rays have been widely used in fields such as industry and medicine. In such digital radiography devices, various image processes are generally performed to improve the image quality of the captured image, and noise reduction processing for improving the granularity of the captured image and the visibility of the diagnostic region is one of them. In particular, noise reduction processing applying machine learning techniques such as deep learning may be able to greatly improve granularity. For example, in Patent Document 1, a technique has been proposed in which, by learning the characteristics of noise according to the characteristics of the imaging device, the signal of the subject structure is not degraded, and only the noise signal component is suitably removed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when the signal of the subject is a minute structure with a small contrast and a small size, a part of the subject signal may be degraded. In particular, minute calcifications in breast tissue in mammography images are about 1 mm or less in diameter, and the morphological characteristics of the signal indicating minute calcifications are similar to the morphological characteristics of the noise signal, so it is difficult to separate the signal indicating minute calcifications from the noise signal.

[0005] Furthermore, in the case of machine learning methods such as those described in Patent Document 1, it is difficult to prepare a sufficient number of cases containing microcalcification as training data. Therefore, it is also difficult to use a method that learns the characteristics of signals indicating microcalcification and the characteristics of noise signals in abundance, and generates a learning model that reduces only the noise signal component.

[0006] The present invention has been made in view of the above problems, and aims to provide an image processing technique suitable for reducing noise in radiographic images that include signals indicating microcalcification. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present invention includes an inference processing unit that generates a second radiographic image with reduced noise compared to the first radiographic image by inputting a first radiographic image to a learning model obtained by learning using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and ground truth data.

[0008] Another aspect of the present invention relates to an image processing method which includes an inference process step of inputting a first radiographic image to a learning model obtained by training using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and ground truth data, thereby generating a second radiographic image with reduced noise compared to the first radiographic image. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image processing technique suitable for reducing noise in radiographic images that include signals indicating microcalcification. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing the schematic configuration of the radiography system according to Embodiment 1. [Figure 2] A diagram showing the schematic configuration of the control unit according to Embodiment 1. [Figure 3A]A diagram showing a schematic example of the neural network model according to Embodiment 1. [Figure 3B] A diagram showing an example of a schematic configuration of a convolutional neural network that constitutes the neural network model according to Embodiment 1. [Figure 3C] A diagram illustrating the parameter update of the neural network model according to Embodiment 1. [Figure 4] A diagram showing an example of the processing flow of the inference process according to Embodiment 1. [Figure 5] A diagram showing an example of the processing flow of the learning process according to Embodiment 1. [Figure 6] A diagram showing an example of the processing flow for the particle shape signal addition process according to Embodiment 1. [Figure 7] A diagram showing an example of the noise characteristics of the detection unit according to Embodiment 1. [Figure 8] A schematic diagram of a radiographic image used to determine the amplitude reduction range of the microcalcification signal in the inference process according to Embodiment 1. [Figure 9] A figure showing an example of the amplitude reduction characteristics of the microcalcification signal according to Embodiment 1. [Figure 10] This figure shows an example of a processing flow for controlling and learning the addition position of particle shape signals according to Embodiment 2. [Modes for carrying out the invention]

[0011] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0012] In the description of the embodiment, a radiation imaging system using X-rays as an example of radiation will be described. However, the radiation may be X-rays or other types of radiation. The term "radiation" can include, for example, electromagnetic radiation such as X-rays and γ-rays, and particle radiation such as α-rays, β-rays, particle beams, proton beams, heavy ion beams, and neutron beams.

[0013] In the embodiment, a machine learning model refers to a learning model by a machine learning algorithm. Specific algorithms of machine learning include the nearest neighbor method, the naive Bayes method, decision trees, and support vector machines. In addition, neural networks or deep learning may be used. Appropriately, the applicable ones among the above algorithms can be used and applied to the following embodiments and modified examples. Also, learning data refers to a data set used for learning a machine learning model, and is composed of a pair of input data input to the machine learning model and correct answer data that is the correct answer of the output result of the machine learning model.

[0014] A trained machine learning model (hereinafter also referred to as a trained model) refers to a model obtained by performing machine learning on a machine learning model according to an arbitrary machine learning algorithm such as deep learning using appropriate learning data in advance. However, although the trained model has been obtained by learning using appropriate learning data in advance, it is not a model that does not perform further learning, and it is assumed that additional learning can be performed. The additional learning can be performed by adding machine learning even after the device is installed at the place of use.

[0015] [Embodiment 1] (Configuration of the Radiation Imaging System 10) FIG. 1 is a diagram showing a schematic configuration of a radiation imaging system 10 according to Embodiment 1. The radiation imaging system 10 includes a radiation imaging device 100 and an image processing device 110. The radiation imaging device 100 and the image processing device 110 are communicably connected. In this Embodiment 1, a mammography device will be described as the radiation imaging device 100 in the radiation imaging system 10. However, the present disclosure is not limited to this example, and the disclosed technology can be applied to any device that images using radiation transmitted through a subject, such as an X-ray CT device or an X-ray imaging device.

[0016] FIG. 1 is a block diagram showing, as a schematic configuration of the radiation imaging system 10, a configuration of the radiation imaging device 100 (mammography device) and an example of a configuration of the image processing device 110. Hereinafter, the radiation imaging system 10 according to Embodiment 1 will be described with reference to FIG. 1.

[0017] (Configuration of Mammography Device) The radiation imaging device 100 (mammography device) functions as an imaging unit that compresses the breast B of a subject and performs radiation imaging (mammography). The mammography device irradiates the breast B with radiation from one direction, detects the radiation transmitted through the breast B, and outputs a detection signal based on the detected radiation to the image processing device 110. The image processing device 110 acquires a radiation image (breast image) of the breast B by processing the detection signal of the radiation detected by the radiation imaging device 100 (mammography device).

[0018] The radiation imaging device 100 (mammography device) includes a radiation generator 101, a high voltage generation unit 102, a detection unit 103, a stage 104, a compression plate 105, a support unit 106, and a base unit 107.

[0019] The high voltage generation unit 102 generates a high voltage according to a control signal output from the control unit 111 of the image processing device 110 and applies the generated high voltage to the radiation generator 101. The radiation generator 101 generates radiation based on the high voltage applied by the high voltage generation unit 102 and irradiates the breast B placed on the stage 104 with the radiation.

[0020] The detection unit 103 is composed of, for example, a flat panel detector (FPD). The detection unit 103 has a sensor panel in which multiple pixels are arranged to form multiple rows and multiple columns. The multiple pixels of the sensor panel function as detection elements that detect radiation. The detection unit 103 is installed inside the mounting table 104 and detects radiation irradiated from the radiation generator 101 that has passed through the compression plate 105, the breast B, and the upper surface of the mounting table 104. The detection unit 103 supplies a detection signal corresponding to the detected radiation amount to the image processing device 110. In this way, the detection unit 103 outputs a detection signal corresponding to the radiation that has passed through the breast B.

[0021] The mounting platform 104 is a housing in which the detection unit 103 can be installed, and is configured so that the breast B is placed on the upper surface of the mounting platform 104. The mounting platform 104 is located below the radiation generator 101 and the compression plate 105 and is supported by the support part 106.

[0022] The compression plate 105 is formed from a material such as a resin that transmits radiation. The compression plate 105 is movably supported on the support portion 106 at a position opposite the mounting base 104. The compression plate 105 moves in a direction toward and away from the mounting base 104 (the Y direction shown in Figure 1), compressing the breast B placed on the mounting base 104. In this way, the compression plate 105 compresses the breast B between itself and the mounting base 104.

[0023] The support portion 106 supports the radiation generator 101, the compression plate 105, and the mounting base 104. The support portion 106 is rotatably supported by the base portion 107 via a shaft portion (not shown) provided between it and the base portion 107. The shaft portion is a rotatable axis on the base portion 107, and the support portion 106 is rotatable around the axis of the shaft portion (Z direction shown in Figure 1) as the center of rotation in accordance with the rotation of the shaft portion. The base portion 107 rotates the support portion 106 via the shaft portion connected to the base portion 107 in response to a rotation control signal output from the control unit 111 of the image processing device 110.

[0024] (Configuration of the image processing device 110) The image processing device 110 comprises a control unit 111, an input unit 112, and a display unit 113. The control unit 111 is connected to the radiation generator 101 via a high-voltage generation unit 102. The control unit 111 can control the high-voltage generation unit 102 to generate a high voltage, apply the high voltage to the radiation generator 101, or control the drive of the radiation generator 101 via the high-voltage generation unit 102. The control unit 111 can also rotate the support unit 106 via a shaft connected to the base unit 107. Furthermore, the control unit 111 is connected to the detection unit 103 and can perform image processing on the radiation image acquired based on the detection signal output from the detection unit 103.

[0025] The input unit 112 can be implemented by, for example, a foot switch, mouse, keyboard, trackball, switch, button, joystick, etc. The input unit 112 receives various input operations from the user and outputs an electrical signal corresponding to the received input operation to the control unit 111. The input unit 112 can receive operations such as performing radiography (mammography), moving the compression plate 105 up and down, and setting the radiography conditions.

[0026] The display unit 113 is implemented by, for example, a liquid crystal display or a CRT (Cathode Ray Tube) display. The display unit 113 displays various information under the control of the control unit 111. The display unit 113 displays, for example, radiographic images or a GUI (Graphical User Interface) that accepts various operations from users such as radiologists.

[0027] Next, with reference to Figure 2, a more specific configuration of the control unit 111 will be described. Figure 2(a) is a diagram showing the schematic configuration of the control unit 111. The control unit 111 is equipped with an acquisition unit 21, an image processing unit 22, a display control unit 23, a drive control unit 24, and a storage unit 25.

[0028] The acquisition unit 21 can acquire detection signals (radiation images) output by the detection unit 103, as well as various types of information input by the input unit 112.

[0029] The image processing unit 22 includes a noise reduction processing unit 26 and a diagnostic image processing unit 27, and can perform image processing on the detection signal (radiation image) acquired by the acquisition unit 21. In this embodiment, noise reduction processing will be described as an example of image processing by the image processing unit 22.

[0030] Figure 2(b) shows a schematic configuration of the noise reduction processing unit 26. The noise reduction processing unit 26 includes a learning processing unit 261. Functionally, the learning processing unit 261 includes, in addition to the inference processing unit 262, an image data selection unit 263, a learning data generation unit 264, a particle shape signal addition unit 265, and a parameter update unit 266. With this configuration, the noise reduction processing unit 26 can learn a machine learning model for noise reduction processing and apply noise reduction processing suitable for radiation images using the machine learning model.

[0031] The diagnostic image processing unit 27 can perform diagnostic image processing on an image that has undergone noise reduction processing by the noise reduction processing unit 26, in order to convert it into an image suitable for diagnosis. The diagnostic image processing unit 27 may perform, for example, gradation processing or enhancement processing as diagnostic image processing.

[0032] Next, the specific configuration of the learning processing unit 261 will be described. The learning processing unit 261 performs the learning processing applied when training a machine learning model, and in addition to the configuration of the inference processing unit 262, it further includes a learning data generation unit 264, a particle shape signal summing unit 265, and a parameter update unit 266.

[0033] During the learning process, images are input to the learning processing unit 261, and learning data is created by the learning data generation unit 264. Learning data is a combination of input data and correct answer data.

[0034] The image data selection unit 263 selects image data stored in the storage unit 25. Various types of image data are stored in the storage unit 25; for example, multiple radiation images of a subject may be stored, or the radiation image of the input data and the radiation image of the correct data may be stored in the storage unit 25.

[0035] The image signal from the detection unit 103 may include system noise and quantum noise in the frequency characteristics affected by the modulation transfer function characteristics. An artificial noise characteristic signal (hereinafter also referred to as artificial noise) that mimics the characteristics of these noises may be added to the image data for machine learning. For example, when performing machine learning with artificial noise added, the image data selection unit 263 can select any ground truth data from the storage unit 25. Alternatively, when performing machine learning without adding artificial noise, the image data selection unit 263 can select any pair of input data and ground truth data from the storage unit 25. The amount of noise in the input data and the ground truth data may be the same, or the amount of noise in the input data may be greater than the amount of noise in the ground truth data.

[0036] In the flowchart described later (S501 in Figure 5), an example configuration is shown in which artificial noise is added to generate input data and ground truth data. However, the configuration of this embodiment is not limited to this example. For example, when performing machine learning without adding artificial noise, the image data selection unit 263 may select N (for example, 2) radiographic images of the same subject taken N times (for example, 2 times) that are stored in the storage unit 25 in advance, each with different levels of noise, and use them as input data and ground truth data, respectively.

[0037] The learning data generation unit 264 generates pairs of input data and correct data based on the data selected by the image data selection unit 263. Details of the processing of the particle shape signal summing unit 265 of the learning data generation unit 264 will be described later.

[0038] The inference processing unit 262 has a trained inference model (machine learning model) that has trained parameters obtained based on the learning process. The inference processing unit 262 performs inference processing based on the machine learning model that is based on the trained parameters obtained by the learning process of the learning data generation unit 264. The inference processing unit 262 generates a second radiographic image with reduced noise compared to the first radiographic image by inputting the first radiographic image into the trained model obtained by training using training data obtained by adding pseudo-signals that mimic granular lesions to the input data and the ground truth data.

[0039] The parameter update unit 266 updates the parameters of the machine learning model held by the inference processing unit 262 based on the inference results from the inference processing unit 262 and the ground truth data. The parameter update unit 266 updates the parameters in a way that reduces the error between the inference results and the ground truth data. The specific processing of the parameter update unit 266 will be described later in the explanation of Figure 3C.

[0040] Furthermore, the learning processing unit 261 does not necessarily have to be included in the control unit 111. For example, the learning processing unit 261, excluding the inference processing unit 262, may be configured on hardware other than the control unit 111, such as a server, and a pre-trained machine learning model may be created by performing training processing in advance using appropriate training data. In this case, the control unit 111 may only perform inference processing using the pre-trained machine learning model by having the inference processing unit 262 access the separate hardware.

[0041] Alternatively, a pre-trained machine learning model may be provided in the noise reduction processing unit 26, and the inference processing unit 262 may use the pre-trained machine learning model provided in the noise reduction processing unit 26. Or, the learning processing unit 261 may be included in the control unit 111, so that additional learning can be performed with learning data acquired after installation.

[0042] The display control unit 23 can control the display of the display unit 113, and can display radiographic images, patient information, etc., before and after image processing by the image processing unit 22 on the display unit 113.

[0043] The drive control unit 24 can control the driving of the detection unit 103, the radiation generator 101, and the high voltage generation unit 102, etc. The control unit 111 can control the acquisition of radiation images by controlling the driving of the detection unit 103, the radiation generator 101, and the high voltage generation unit 102, etc., as controlled by the drive control unit 24.

[0044] The memory unit 25 can store programs for implementing various application software, including the operating system (OS), device drivers for peripheral devices, and programs for performing the processing described later. The memory unit 25 can also store information acquired by the acquisition unit 21 and radiation images processed by the image processing unit 22. For example, the memory unit 25 can store radiation images acquired by the acquisition unit 21, or radiation images that have undergone noise reduction processing described later.

[0045] The control unit 111 can be configured using a general-purpose computer including a processor and memory, but it may also be configured as a computer dedicated to the radiography system. Here, the control unit 111 functions as an example of an image processing device according to this embodiment, but the image processing device according to this embodiment may be a separate (external) computer that is communicatively connected to the control unit 111. Furthermore, the control unit 111 and the image processing device may be, for example, personal computers, and desktop PCs, notebook PCs, or tablet PCs (portable information terminals) may be used. The processor may be a CPU (Central Processing Unit). Alternatively, the processor may be, for example, an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or an FPGA (Field-Programmable Gate Array).

[0046] Each function of the control unit 111 may be realized by a processor such as a CPU or MPU executing software modules stored in the storage unit 25. The processor may be, for example, a GPU or FPGA. Furthermore, each function may be configured by a circuit that performs a specific function, such as an ASIC. For example, the image processing unit 22 may be realized by dedicated hardware such as an ASIC, and the display control unit 23 may be realized using a dedicated processor such as a GPU that is different from the CPU. The storage unit 25 may be configured by any storage medium such as an optical disk such as a hard disk or memory.

[0047] <Machine learning model configuration> Next, with reference to Figures 3A-3C, an example of a machine learning model that constitutes the trained model according to this embodiment will be described. An example of a machine learning model used by the inference processing unit 262 according to this embodiment is a multi-layer neural network.

[0048] Figure 3A is a diagram illustrating a schematic configuration example of a neural network model as an example of the configuration of a trained model according to this embodiment. The neural network model 33 shown in Figure 3A is designed to output noise-reduced inference data 32 (radiography image) in response to input data 31 according to pre-learned trends. The noise-reduced inference data 32 (radiography image) output from the neural network model 33 is based on the learning content in the machine learning process, and the neural network according to this embodiment learns features for distinguishing between signals and noise contained in radiography images.

[0049] Furthermore, at least a portion of the multi-layer neural network may utilize, for example, a convolutional neural network (hereinafter referred to as CNN). Additionally, at least a portion of the multi-layer neural network may employ techniques related to autoencoders.

[0050] Here, we will describe the case where a convolutional neural network (CNN) is used as a machine learning model for noise reduction processing of radiographic images. Figure 3B is a diagram showing an example of the schematic configuration of the convolutional neural network (CNN) that constitutes the neural network model 33 according to this embodiment. In the example of the trained model according to this embodiment, when a radiographic image is input as input data 31, it can output noise-reduced inference data 32 (radiographic image). In the following description, the neural network model 33 will also be referred to as the convolutional neural network model (CNN) 33.

[0051] The schematic configuration of the convolutional neural network (CNN) model 33 shown in Figure 3B consists of multiple layers responsible for processing the input data set and producing the output. The types of layers included in the CNN 33 configuration are convolutional layers, downsampling layers, upsampling layers, and merge layers. Here, the difference layer 34 is configured as a shortcut to calculate the difference between the data processed by the multiple layers and the input data before the output of the CNN. This allows the CNN to be configured to learn the difference between the input data and the output data, making it suitable for handling systems that target noise.

[0052] A convolutional layer is a layer that performs convolution on an input set of values ​​according to parameters such as the kernel size of the set filter, the number of filters, the stride value, and the dilation value. The dimensionality of the filter kernel size may also be changed depending on the dimensionality of the input image.

[0053] A downsampling layer is a layer that performs a process to reduce the number of output values ​​to less than the number of input values ​​by decimating or combining input values. Specifically, one example of such a process is Max Pooling.

[0054] An upsampling layer is a layer that performs a process to increase the number of output values ​​to the number of input values ​​by duplicating the input values ​​or adding interpolated values ​​from the input values. Specifically, one example of such a process is upsampling by deconvolution.

[0055] A synthesis layer is a layer that takes a set of values, such as the output values ​​of a certain layer or the pixel values ​​that make up an image, as input from multiple sources and performs processing to combine them by concatenating or adding them together.

[0056] Furthermore, if the parameter settings for the layers and nodes that make up the neural network differ, the degree to which the trained trends from the training data can be reproduced during inference may vary. In other words, the appropriate parameters often differ depending on the implementation, so you should change them to preferred values ​​as needed.

[0057] In addition to changing the parameters as described above, CNN33 can sometimes achieve better characteristics by changing its configuration. These better characteristics include, for example, outputting radiation images with better noise reduction, shorter processing times, and shorter training times for machine learning models.

[0058] The CNN33 used in this embodiment is a U-net type machine learning model having the functionality of an encoder consisting of multiple layers including multiple downsampling layers, and the functionality of a decoder consisting of multiple layers including multiple upsampling layers. In the U-net type machine learning model, the positional information (spatial information) obscured in the multiple layers configured as the encoder is configured to be usable in the multiple layers configured as the decoder in layers of the same dimension (layers corresponding to each other) (for example, by using skip connections). In the U-net type machine learning model, the encoder gradually reduces the input data (radiographic image) to extract features, and the decoder returns the image reduced by the encoder to its original resolution to generate the final image. In the skip connection, the feature maps obtained in each layer of the encoder are saved and combined with the upsampling of the decoder. The decoder can use the combined feature maps to generate a more detailed final image. As an example of modifying the configuration of the convolutional neural network model (CNN)33, for example, layers of activation functions (for example, ReLU: Rectifier Linear Unit: ReLU) may be incorporated before and after the convolutional layers. While the convolutional layer itself performs a linear transformation, incorporating an activation function (ReLU) before and after the convolutional layer introduces nonlinearity into the machine learning model. Furthermore, because the ReLU activates negative values ​​to zero and allows positive values ​​to pass through unchanged, it is computationally simpler than other activation functions (such as the sigmoid function), enabling faster learning and inference processes.

[0059] Through these steps in CNN33, noise features can be extracted from the input radiation image.

[0060] Figure 3C illustrates the updating of parameters of the neural network model 33 according to Embodiment 1. As shown in Figure 3C, the parameter update unit 266 of the learning processing unit 261 updates the parameters of the machine learning model (convolutional neural network model 33) of the inference processing unit 262 based on the inference data 32, which shows the inference results of the inference processing unit 262, and the ground truth data 35. Here, the inference data 32 is data showing the inference results obtained by applying the neural network model 33 of the inference processing unit 262 to the input data 31 in the learning data. The parameter update unit 266 calculates a loss function from the inference data 32 and the ground truth data 35 in the learning data, and updates the parameters of the neural network model 33 based on the loss function. Here, the loss function shows the error between the inference data 32 and the ground truth data 35.

[0061] The parameter update unit 266 is configured to update the filter coefficients of the convolutional layer, for example, using backpropagation, so that the error between the inference data 32 and the ground truth data 35, as represented by the loss function, is reduced. Here, backpropagation is a method for adjusting the parameters between each node of the neural network model 33 so that the error between the inference data 32 and the ground truth data 35 is reduced.

[0062] Furthermore, the learning process may employ a method (dropout) that randomly deactivates the units (each neuron or each node) that make up the neural network model 33. The detailed operation of the learning processing unit 261 will be described later.

[0063] The pre-trained model used by the inference processing unit 262 may be generated using transfer learning. In this case, for example, a pre-trained model used for noise reduction processing may be generated by performing transfer learning on a machine learning model trained on radiographic images of different types of objects under inspection. By performing such transfer learning, it is possible to efficiently generate pre-trained models even for objects under inspection for which it is difficult to obtain a large amount of training data. The objects under inspection of different types may be, for example, animals, plants, or objects subject to non-destructive testing.

[0064] Here, as the processor of the control unit 111, for example, a GPU can perform calculations efficiently by processing more data in parallel. For this reason, when performing training multiple times using a learning model that utilizes CNN33 as shown in Figure 3B, it is effective to perform the processing on a GPU. Therefore, the learning processing unit 261 in this embodiment may be configured to perform learning processing using a GPU in addition to a CPU as the processor configuration. Specifically, when the learning processing unit 261 executes a learning program that includes a learning model, it may perform learning processing so that the CPU and GPU cooperate in performing calculations. In addition, in the learning processing, for example, the calculations for the learning processing may be performed using only the CPU or the GPU. Furthermore, each process of the inference processing unit 262 may also be performed using a GPU in addition to the CPU, similar to the learning processing unit 261.

[0065] The above describes the configuration of a machine learning model, but it is not limited to the neural network model using CNNs exemplified above. Any machine learning-like model that can extract (represent) the features of training data such as images through learning is acceptable.

[0066] <Processing flow of inference processing unit 262> Figure 4 is a diagram showing an example of the processing flow of the inference process in the image processing unit 22 according to Embodiment 1. The inference process of the inference processing unit 262 in the image processing unit 22 will be described below using Figure 4.

[0067] In step S401, the inference processing unit 262 performs preprocessing on the radiation image acquired by the acquisition unit 21 in order to perform appropriate inference processing. The method of preprocessing is not limited, but for example, it is possible to perform a process that converts the quantum noise of radiation following a Poisson distribution to be approximately constant regardless of the intensity of the input radiation, by performing a square root transformation or logarithmic transformation, so that additive noise can be handled, or a process that sets the mean value to 0. Not limited to these, appropriate preprocessing can be performed, such as normalizing the data to 0-1, or standardizing it so that the mean value is 0 and the standard deviation is 1.

[0068] In step S402, the inference processing unit 262 performs inference processing using the neural network model 33 on the preprocessing results from step S401. The method for training the neural network model 33 will be described later. Note that the neural network model 33 can be any machine learning model that has been trained in advance, and it is not necessary to train it each time this series of image processing is performed.

[0069] In step S403, the inference processing unit 262 performs post-processing on the inference processing result from step S402. Post-processing involves the inverse processing of various transformations such as normalization and leveling performed in the pre-processing performed in step S401. As a result, a radiation image with noise reduction processing is obtained.

[0070] In step S404, the diagnostic image processing unit 27 performs the various diagnostic image processing operations described above on the radiographic image obtained in step S403 after noise reduction processing, and outputs an image suitable for diagnosis.

[0071] By processing steps S401 to S404 described above, a radiation image with reduced noise can be generated.

[0072] <Processing flow of the learning unit> Figure 5 shows an example of the processing flow of the learning process in the learning processing unit 261 according to Embodiment 1. The learning process in the learning processing unit 261 will be described below using Figure 5.

[0073] In step S501, the image data selection unit 263 and the learning data generation unit 264 generate pairs of input data and correct answer data.

[0074] First, the image data selection unit 263 selects the training data. As training data, any radiation image is selected from the image set stored in the memory unit 25.

[0075] Next, the training data generation unit 264 adds artificial noise to the radiation images selected by the image data selection unit 263, uses the radiation images with added artificial noise as input data, and generates radiation images without added artificial noise as ground truth data. The training data generation unit 264 generates pairs of input data and ground truth data as training data.

[0076] Here, the artificial noise added to the selected radiation image is determined according to the characteristics of the noise to be reduced in the noise reduction process performed during the inference process. For example, the artificial noise may be a combination of white noise corresponding to the system noise in the detection unit 103 and quantum noise with frequency characteristics affected by the modulation transfer function characteristics in the detection unit 103, and this combination may be added to the radiation image as artificial noise.

[0077] Here, as an example, we have described a configuration in which artificial noise is added to generate input data and ground truth data, but this is not the only example. For example, to generate the pair of input data and ground truth data, two radiographic images obtained by taking two radiographs of the same subject, which are stored in the memory unit 25 in advance, may be selected, and one of the radiographic images may be used as input data and the other as ground truth data to generate training data. In other words, it may be a Noise2Noise learning system.

[0078] Alternatively, two radiation images of the same subject taken at different radiation doses, which are stored in the memory unit 25 beforehand, may be selected. The radiation image with the lower radiation dose may be used as input data, and the radiation image with the higher radiation dose may be used as ground truth data to generate training data.

[0079] In step S502, the particle shape signal summarizer 265 adds a pseudo-signal (hereinafter also referred to as the particle shape signal) having a granular shape that mimics microcalcification structures in the breast as granular lesions to the same positions in the input data and ground truth data images generated in step S501. Here, the particle shape signal is a pseudo-signal that mimics a microcalcification signal showing microcalcification images that may be included in actual radiographic images. By using this training data, it is possible to learn the characteristics of noise signals that may be included in radiographic images and the characteristics of particle shape signals, separate the noise signal component from the microcalcification signal component, and generate a trained model that suitably reduces only the noise signal component. Details of the processing of the particle shape signal summarizer 265 will be described later.

[0080] In step S503, the inference processing unit 262 performs appropriate preprocessing on the input data and ground truth data generated in step S501. This preprocessing is the same as the processing in step S401 described above.

[0081] In step S504, the inference processing unit 262 performs inference processing on the input data using a machine learning model and outputs inference data to which noise reduction processing has been applied.

[0082] In step S505, the parameter update unit 266 compares the inference data with the ground truth data and calculates a loss function that quantifies the error between the two. The parameter update unit 266 also updates the parameters of the machine learning model (for example, the filter coefficients of the convolutional layer in a CNN) based on the loss function. Examples of loss functions include the mean absolute error (L1 loss) and the mean squared error (L2 loss).

[0083] In step S506, the learning processing unit 261 determines whether the learning process has ended. If it is determined to be finished (YES in S506), the flow ends; otherwise (NO in S506), it returns to S501 and repeats the processing flow from S501 to S505 using different data. Here, the criteria for determining termination could be, for example, whether a specific number of loops have been executed, or whether the loss function is below a certain level. Alternatively, it could be determined whether overfitting has occurred, or it could be used as a criterion for evaluation of commonly used evaluation metrics in machine learning, such as PSNR (Peak Signal-to-Noise Ratio) or SSIM (Structural Similarity Index Measure), which are indicators of noise reduction performance in inference images, to determine whether the performance has reached a sufficient level.

[0084] By performing the above steps S501 to S506, it is possible to generate a trained model for noise reduction processing that can suppress the reduction in the amplitude of microcalcifications and suitably reduce only the noise signal component, even for radiographic images that have microcalcifications in the breast.

[0085] In this embodiment, as an example, the processing order shown involves adding artificial noise to generate input data and ground truth data, and then adding the particle shape signal to the same position in the input data and ground truth data. However, the processing order may also be to add artificial noise to the data to which the particle shape signal has been added, thereby generating input data and ground truth data, and the processing order is not limited.

[0086] Furthermore, if the entire image cannot be processed at once due to performance limitations such as the memory capacity of the image processing unit 22, the image may be divided into small regions of appropriate size (for example, 256 x 256 pixels) for processing during the inference and training processes, and the images of the small regions may be combined after processing.

[0087] <Processing by particle shape signal summing unit 265> Figure 6 is a diagram showing an example of the processing flow of the particle shape signal addition process according to Embodiment 1. The processing of the particle shape signal addition unit 265, which is a feature of this embodiment, will be described below using Figure 6.

[0088] In step S601, the particle shape signal summing unit 265 determines the summing position of the particle shape signals in the input data and ground truth data images generated by the learning data generation unit 264.

[0089] To increase the variety of the training data, the particle shape signal may be added by appropriately changing the addition position within the training loop. When multiple training processes are performed, the particle shape signal summarization unit 265 may add a pseudo-signal by changing the coordinates of the same addition position in the input data and the ground truth data. For example, the coordinates (xp, yp) of the addition position in the input data and the ground truth data may be appropriately changed using uniformly distributed random numbers. When multiple training processes are performed, the particle shape signal is added to the position where the coordinates of the same addition position in the input data and the ground truth data have been changed in each training process.

[0090] In step S602, the particle shape signal summer 265 determines the shape of the particle shape signal. The shape of the particle shape signal can be determined using a Gaussian function, for example, as shown in equation [Equation 1].

[0091]

number

[0092] Here, in equation [Equation 1], A represents the amplitude of the particle shape signal (A>0), and σx and σy represent the particle diameters of the particle shape signal in the x and y directions in the radiation image. Here, the x direction is the first direction in the radiation image, and the y direction is the second direction intersecting the x direction (first direction). In order to ensure variations in the particle shape signal to be learned for amplitude A and particle diameter (σx, σy), the values ​​of amplitude A and particle diameter σx and σy may be appropriately changed using random numbers or the like within the learning loop. However, it is preferable to give values ​​for amplitude A and particle diameter σx and σy within an appropriate range, and the particle shape signal summing unit 265 randomly determines the amplitude A from values ​​within an appropriate amplitude range and randomly determines the values ​​of particle diameter σx and σy from values ​​within an appropriate particle diameter range. The particle shape signal is a signal with a circular or elliptical shape having a width of 1 pixel or more. If the particle diameters σx and σy are set to equal values, the particle shape signal will mimic a circular shape, and if the particle diameters σx and σy are set to different values, the particle shape signal may mimic an elliptical shape.

[0093] First, let's explain the appropriate amplitude range for amplitude A. The amplitude A of the particle shape signal needs to be set to an appropriate value relative to the amplitude of the noise signal at the summation position (xp, yp) of the particle shape signal in the input data.

[0094] If the amplitude A of the particle shape signal is smaller than a predetermined lower limit relative to the amplitude of the noise signal, it can become difficult in principle to separate the noise signal component from the particle shape signal component. If many such particle shape signals are trained, the generated trained model may behave in a way that creates a false image from the noise signal.

[0095] On the other hand, if the amplitude A of the particle shape signal is significantly larger than a predetermined upper limit relative to the amplitude of the noise signal, it may distort the signal of the subject structure, potentially preventing the proper learning of the features of the subject structure.

[0096] Based on the above, regarding the range of the amplitude A of the particle shape signal, if the amplitude of the noise signal at the addition position of the particle shape signal in the input data is N, the lower limit coefficient for setting a predetermined lower limit value with respect to the amplitude N is a, and the upper limit coefficient for setting a predetermined upper limit value with respect to the amplitude N is b, then the appropriate amplitude range of the amplitude A can be set as a×N < A < b×N (a < b).

[0097] Here, the method for calculating the amplitude N of the noise signal will be described. The amplitude N of the noise signal is obtained by calculating the standard deviation of the pixel values of the surrounding image region (small region) at the addition position (xp, yp) of the particle shape signal in the input data determined in step S601. Here, an example of using the standard deviation of the pixel values of the small region as the method for calculating the amplitude N of the noise signal has been described, but the standard deviation is just an example, and statistical information such as the average value or median value of the pixel values of the surrounding small region at the addition position (xp, yp) may also be used. The amplitude of the noise signal at the addition position is obtained using the statistical information of the pixel values at the addition position of the input data, and the amplitude of the amplitude A of the particle shape signal is obtained based on the amplitude of the noise signal.

[0098] Alternatively, the amplitude N of the noise signal may be determined based on the noise characteristics of the detection unit 103 that detects radiation. It is known that the amount of quantum noise in radiation images depends on the radiation dose. Figure 7 is a diagram showing an example of the noise characteristics of the detection unit 103, illustrating the relationship between pixel values ​​and the amplitude of the noise signal. For example, the amplitude characteristics of the noise signal with respect to the pixel values ​​of the detection unit 103 may be acquired in advance, as shown in Figure 7, and the amplitude N of the noise signal may be estimated from the pixel values ​​at the addition positions (xp, yp) of the particle shape signal based on these characteristics. The amplitude of the noise signal corresponding to the pixel values ​​at the addition positions in the input data is the noise characteristics of the detection unit 103 that detects radiation, and is acquired using the noise characteristics (Figure 7) that show the relationship between pixel values ​​and the amplitude of the noise signal, and the amplitude A of the particle shape signal may be acquired based on the amplitude N of the noise signal. For example, information indicating the noise characteristics of the detection unit 103, as shown in Figure 7, may be stored in the storage unit 25 beforehand, and the particle shape signal summing unit 265 may refer to the noise characteristics of the detection unit 103 obtained from the storage unit 25 to estimate the amplitude N of the noise signal corresponding to the pixel value at the summing position (xp, yp).

[0099] Next, using Figures 8 and 9, we will explain how to determine the lower limit coefficient a and the upper limit coefficient b used to set the amplitude range A. In the following explanation, we will use the signal showing the microcalcification pattern contained in the actual radiation image as the microcalcification signal, and the pseudo-signal that mimics the microcalcification signal as the particle shape signal, and explain them separately.

[0100] The inference processing unit 262 can determine the range of amplitude of the microcalcification signal in the inference data when it performs inference processing on the input data including the microcalcification signal, and then determine the lower limit coefficient a and upper limit coefficient b for setting the range of amplitude A of the particle shape signal to be added in the processing of step S502 based on the range of amplitude of the microcalcification signal.

[0101] In determining the lower limit coefficient a and upper limit coefficient b for setting the range of amplitude A, the trained model used in the inference process is trained without performing step S502 of the training process flow in Figure 5. In other words, a reference trained model (hereinafter referred to as the reference model) is used, which is generated by performing training using training data without adding particle shape signals.

[0102] Figure 8 shows a schematic diagram of a radiographic image used to determine the range of amplitude reduction of the microcalcification signal in the inference process. Radiographic image 801 shows a radiographic image in which multiple microcalcification signals 802 to 804 with different amplitudes are plotted. Here, radiographic image 801 may be a mammary image containing multiple cases of microcalcification, or it may be a radiographic image of a phantom having microcalcification structures of varying sizes. Radiographic image 805 shows the inference data obtained by the inference processing unit 262 performing inference processing on radiographic image 801 using a reference model. Here, microcalcification signals 806 to 808 show the microcalcification signals in the inference data (radiographic image 805). Each of the microcalcification signals 802 to 804 in radiographic image 801 corresponds to each of the microcalcification signals 806 to 808 in radiographic image 805.

[0103] Figure 9 shows an example of the amplitude reduction characteristics of the microcalcification signal according to Embodiment 1. Here, the amplitudes of the microcalcification signals 802 to 804 included in the input data (radiation image 801) are denoted as Bin, and the amplitudes of the microcalcification signals 806 to 808 at the same location in the inference data (radiation image 805), which is the output data, are denoted as Bout. When the amplitude reduction ratio Bout / Bin is plotted with respect to the amplitude Bin / N of the input data for each microcalcification signal, an amplitude reduction characteristic 901 showing a change in the amplitude reduction ratio Bout / Bin as shown in Figure 9(a) is obtained.

[0104] The amplitudes (Bin) of microcalcification signals 802-804 represent the amplitudes of microcalcification signals that indicate the microcalcification patterns included in the input data of a trained model (reference model) obtained by training with training data that does not add particle shape signals. The amplitudes (Bout) of microcalcification signals 806-808 represent the microcalcification signals included in the inference data output from the trained model (reference model), and are the amplitudes of the microcalcification signals at the same position as the microcalcification signals in the input data. The amplitude A of the particle shape signal may be obtained based on the ratio of the two amplitudes (Bout / Bin).

[0105] The values ​​of the lower limit coefficient a and the upper limit coefficient b used to set the range of amplitude A are determined from the amplitude reduction characteristic 901 of the microcalcification signal shown in Figure 9(a). For example, the Bin / N value 902 at which the amplitude reduction characteristic 901 is 0.01 can be set as the lower limit coefficient a to set the lower limit of the range of amplitude A, and the Bin / N value 903 at which the amplitude reduction characteristic 901 is 0.99 can be set as the upper limit coefficient b to set the upper limit of the range of amplitude A. The amplitude range of the particle shape signal amplitude A is set using the values ​​of the lower limit coefficient a and the upper limit coefficient b determined based on the ratio of the amplitude reduction of the microcalcification signal in the input data and inference data to the amplitude of the microcalcification signal in the input data. The lower limit of the amplitude range is set based on the amplitude N of the noise signal and the lower limit coefficient a (N × a), and the upper limit of the amplitude range is set based on the amplitude N of the noise signal and the upper limit coefficient b (N × b). The amplitude A of the particle shape signal is determined within the range of the lower limit (N × a) and the upper limit (N × b).

[0106] Based on the above, the amplitude range of the microcalcification signals 802 to 804 contained in the input data (radiation image 801) that cause amplitude reduction through inference processing can be determined, and the particle shape signal summing unit 265 can then determine the lower limit coefficient a and the upper limit coefficient b used to set the range of amplitude A of the particle shape signals to be added in the processing of step S502, based on this range.

[0107] Next, a method for determining the particle diameters σx and σy of the particle shape signal will be described. For σx and σy as well, it is necessary to set an appropriate range, similar to the lower limit coefficient a and the upper limit coefficient b used for setting the range of the amplitude A.

[0108] If a particle shape signal with a particle diameter smaller than a predetermined lower limit value, such as 1 pixel, is added and learned, the generated learning model may create artifacts. Also, if a particle shape signal with a particle diameter larger than a predetermined upper limit value is added, it may interfere with the learning of the subject's structure.

[0109] Based on the above, the setting range of the particle diameter σ of the particle shape signal can be set as c < σ < d. Here, "c" indicates the lower limit value of the setting range, and "d" indicates the upper limit value of the setting range. The setting ranges c and d of the particle diameter σ may use the same values for the particle diameter σx in the x - direction and the particle diameter σy in the y - direction in the radiographic image. Note that the disclosed technology is not limited to this example, and different particle diameters may be set in each direction. The method for determining the setting ranges c and d of the particle diameter σ is the same as the method for determining the lower limit coefficient a and the upper limit coefficient b used for setting the range of the amplitude A. In the inference process using the reference model, values are set from the range of particle diameters where the amplitude decrease of the microcalcification signal occurs.

[0110] Let the particle diameter of each of the microcalcification signals 802 - 804 included in the radiographic image 801 be Rin, the amplitude of each of the microcalcification signals 802 - 804 be Bin, and the amplitude of each of the microcalcification signals 806 - 808 at the same position in the inference data (radiographic image 805) be Bout. When plotting the ratio Bout / Bin of the amplitude decrease with respect to each particle diameter Rin of the input data for each microcalcification signal, an amplitude decrease characteristic 904 as shown in FIG. 9(b) is obtained.

[0111] The values ​​of the setting ranges c and d for the particle diameter σ are determined from the amplitude reduction characteristic 904 of the microcalcification signal shown in Figure 9(b). For example, the value of Rin 905 at which the amplitude reduction characteristic 904 is 0.01 can be set as the lower limit c of the particle diameter setting range, and the value of Rin 906 at which the amplitude reduction characteristic 904 is 0.99 can be set as the upper limit d of the particle diameter setting range, thereby defining the setting ranges for the particle diameters σx and σy of the particle shape signal. The range of particle diameters for the particle shape signal is set using the lower limit c and upper limit d determined based on the ratio of the amplitude reduction of the microcalcification signal in the input data and inference data to the particle diameter of the microcalcification signal included in the input data of the trained model (reference model), and the particle diameters σx and σy of the particle shape signal are determined within the range of the lower limit c and upper limit d.

[0112] Finally, the particle shape signal summing unit 265 randomly performs affine transformations such as free rotation, inversion, and skew on the particle shape signal P(x, y) determined as described above.

[0113] In this embodiment, an example of a configuration for generating a particle shape signal using equation [Equation 1] has been described. However, it is also possible to use a method in which multiple microcalcification images, each containing only the areas in the breast where microcalcifications are depicted in the radiographic image, are stored in the storage unit 25 in advance, and the particle shape signal summing unit 265 selects an arbitrary image from the multiple microcalcification images stored in the storage unit 25. Alternatively, a method may be used in which the selected microcalcification image is subjected to modulation, scaling, or reduction of the signal value so that it falls within the above-mentioned amplitude and particle diameter range, and then added as a particle shape signal.

[0114] In step S603, the particle shape signal summarization unit 265 adds the particle shape signal P(x, y), generated based on the shape (amplitude, particle diameter) determined in step S602, to the summarization position determined in step S601. By adding the particle shape signal P(x, y) determined in step S602 to the same position in the input data and the ground truth data images, the unit generates input data with the particle shape signal P(x, y) added and the ground truth data.

[0115] In step S604, the particle shape signal summarization unit 265 determines the completion of the particle shape signal summarization process. The completion condition is that a predetermined number of particle shape signals have been added. Here, the number of particle shape signals to add is a hyperparameter set before the learning process. The ability of the generated trained model to suppress the amplitude reduction of microcalcifications can be adjusted by the number of particle shape signals to add, and can be adjusted to an appropriate value as needed in the learning system. Microcalcifications in the breast in mammography images are often present in small regions (e.g., 256 × 256 pixels) obtained by dividing the radiographic image, with only a few present. Therefore, in this embodiment, during training, the training data is divided into small region images of 256 × 256 pixels, and N (e.g., 5) particle shape signals are added to these small region images. The particle shape signal summarization unit 265 may change the particle shape signals to be added according to the image size of the small region images obtained by dividing the radiographic image. For example, if the image size of a small-region image increases, the particle shape signal to be added may be increased, and if the image size of a small-region image decreases, the particle shape signal to be added may be decreased.

[0116] In the completion check of the addition process in step S604, if the condition is not met (NO in S604), the process returns to step S601, and the same process is repeated until a predetermined number of particle shape signals are added. On the other hand, if a predetermined number of particle shape signals are added and the condition in step S604 is met (YES in S604), this process is terminated.

[0117] Through the process described above, a trained model for noise reduction processing can be created that suppresses the amplitude reduction of the microcalcification signal and effectively reduces noise by training with input data to which particle shape signals have been added and with the correct data.

[0118] [Embodiment 2] <Processing 2 of the particle shape signal summing unit 265> In the first embodiment, process 1 describes an example where the particle shape signal addition position is determined in the input data and ground truth data using uniformly distributed random numbers. However, since microcalcifications exist only in the breast, it is desirable that the particle shape signals added during learning are also added only within the breast. In the second embodiment, process 2 by the particle shape signal addition unit 265 describes a method for determining an appropriate addition position for the particle shape signals.

[0119] Figure 10 shows an example of a processing flow for controlling and learning the addition position of particle shape signals according to Embodiment 2.

[0120] In step S1001, the particle shape signal summarization unit 265 determines the summarization position of the particle shape signals in the input data and ground truth data images generated by the learning data generation unit 264. The method for determining the summarization position is the same as in Embodiment 1, for example, the coordinates (xp, yp) of the summarization position in the input data and ground truth data are determined by uniformly distributed random numbers.

[0121] In step S1002, the particle shape signal summing unit 265 determines whether or not to add the particle shape signal to the summing position determined in step S1001. Here, as an example, we will describe a process in which the particle shape signal is added only within the subject.

[0122] The particle shape signal summing unit 265 generates a binarized image (hereinafter referred to as a mask image) in which the subject region is 1 and the background region is 0, using the input data generated by the learning data generation unit 264 or the correct data. The method for generating the mask image may be any known method, such as using a rule-based method such as Otsu's binarization to binarize the subject region, or using a machine learning model trained to segment the subject region using a U-net type model.

[0123] The particle shape signal summarization unit 265 identifies the subject area where the subject is photographed and the background area where the subject is not photographed based on the binary information set in the binarized image (mask image) generated by image processing of the input data and the ground truth data. It adds a pseudo-signal to the subject area and does not add a pseudo-signal to the background area. If the binary information of the coordinates (xp, yp) in the mask image is 0, the particle shape signal summarization unit 265 does not add a particle shape signal to the background area (outside the summarization area) (NO in S1002), and proceeds to the determination process in step S1005.

[0124] On the other hand, in the determination process of step S1002, if the binary information of the coordinates (xp, yp) in the mask image is 1, the particle shape signal summing unit 265 proceeds to step S1003, considering it to be within the subject area (within the addition area). In step S1003, the particle shape signal summing unit 265 determines the shape of the particle shape signal. In step S1004, the particle shape signal summing unit 265 adds the particle shape signal P(x, y) generated based on the shape (amplitude, particle diameter) determined in step S1003 to the addition position determined in step S1001. The processing in steps S1003 to S1005 is the same as in Embodiment 1, so a detailed explanation of the processing is omitted.

[0125] In this embodiment, an example configuration is shown in which mask images are generated as appropriate within the learning loop. However, it is also possible to generate mask images corresponding to each learning data in advance and store them in the storage unit 25, and then have the particle shape signal summing unit 265 read the mask images from the storage unit 25.

[0126] Furthermore, in this embodiment, the ability to add particle shape signals was determined based on whether or not they were within the subject area. However, a mask image may be generated to add particle shape signals to specific locations within the subject.

[0127] Through the process described above, a trained model for noise reduction can be created that suppresses the amplitude reduction of the microcalcification signal and effectively reduces noise by training with input data in which the particle shape signal is added at an appropriate summation position and with the correct data.

[0128] <Summary of Embodiments> The technologies disclosed herein include the following image processing devices, radiography systems, image processing methods, and programs. [Item 1] An image processing device characterized by comprising an inference processing unit that generates a second radiographic image with reduced noise compared to the first radiographic image by inputting a first radiographic image to a learning model obtained by training using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and ground truth data. [Item 2] The image processing apparatus according to item 1, characterized in that the amount of noise contained in the input data and the ground truth data are equivalent or the amount of noise contained in the input data is greater than the amount of noise contained in the ground truth data. [Item 3] The image processing apparatus according to item 1 or 2, characterized in that the learning model is a learning model trained using the input data to which a noise characteristic signal that mimics the characteristics of noise contained in the image signal is further added, and the ground truth data to which the noise characteristic signal is not added. [Item 4] The image processing apparatus according to any one of items 1 to 3, characterized in that the learning model is a learning model that has been trained using one of two images obtained by performing two radiographic imagings on the same subject as the input data and the other image as the ground truth data. [Item 5] The image processing apparatus according to any one of items 1 to 3, characterized in that the learning model is a learning model that has been trained using the image with the lower radiation dose as input data and the image with the higher radiation dose as ground truth data, out of two images obtained by performing two radiation scans on the same subject with different radiation doses. [Item 6] The image processing apparatus according to any one of items 1 to 5, characterized in that the pseudo-signal is added to the same addition position in the input data and the correct data. [Item 7] The image processing apparatus according to item 6, characterized in that when multiple learning processes are performed, the pseudo-signal is added to a position in which the coordinates of the same addition position in the input data and the correct data have been changed in each learning process. [Item 8] The image processing apparatus according to any one of items 1 to 7, characterized in that the pseudo-signal is a signal having a circular or elliptical shape with a width of 1 pixel or more. [Item 9] The amplitude of the noise signal at the summing position in the input data is obtained using statistical information of the pixel values ​​at the summing position. The image processing apparatus according to any one of items 1 to 8, characterized in that the amplitude of the pseudo-signal is obtained based on the amplitude of the noise signal. [Item 10] The amplitude of the noise signal corresponding to the pixel value at the addition position in the input data is obtained using the noise characteristics of the detection unit that detects radiation, which show the relationship between the pixel value and the amplitude of the noise signal. The image processing apparatus according to any one of items 1 to 8, characterized in that the amplitude of the pseudo-signal is obtained based on the amplitude of the noise signal. [Item 11] The amplitude of the pseudo-signal is, The amplitude of the microcalcification signal, which shows the microcalcification pattern included in the input data of the trained model obtained by training using training data without adding the aforementioned pseudo-signals, The image processing apparatus according to item 10, characterized in that a microcalcification signal included in the inference data output from the trained model is acquired based on the ratio of the position of the microcalcification signal in the input data to the amplitude of the microcalcification signal at the same position. [Item 12] The image processing apparatus according to item 11, characterized in that the amplitude range of the pseudo-signal is set using the values ​​of a lower limit coefficient and an upper limit coefficient determined based on the ratio of the amplitude reduction of the microcalcification signal in the input data and the inference data to the amplitude of the microcalcification signal in the input data. [Item 13] The lower limit of the amplitude range of the pseudo-signal is set based on the amplitude of the noise signal and the lower limit coefficient, and the upper limit of the amplitude range of the pseudo-signal is set based on the amplitude of the noise signal and the upper limit coefficient. The image processing apparatus according to item 12, characterized in that the amplitude of the pseudo-signal is determined within the range of the lower limit and the upper limit. [Item 14] The range of particle diameters for the pseudo-signal is set using a lower limit and an upper limit determined based on the ratio of the amplitude reduction of the microcalcification signal in the input data and the inference data to the particle diameter of the microcalcification signal included in the input data. The image processing apparatus according to item 11, characterized in that the particle size of the pseudo-signal is determined within the range of the lower limit and the upper limit. [Item 15] The image processing apparatus according to any one of items 1 to 14, characterized in that a predetermined number of the pseudo-signals are added to each small region obtained by dividing the input data and the correct data. [Item 16] Based on the binary information set in the coordinates of the binarized image generated by image processing of the input data and the ground truth data, the subject area in which the subject is photographed and the background area in which the subject is not photographed are identified. The image processing apparatus according to any one of items 1 to 15, characterized in that the pseudo-signal is added to the subject area, and the pseudo-signal is not added to the background area. [Item 17] The first and second radiographic images are mammography images of the subject's breast. The image processing apparatus according to any one of items 1 to 16, characterized in that the pseudo-signal is a signal that mimics the microcalcification structures in the breast contained in the mammography image. [Item 18] A radiography device for taking radiographic images, An image processing device described in any one of items 1 to 17, which is communicably connected to the aforementioned radiography apparatus, A radiography system characterized by having the following features. [Item 19] An image processing method characterized by having an inference processing step in which a first radiographic image is input to a learning model obtained by training using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and ground truth data, thereby generating a second radiographic image with reduced noise compared to the first radiographic image. [Item 20] A program that causes a computer to perform the image processing method described in item 19.

[0129] [Other embodiments] The disclosed technology can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0130] The disclosed technology is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to make the scope of the invention public. [Explanation of Symbols]

[0131] 103: Detection unit, 111: Control unit, 22: Image processing unit

Claims

1. An image processing device characterized by comprising an inference processing unit that generates a second radiographic image with reduced noise compared to the first radiographic image by inputting a first radiographic image to a learning model obtained by training using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and ground truth data.

2. The image processing apparatus according to claim 1, characterized in that the amount of noise contained in the input data and the ground truth data are equivalent or the amount of noise contained in the input data is greater than the amount of noise contained in the ground truth data.

3. The image processing apparatus according to claim 1, characterized in that the learning model is a learning model trained using the input data to which a noise characteristic signal that mimics the characteristics of noise contained in the image signal is further added, and the ground truth data to which the noise characteristic signal is not added.

4. The image processing apparatus according to claim 1, characterized in that the learning model is a learning model that has been trained using one of two images obtained by performing two radiographic imagings on the same subject as the input data and the other image as the ground truth data.

5. The image processing apparatus according to claim 1, characterized in that the learning model is a learning model that has been trained using the image with the lower radiation dose as input data and the image with the higher radiation dose as ground truth data, out of two images obtained by performing two radiation scans on the same subject with different radiation doses.

6. The image processing apparatus according to claim 1, characterized in that the pseudo-signal is added to the same addition position in the input data and the correct data.

7. The image processing apparatus according to claim 6, characterized in that when multiple learning processes are performed, the pseudo-signal is added to the position where the coordinates of the same addition position in the input data and the correct data have been changed in each learning process.

8. The image processing apparatus according to claim 1, characterized in that the pseudo-signal is a signal having a circular or elliptical shape with a width of 1 pixel or more.

9. The amplitude of the noise signal at the summing position in the input data is obtained using statistical information of the pixel values ​​at the summing position. The image processing apparatus according to claim 1, characterized in that the amplitude of the pseudo-signal is obtained based on the amplitude of the noise signal.

10. The amplitude of the noise signal corresponding to the pixel value at the addition position in the input data is obtained using the noise characteristics of the detection unit that detects radiation, which show the relationship between the pixel value and the amplitude of the noise signal. The image processing apparatus according to claim 1, characterized in that the amplitude of the pseudo-signal is obtained based on the amplitude of the noise signal.

11. The amplitude of the pseudo-signal is, The amplitude of the microcalcification signal, which shows the microcalcification pattern included in the input data of the trained model obtained by training using training data without adding the aforementioned pseudo-signals, The image processing apparatus according to claim 10, characterized in that a microcalcification signal included in the inference data output from the trained model is acquired based on the ratio of the amplitude of the microcalcification signal at the same position as the position of the microcalcification signal in the input data.

12. The image processing apparatus according to claim 11, characterized in that the amplitude range of the pseudo-signal is set using the values ​​of a lower limit coefficient and an upper limit coefficient determined based on the ratio of the amplitude reduction of the microcalcification signal in the input data and the inference data to the amplitude of the microcalcification signal in the input data.

13. The lower limit of the amplitude range of the pseudo-signal is set based on the amplitude of the noise signal and the lower limit coefficient, and the upper limit of the amplitude range of the pseudo-signal is set based on the amplitude of the noise signal and the upper limit coefficient. The image processing apparatus according to claim 12, characterized in that the amplitude of the pseudo-signal is determined within the range of the lower limit and the upper limit.

14. The range of particle diameters for the pseudo-signal is set using a lower limit and an upper limit determined based on the ratio of the amplitude reduction of the microcalcification signal in the input data and the inference data to the particle diameter of the microcalcification signal included in the input data. The image processing apparatus according to claim 11, characterized in that the particle size of the pseudo-signal is determined within the range of the lower limit and the upper limit.

15. The image processing apparatus according to claim 1, characterized in that a predetermined number of pseudo signals are added to each small region obtained by dividing the input data and the correct data.

16. Based on the binary information set in the coordinates of the binarized image generated by image processing of the input data and the ground truth data, the subject area in which the subject is photographed and the background area in which the subject is not photographed are identified. The image processing apparatus according to claim 1, characterized in that the pseudo-signal is added to the subject area, and the pseudo-signal is not added to the background area.

17. The first and second radiographic images are mammography images of the subject's breast. The image processing apparatus according to claim 1, characterized in that the pseudo-signal is a signal that mimics the microcalcification structures in the breast contained in the mammography image.

18. A radiography device for taking radiographic images, An image processing apparatus according to any one of claims 1 to 17, which is communicably connected to the radiography apparatus, A radiography system characterized by having the following features.

19. An image processing method characterized by having an inference processing step in which a first radiographic image is input to a learning model obtained by training using learning data obtained by adding pseudo-signals that mimic granular lesions to input data and correct data, thereby generating a second radiographic image with reduced noise compared to the first radiographic image.

20. A program that causes a computer to execute the image processing method described in claim 19.

Images