Image processing method, image processing device, and image processing program

Abstract

The present invention addresses the problem of achieving stable noise removal in an image having various luminance values. A control device 121 calculates the relationship between luminance values and a statistical value of the luminance values in an X-ray image, acquires an X-ray image to be processed, specifies, on the basis of the luminance value of each pixel and the aforementioned relationship in the X-ray image, outlier pixels that deviate from the trend of output fluctuation indicated by the statistical value by using an evaluation value related to the fluctuation in the luminance value, calculated for each pixel of the X-ray image, and processes the image by using a filter for reducing the fluctuation in the luminance value for the outlier pixels in the X-ray image.

Description

Image processing method, image processing device, and image processing program

[0001] The present invention relates to an image processing method, an image processing device, and an image processing program.

[0002] Conventionally, imaging systems have been used that perform noise reduction processing on signals output from some of the pixels in an image output from a detector having a plurality of pixels (see, for example, Patent Document 1). This imaging system extracts some of the pixels by dividing two images obtained in accordance with radiation irradiated in two different periods.

[0003] JP 2014-183475 A

[0004] The noise reduction techniques used in conventional imaging systems as described above have tended to have difficulty in stably reducing noise from pixels with large noise in images with a variety of brightness values. Therefore, there is a demand for a method for stably reducing noise in images with a variety of brightness values.

[0005] The present disclosure has been made in consideration of such problems, and aims to provide an image processing method, an image processing device, and an image processing program that can achieve stable noise removal in images with various brightness values.

[0006] An image processing method according to a first aspect of the embodiment includes a calculation step of calculating the relationship between the luminance values ​​in an image and the statistical values ​​of the luminance values; an acquisition step of acquiring an image to be processed; an identification step of identifying outlying pixels that deviate from the trend of output fluctuations indicated by the statistical values ​​using an evaluation value relating to the fluctuations in luminance values ​​calculated for each pixel of the image based on the luminance values ​​and the relationship for each pixel of the image; and a processing step of processing the image using a filter that reduces the fluctuations in luminance values ​​for the outlying pixels in the image.

[0007] Alternatively, an image processing device according to a second aspect of the embodiment includes a processor that calculates the relationship between the luminance values ​​in an image and the statistical values ​​of the luminance values, acquires the image to be processed, identifies outlying pixels that deviate from the trend of output fluctuations indicated by the statistical values ​​using an evaluation value relating to the fluctuations in luminance values ​​calculated for each pixel of the image based on the luminance values ​​and the relationship for each pixel of the image, and processes the image using a filter that reduces the fluctuations in luminance values ​​for the outlying pixels in the image.

[0008] Alternatively, an image processing program according to a third aspect of the embodiment causes a processor to perform the following steps: a calculation step of calculating the relationship between the luminance values ​​in an image and the statistical values ​​of the luminance values; an acquisition step of acquiring an image to be processed; an identification step of identifying outlying pixels that deviate from the trend of output fluctuations indicated by the statistical values ​​using an evaluation value related to the fluctuations in luminance values ​​calculated for each pixel of the image based on the luminance values ​​and the relationship for each pixel of the image; and a processing step of processing the image using a filter that reduces the fluctuations in luminance values ​​for the outlying pixels in the image.

[0009] According to any one of the first to third aspects, an evaluation value calculated for each pixel of an image is used to identify outlier pixels that deviate from the trend of output fluctuations indicated by the statistical values ​​calculated for the luminance values ​​of each pixel, and a process for reducing the luminance value fluctuations is performed on the identified outlier pixels. This allows noise removal processing to be performed on pixels with statistically different output fluctuation behaviors among pixels with various luminance values. As a result, stable noise removal can be achieved for images with various luminance values.

[0010] According to any aspect of the present embodiment, it is possible to achieve stable noise removal in images having various brightness values.

[0011] FIG. 1 is a configuration diagram of an X-ray imaging system 100, which is an image processing apparatus according to a first embodiment. FIG. 2 is a diagram illustrating the hardware configuration of a control device 121 in FIG. 1. FIG. 3 is a block diagram illustrating the functional configuration of the control device 121 in FIG. 1. FIG. 4 is a flowchart illustrating the procedure for noise removal processing performed by the control device 121 in FIG. 1. FIG. 5 is a diagram illustrating an example of an X-ray image before and after noise removal processing performed by the control device 121. FIG. 6 is a block diagram illustrating the functional configuration of a control device 121A according to a second embodiment. FIG. 7 is a graph illustrating the calculation function of the z-average log-likelihood MLH(1,1) performed by the pixel specifying unit 204A in FIG. 6. FIG. 8 is a graph illustrating the calculation function of the z-average log-likelihood MLH(1,1) performed by the pixel specifying unit 204A in FIG. 6. FIG. 9 is a graph illustrating the threshold setting process performed by the pixel specifying unit 204A in FIG. 6. FIG. 10 is a flowchart illustrating the procedure for noise removal processing performed by the control device 121A in FIG. 6.

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, the same or corresponding parts are designated by the same reference numerals, and duplicated explanations will be omitted. [First Embodiment]

[0013] 1 is a configuration diagram of an X-ray imaging system 100, which is an image processing apparatus according to the first embodiment. The X-ray imaging system 100 of this embodiment includes a solid-state imaging device and an X-ray generator, and captures an image of X-rays output from the X-ray generator and transmitted through an object to be imaged using the solid-state imaging device, which can be used to inspect the object.

[0014] In the X-ray imaging system 100 shown in the figure, an X-ray generator 106 generates X-rays toward a subject (object to be imaged). The irradiation field of the X-rays generated by the X-ray generator 106 is controlled by a primary slit plate 106b. The X-ray generator 106 has a built-in X-ray tube, and the X-ray irradiation amount to the subject is controlled by adjusting conditions such as the tube voltage, tube current, and power-on time of the X-ray tube. The X-ray imager (detector) 107 has a built-in CMOS solid-state imager with multiple pixels arranged two-dimensionally, and captures (detects) an X-ray image (image) that has passed through the subject. A secondary slit plate 107a is provided in front of the X-ray imager 107 to limit the X-ray incidence area.

[0015] The rotating arm 104 holds the X-ray generator 106 and the X-ray imager 107 so that they face each other, and rotates them around the subject during panoramic tomography. A slide mechanism 113 is also provided for linearly displacing the X-ray imager 107 relative to the subject during linear tomography. The rotating arm 104 is driven by an arm motor 110 that constitutes a rotary table, and the angle of rotation is detected by an angle sensor 112. The arm motor 110 is mounted on a movable part of an XY table 114, and the center of rotation can be adjusted freely within a horizontal plane.

[0016] The image signal output from the X-ray imager 107 is first captured by a control device 121 and then stored in a frame memory 122. A tomographic image along an arbitrary tomographic plane is reconstructed from the image data stored in the frame memory 122 by predetermined arithmetic processing. The reconstructed tomographic image is output to a video memory 124, converted into an analog signal by a DA converter 125, and then displayed on an image display unit 126 such as a CRT (cathode ray tube) for use in various diagnoses.

[0017] A work memory 123 required for signal processing is connected to the control device 121, and further connected to an operation panel 119 having panel switches, an X-ray exposure switch, etc. The control device 121 is also connected to a motor drive circuit 111 that drives the arm motor 110, slit control circuits 115 and 116 that control the opening ranges of the primary slit plate 106b and the secondary slit plate 107a, and an X-ray control circuit 118 that controls the X-ray generator 106, and further outputs a signal for driving the X-ray imager 107.

[0018] The X-ray control circuit 118 is capable of feedback-controlling the amount of X-ray irradiation onto the subject based on a signal captured by the X-ray imager 107 .

[0019] In the X-ray imaging system 100 configured as described above, for example, a flat panel sensor including a solid-state imaging element including a plurality of pixels two-dimensionally arranged on a light-receiving surface and a scintillator covering the light-receiving surface of the solid-state imaging element is used as the X-ray imager 107. In this X-ray imaging system 100, the control device 121 performs a predetermined noise removal process on image data before reconstructing a tomographic image, and then outputs the tomographic image reconstructed using the image data to the video memory 124.

[0020] Solid-state imaging devices used in flat panel sensors using LTPS (Low-temperature polysilicon) have excellent on-resistance (mobility) and can achieve fast response times. However, they are known to contain blinker pixels, which exhibit larger output fluctuations than normal pixels, among the multiple pixels arranged in a two-dimensional array. The output fluctuation behavior of blinker pixels can be divided into two types: one in which the signal output suddenly fluctuates significantly and then quickly returns to normal, and one in which the signal output fluctuates significantly for a certain period (several to several tens of frames) and then returns to normal. In addition, even common CMOS flat panel sensors other than LTPS (Low-temperature polysilicon) exhibit a noise called RTN (random telegraph noise), which fluctuates irregularly over time.

[0021] The following describes the function related to noise removal processing (noise removal function) of the control device 121. Fig. 2 is a block diagram showing the hardware configuration of the control device 121, and Fig. 3 is a block diagram showing the functional configuration for realizing the noise removal function in the control device 121.

[0022] 2 , the control device 121 is physically a computer or the like including processors such as a CPU (Central Processing Unit) 131 and a GPU (Graphic Processing Unit) 105, storage media such as a RAM (Random Access Memory) 132 and a ROM (Read Only Memory) 133, a communication module 134, and an input / output module 136, all of which are electrically connected to one another. The control device 121 may include input / output devices such as a display, a keyboard, a mouse, a touch panel display, or a data recording device such as a hard disk drive or semiconductor memory. The control device 121 may also be composed of multiple computers.

[0023] As shown in FIG. 3 , the control device 121 includes, as functional components related to the noise removal function, an image acquisition unit 201, a relational calculation unit 202, a noise map generation unit 203, a pixel identification unit 204, and a processing unit 205. The functional units of the control device 121 shown in FIG. 3 are realized by loading a program (the image processing program of this embodiment) onto hardware such as the CPU 131, GPU 135, and RAM 132, thereby operating the communication module 134 and input / output module 136 under the control of the CPU 131 and GPU 135 and reading and writing data from and to the RAM 132. The CPU 131 and GPU 135 of the control device 121 execute the computer program to cause the control device 121 to function as the functional units of FIG. 3 and sequentially execute processes corresponding to the image processing method described below. The CPU 131 and GPU 135 may be standalone hardware, or either one of them may be used. The CPU 131 and GPU 135 may be implemented in programmable logic such as an FPGA, like a software processor. The RAM and ROM may be standalone hardware or may be built into programmable logic such as an FPGA. All of the various data required to execute this computer program and all of the various data generated by the execution of this computer program are stored in built-in memory such as the ROM 133 and RAM 132, or in storage media such as a hard disk drive or solid-state drive.

[0024] The functions of each functional unit of the control device 121 will be described in detail below.

[0025] The image acquisition unit 201 acquires image data of a two-dimensional X-ray image of a subject from the frame memory 122 ( FIG. 1 ). The image acquisition unit 201 then performs filtering processing to reduce fluctuations in luminance values ​​(hereinafter also referred to as pixel values) of all pixels included in the acquired X-ray image, thereby generating a noise-reduced image. The filtering processing performed by the image acquisition unit 201 may be processing using a median filter, bilateral filter, Gaussian filter, or NLM (Non Local Means) filter, wavelet threshold processing, or noise reduction processing using AI.

[0026] The relationship calculation unit 202 calculates the relationship between the luminance values ​​in the noise-reduced image acquired by the image acquisition unit 201 and the statistical value of the luminance values ​​as follows. In this embodiment, a noise evaluation value that evaluates the spread of noise values, i.e., a noise sigma value that is the standard deviation of pixel values, is used as the statistical value of pixel values. Alternatively, the variance of pixel values ​​may be used as the statistical value of pixel values.

[0027] That is, the relationship calculation unit 202 calculates, by simulation, first relationship data indicating the relationship between pixel values ​​and the standard deviation of pixel values. The relationship between pixel values ​​and the standard deviation of pixel values ​​used in the simulation by the relationship calculation unit 202 is expressed by the following equations (1) to (4).

[0028] In the above formulas (1) to (3), the variable Noise is the standard deviation of pixel values, the variable Signal is the signal value of a pixel (pixel value), the constant F is the noise factor, and the variable E ｍ is the average energy of the X-ray, the constant M is the multiplication factor by the scintillator, and the constant coeff M is information indicating a multiplication factor adjustment coefficient for adjusting the multiplication factor of the scintillator, constant C is information indicating the coupling efficiency between the solid-state imaging element and the scintillator in the X-ray imaging device 107, and constant Q is information indicating the quantum efficiency of the solid-state imaging element.

[0029] In the above formula (1), the constant cf is a conversion coefficient for converting pixel signal values ​​into electric charges in the solid-state imaging device, and the constant R is information indicating readout noise in the solid-state imaging device. The conversion coefficient cf and readout noise R are determined by the gain setting in the solid-state imaging device.

[0030] In the above formula (1), the constant M Ｓｉ is the multiplication factor of the solid-state image sensor (silicon) when X-rays incident on the scintillator are incident on the solid-state image sensor without being converted into visible light, and the constant rate si is the silicon direct incidence rate, which indicates the probability that X-rays incident on the scintillator will be incident on the solid-state image sensor without being converted into visible light, and is the constant C Ｓ is information indicating a shading correction value, and the constant offset is information indicating a camera offset indicating an offset value of the solid-state imaging element. In the above formula (1), the constant N is information indicating the number of sensors. The constant N may be information indicating, for example, the number of solid-state imaging elements (number of lines), or may be information indicating a binning setting in the solid-state imaging element. In the above formula (1), coeff noise is a noise adjustment coefficient for adjusting noise. In the above equations (2) and (4), the constant F ｐ is the blur coefficient, which indicates the blur, and the constant coeff Fp are pieces of information indicating a blur coefficient adjustment coefficient for adjusting a coefficient indicating blur.

[0031] Here, when using the above formulas (1) to (4), the relational calculation unit 202 substitutes the pixel value of each pixel of the noise-reduced image generated by the image acquisition unit 201 for the variable Signal, and sets the other parameters in the above formulas (1) to (4) based on information input by the user accepted via the operation panel 119. Then, the relational calculation unit 202 acquires the variable Noise calculated using the above formulas (1) to (4) as a numerical value of the standard deviation of the pixel values ​​of each pixel of the noise-reduced image. Note that the parameters in the above formulas (1) to (4) may be set based on information accepted by the user, or may be set by reading out data stored in the RAM 132, the ROM 133, or the like within the control device 121 based on the accepted information.

[0032] For example, the relational calculation unit 202 receives the following information from the user: Measurement camera model Gain setting Number of binnings, number of lines Mean X-ray energy E ｍ ・Camera offset offset ・Silicon direct generation rate rate si Shading correction value C Ｓ ・Noise adjustment coefficient coeff noise ・Multiplication factor adjustment coefficient coeff M ・Blur coefficient adjustment coefficient coeff Fp Then, based on the received information "measurement camera model", the relation calculation unit 202 calculates the scintillator multiplication factor M, the coupling efficiency C, the blur coefficient F, etc., which are stored in association with the "measurement camera model" from the control device 121. ｐ , and quantum efficiency Q. For example, when the "measurement camera model" is "camera 1", the relation calculation unit 202 reads out the parameters such as the scintillator multiplication factor M=10, the coupling efficiency C=0.7, and the blur coefficient F ｐ = 0.9 and quantum efficiency Q = 0.9 are read out, and if the "measurement camera model" is "camera 2", the parameters of scintillator multiplication factor M = 100, coupling efficiency C = 0.8, and blur coefficient F ｐ = 0.8 and quantum efficiency Q = 0.7. Similarly, the relational calculation unit 202 reads out parameters such as the conversion coefficient cf and readout noise R stored in association with the "gain setting" from the control device 121 based on the received information "gain setting", and reads out parameters such as the number of sensors N stored in association with the "binning number, number of lines" from the control device 121 based on the received information "binning number, number of lines". Furthermore, the relational calculation unit 202 calculates other X-ray mean energy E ｍ The parameters such as the above are set based on information received from the user.

[0033] Furthermore, the relationship calculation unit 202 may derive first relationship data indicating the relationship between pixel values ​​and the standard deviation of pixel values ​​based on an image obtained by actually capturing an image. As an example, the relationship calculation unit 202 may acquire an X-ray image (test image) captured by irradiating a jig with X-rays. The jig may be a flat plate-like member or the like with a known thickness and material. The relationship calculation unit 202 may derive relationship data indicating the relationship between pixel values ​​and the standard deviation of pixel values ​​of a noise-reduced image generated from the acquired test image.

[0034] The following describes how the relationship calculation unit 202 derives first relationship data representing the relationship between pixel values ​​and the standard deviation of pixel values ​​from an X-ray image of a jig. For example, the jig may be a member whose thickness changes stepwise in one direction. First, the relationship calculation unit 202 derives noise-free pixel values ​​(hereinafter referred to as true pixel values) for each step of the jig in a noise-reduced image obtained from a test image of the jig, and derives the standard deviation of the pixel values ​​based on the true pixel values. Specifically, the relationship calculation unit 202 derives the average value of pixel values ​​at a certain step of the jig. Then, the relationship calculation unit 202 sets the derived average value of pixel values ​​as the true pixel value for that step. The relationship calculation unit 202 derives the difference between each pixel value and the true pixel value at that step as a noise value. The relationship calculation unit 202 derives the standard deviation of pixel values ​​from the derived noise value for each pixel value.

[0035] The relationship calculation unit 202 then derives first relationship data based on the relationship between the true pixel values ​​and the standard deviation of the pixel values. Specifically, the relationship calculation unit 202 derives the true pixel values ​​and the standard deviation of the pixel values ​​for each jig step. The relationship calculation unit 202 plots the derived relationship between the true pixel values ​​and the standard deviation of the pixel values ​​on a graph and draws an approximation curve, thereby deriving a relationship graph that represents the relationship between the pixel values ​​and the standard deviation of the pixel values. Note that the approximation curve may be an exponential approximation, a linear approximation, a logarithmic approximation, a polynomial approximation, a power approximation, or the like.

[0036] Instead of an X-ray image captured of a jig as the test image, the relationship calculation unit 202 may use a plurality of X-ray images captured with varying brightness in a state where no object is present. In this case, a plurality of test images obtained while varying the output intensity of the X-ray generator 106 or a plurality of test images obtained while varying the exposure time of the X-ray imager 107 are used. Then, the relationship calculation unit 202 can derive a relationship graph that represents the relationship between pixel values ​​and the standard deviation of pixel values ​​in the same manner as described above.

[0037] The noise map generation unit 203 specifies the value of the standard deviation from the pixel value of each pixel of the noise-reduced image based on the first relationship data indicating the relationship between the pixel value of each pixel of the noise-reduced image and the standard deviation of the pixel values ​​calculated by the relationship calculation unit 202. The noise map generation unit 203 generates a noise map, which is data that associates the specified standard deviation with each pixel of the noise-reduced image.

[0038] The pixel specifying unit 204 specifies, from all pixels of the X-ray image, outliers that deviate from the trend of output fluctuations indicated by the statistical values ​​of pixel values ​​specified by the first relationship data, based on the noise map generated by the noise map generating unit 203. These outliers are pixels that may have outliers that deviate from the trend of output fluctuations indicated by the statistical values ​​of pixel values ​​specified by the first relationship data, from all pixels of the X-ray image, based on the noise map. These outliers may also include the blinker pixels described above.

[0039] That is, the pixel specifying unit 204 calculates an evaluation value Th(i, j) regarding the fluctuation of the pixel value f for each pixel in the X-ray image acquired by the image acquiring unit 201 using the following formula (5): Here, in the above (5), i is an integer representing the horizontal position of the pixel to be calculated, j is an integer representing the vertical position of the pixel to be calculated, m and n are variables for defining the range of surrounding pixels, and w is a constant (integer) for defining the range of surrounding pixels. The pixel specifying unit 204 calculates the integrated value of the difference between the pixel value f(i, j) of the specific pixel and the pixel values ​​f(i+m, j+n) of the pixels surrounding the specific pixel as the evaluation value Th(i, j) of the specific pixel. By using such a calculation formula, it is possible to calculate an evaluation value that evaluates the degree to which the pixel value f of each pixel varies compared to the surrounding pixels.

[0040] The pixel specifying unit 204 then determines whether each pixel is an outlier pixel as follows: The pixel specifying unit 204 compares the evaluation value Th(i,j) of the pixel to be determined with a threshold K×σ(i,j) determined from the standard deviation σ(i,j) of each pixel read from the noise map, and determines that the pixel to be determined is an outlier pixel if the evaluation value Th(i,j) is greater than the threshold (i.e., if the evaluation value Th is an outlier). At this time, the coefficient K that determines the threshold is set to a value in the range of K=8×3 to 8×5 when the constant w=1. This determination is repeated for all pixels in the X-ray image to identify outlier pixels from all pixels.

[0041] The processing unit 205 replaces the pixel values ​​of the outlier pixels in the original X-ray image acquired by the image acquisition unit 201 with the pixel values ​​of the corresponding pixels in the noise-reduced image, for the X-ray image in which the outlier pixels have been identified. At this time, the processing unit 205 repeats the pixel value replacement for all outlier pixels identified in the X-ray image. This processing is equivalent to performing a filtering process on multiple outlier pixels in the X-ray image. The processing unit 205 also sequentially stores the filtered X-ray images in the frame memory 122 ( FIG. 1 ). The X-ray images stored in the frame memory 122 are reconstructed into tomographic images by processing by the control device 121.

[0042] Next, a description will be given of the procedure for noise removal processing on an X-ray image using the control device 121, i.e., the flow of the image processing method according to this embodiment. Fig. 4 is a flowchart showing the procedure for noise removal processing by the control device 121.

[0043] First, in the control device 121, the image acquisition unit 201 acquires an X-ray image from the frame memory 122 (step S101, acquisition step). Next, the image acquisition unit 201 applies filtering to the X-ray image to generate a noise-reduced image (step S102, processing step). Next, the relationship calculation unit 202 generates first relationship data indicating the relationship between pixel values ​​and their standard deviations, and then generates a noise map, which is data associating each pixel of the noise-reduced image with a standard deviation, based on the first relationship data (step S103: calculation step).

[0044] Next, the pixel identification unit 204 of the control device 121 calculates an evaluation value Th(i, j) for the specific pixel of the X-ray image acquired in step S101 (step S104). The pixel identification unit 204 then compares the evaluation value Th(i, j) of the specific pixel with a threshold determined from the standard deviation σ(i, j) associated with the specific pixel read from the noise map (step S105). If the comparison results in the evaluation value Th(i, j) being greater than the threshold (step S105; Yes), the specific pixel is determined to be an out-of-range pixel. On the other hand, if the evaluation value Th(i, j) is determined to be equal to or less than the threshold (step S105; No), the specific pixel is determined not to be an out-of-range pixel. The processes of steps S104 to S107 are repeated for all pixels in the X-ray image (step S108, identification step).

[0045] Furthermore, the processing unit 205 of the control device 121 replaces the pixel values ​​of the outlier pixels in the X-ray image with the pixel values ​​of the corresponding pixels in the noise-reduced image (step S109, processing step). Finally, the processing unit 205 stores the image data of the X-ray image that has been subjected to the filtering process in the frame memory 122 (step S110).

[0046] In the control device 121 according to the first embodiment described above, an evaluation value calculated for each pixel of an X-ray image is used to identify outlier pixels that deviate from the output fluctuation trend indicated by the statistical value calculated for the luminance value of each pixel, and a process for reducing the luminance value fluctuation is performed on the identified outlier pixels. This allows noise removal processing to be performed on pixels with statistically different output fluctuation behaviors among pixels with various luminance values. As a result, stable noise removal can be achieved for images with various luminance values. In particular, blinker pixels included in the solid-state imaging element of the X-ray imager 107 can be stably identified, and filtering processing is performed on only pixels corresponding to the blinker pixels, thereby preventing degradation of resolution in the X-ray image.

[0047] In the first embodiment, an out-of-range pixel is identified based on a comparison result between an evaluation value calculated based on the luminance value of each pixel in an X-ray image and a statistical value of the luminance values. Specifically, in the first embodiment, whether a specific pixel is an out-of-range pixel is determined based on a comparison result between an integrated value of the differences between the luminance value of the specific pixel and the luminance values ​​of pixels surrounding the specific pixel and the statistical value of the luminance value of the specific pixel. As a result, pixels with statistically different output fluctuation behaviors among pixels having various luminance values ​​can be identified by a simple calculation.

[0048] 5 shows an example of an X-ray image processed by the control device 121 according to the first embodiment, where (a) shows the X-ray image before noise removal processing, and (b) shows the X-ray image after noise removal processing has been applied to the X-ray image shown in (a). In these X-ray images, areas identified as outlier pixels are marked with circles. In this processing example, it was confirmed that noise was reduced only in pixel areas with noise that exhibited larger fluctuations in brightness compared to normal noise, and that degradation of resolution was also reduced. [Second Embodiment]

[0049] 6 is a block diagram showing the functional configuration of a control device 121A, which is an image processing device according to the second embodiment. The control device 121A differs from the control device 121 according to the first embodiment in the function of identifying deviating pixels by a pixel identifying unit 204A. The following description will focus on the differences from the first embodiment.

[0050] The pixel identification unit 204A processes multiple X-ray images captured consecutively over time and acquired by the image acquisition unit 201, and multiple noise maps generated by the noise map generation unit 203 for these multiple X-ray images.

[0051] Specifically, the pixel specifying unit 204A generates a plurality of noise-extracted images in which noise is extracted based on a plurality of X-ray images as follows. The pixel specifying unit 204A generates a noise-extracted image by subtracting the pixel value of each pixel of a noise-reduced image generated based on the X-ray image from the pixel value of each pixel of the X-ray image. Furthermore, the pixel specifying unit 204A calculates an evaluation value related to the probability of noise occurrence for each pixel of the plurality of noise-extracted images as follows. First, the pixel specifying unit 204A reads out the standard deviation σ(i, j, z) associated with a specific pixel in the noise-extracted image based on the pixel value Noise(i, j, z) (i is an integer representing the horizontal position of the pixel, j is an integer representing the vertical position of the pixel, and z is an integer representing the order in the time direction of the image) of the specific pixel in the noise-extracted image and a noise map generated from the X-ray image that is the source of the noise-extracted image. Then, the pixel specifying unit 204A calculates the standard deviation σ(i, j, z) associated with the specific pixel ... following equation (6): Then, the pixel specifying unit 204A similarly successively calculates a plurality of noise occurrence probabilities p(i, j, z) (z = 1, 2, ..., K) of the specific pixel based on the pixel values ​​Noise(i, j, z) (z = 1, 2, ..., K) of the specific pixel in the K noise-extracted images and the K noise maps.

[0052] Next, the pixel specifying unit 204A calculates the log likelihood LH(i, j, z) of each noise occurrence probability from the multiple noise occurrence probabilities p(i, j, z) (z=1, 2, ..., K) of the specified pixel using the following formula (7): LH(i, j, z) = -log{p(i, j, z)} (7) Then, the pixel specifying unit 204A calculates the z-averaged log likelihood MLH(i, j) by averaging the log likelihood in the time direction (z direction) using the following formula (8): This z-average log-likelihood MLH(i,j) is an evaluation value related to the probability of noise occurrence at the specific pixel. Furthermore, the pixel identification unit 204A compares the z-average log-likelihood MLH(i,j) calculated for the specific pixel with a preset threshold, and determines that the specific pixel is an outlier pixel if the z-average log-likelihood MLH(i,j) is greater than the threshold. The pixel identification unit 204 repeats the above-described determination process for each pixel in the multiple noise-extracted images.

[0053] 7 and 8 are graphs illustrating the function of calculating the z-average log-likelihood MLH(1,1) by the pixel identification unit 204A. In this manner, noise occurrence probabilities of 0.7, 0.6, 0.71, ..., 0.8 are calculated for each pixel value Noise(1,1,z) (z = 1, 2, ..., K) of a specific pixel in multiple noise-extracted images. When calculating each noise occurrence probability p(1,1,z) (z = 1, 2, ..., K), a normal distribution probability formula p(1,1,z) is used, which is set based on the standard deviation σ(1,1,z) determined from the noise map corresponding to the order of each noise-extracted image. Furthermore, based on the noise occurrence probabilities calculated in time series in this manner, the z-average log-likelihood MLH(1,1), which is an evaluation value for the noise occurrence probability of a specific pixel, is calculated.

[0054] The threshold value referenced by the pixel identification unit 204A when identifying outlier pixels may be preset or determined on an ad hoc basis. FIG. 9 is a graph illustrating the threshold setting process performed by the pixel identification unit 204A. The pixel identification unit 204A calculates the z-average log-likelihood MLH for multiple noise-extracted images based on multiple X-ray images acquired in advance, and aggregates the pixel frequency for each z-average log-likelihood MLH value. The pixel identification unit 204A then sets the log-likelihood at which the pixel frequency value falls below a given value as the threshold. In the example shown in FIG. 9, the threshold value is set to 4.6, which is greater than the peak position of the histogram.

[0055] Next, a procedure for noise removal processing on an X-ray image using the control device 121A, i.e., the flow of an image processing method according to the second embodiment, will be described. Fig. 10 is a flowchart showing the procedure for noise removal processing by the control device 121A.

[0056] First, in the control device 121A, the image acquisition unit 201 acquires multiple X-ray images captured consecutively from the frame memory 122 (step S201, acquisition step). Next, the image acquisition unit 201 performs filtering on the multiple X-ray images to generate multiple noise-reduced images (step S202, processing step). The filtering process may involve, for example, processing using a 3x3 median filter. Next, the relationship calculation unit 202 generates first relationship data indicating the relationship between pixel values ​​and their standard deviations, and generates multiple noise maps, which are data associating standard deviations with each pixel of the multiple noise-reduced images, based on the first relationship data (step S203: calculation step).

[0057] Next, the pixel identification unit 204A of the control device 121A generates multiple noise-extracted images based on the multiple X-ray images and the multiple noise-reduced images (step S204). Next, the pixel identification unit 204A calculates the z-average log-likelihood MLH for specific pixels in the multiple noise-extracted images using the multiple noise maps (step S205). The pixel identification unit 204A then compares the z-average log-likelihood MLH(i,j) of the specific pixel with a threshold (step S206). As a result of the comparison, if it is determined that the z-average log-likelihood MLH(i,j) is greater than the threshold (step S206; Yes), the specific pixel is determined to be an out-of-range pixel. On the other hand, if it is determined that the z-average log-likelihood MLH(i,j) is equal to or less than the threshold (step S206; No), the specific pixel is determined not to be an out-of-range pixel. The processing from steps S205 to S208 is repeated for all pixels in the X-ray image (step S209, identification step).

[0058] Furthermore, the processing unit 205 of the control device 121A replaces the pixel values ​​of the outlier pixels in the plurality of X-ray images with the pixel values ​​of the corresponding pixels in the noise-reduced images corresponding to the plurality of X-ray images (step S210, processing step). Finally, the processing unit 205 stores the image data of the plurality of X-ray images that have been subjected to the filtering process in the frame memory 122 (step S211).

[0059] In the control device 121A according to the second embodiment described above, an evaluation value relating to the probability is calculated from the brightness value of each pixel of the X-ray image and the statistical value of the brightness value. With this configuration, pixels with statistically different behaviors of output fluctuations can be identified with high accuracy among pixels having various brightness values.

[0060] In particular, the control device 121A acquires multiple X-ray images captured consecutively in time, calculates multiple evaluation values ​​for specific pixels in the multiple X-ray images, and determines whether the specific pixel is an outlier pixel based on the multiple evaluation values. This configuration stably identifies pixels with statistically different output fluctuation behaviors among pixels with various brightness values ​​without omission. In other words, by evaluating the behavior of pixel brightness fluctuations in the time direction, even blinker pixels that are buried in noise can be stably identified. When multiple actually captured X-ray images were processed using the control device 121A, it was confirmed that all five blinker pixels that caused artifacts in tomographic images based on X-ray images could be identified. [Modification]

[0061] Although various embodiments of the present invention have been described above, the embodiments of the present invention are not limited to the above-described embodiments.

[0062] In the control device 121 according to the first embodiment and the control device 121A according to the second embodiment described above, out-of-focus pixels including blinker pixels are identified in an X-ray image, but the processing target and the target to be identified are not limited to this. For example, in an image captured by an image sensor such as a CCD or CMOS, pixels that generate other types of noise such as burst noise may be identified.

[0063] Furthermore, the noise map generation unit 203 of the control device 121, 121A generates a noise map by identifying the standard deviation value from the pixel values ​​of each pixel of the noise-reduced image, but the noise map may also be generated by identifying the standard deviation value from the pixel values ​​of the original X-ray image.

[0064] The pixel identification unit 204A of the control device 121A according to the second embodiment calculates the log-likelihood as the evaluation value, but may use likelihood instead of the log-likelihood. Furthermore, the pixel identification unit 204A may derive the noise occurrence probability value used in calculating the likelihood using a Gaussian distribution, Poisson distribution, or t-distribution formula, or an actually measured probability distribution. Furthermore, the pixel identification unit 204A calculates the average value of the log-likelihood in the z direction as the evaluation value, but may also calculate the median, maximum value, minimum value, or the like instead of the average value.

[0065] Furthermore, the pixel specifying unit 204A of the control device 121A according to the second embodiment calculates an evaluation value for a specific pixel of a plurality of consecutively acquired X-ray images, but it may also calculate a plurality of evaluation values ​​for a plurality of pixels included in a pixel group in the same X-ray image and determine whether or not the pixel group includes an outlying pixel by using the average value of the plurality of evaluation values, etc. For example, the pixel specifying unit 204 may make a determination for a pixel group including pixels in the same column of the X-ray image, or for a pixel group including pixels in the same row of the X-ray image, or for a pixel group in any range in which an abnormality is to be determined.

[0066] Furthermore, when determining whether a pixel is an out-of-range pixel, the pixel specifying unit 204A may determine whether the evaluation value is greater than a threshold value, or may determine whether the judgment value is less than a threshold value, or whether the judgment value is within a range between two threshold values. Furthermore, the threshold value for determining whether a pixel is an out-of-range pixel may be changed for each pixel position, taking into consideration that there are pixel positions where blinker pixels are likely to appear.

[0067] In the first aspect, in the identifying step, the outlier pixels may be identified according to a comparison result between an evaluation value calculated based on the luminance value of each pixel of the image and a statistical value of the luminance values. In the second aspect, the processor may identify the outlier pixels according to a comparison result between an evaluation value calculated based on the luminance value of each pixel of the image and a statistical value of the luminance values. In this way, pixels having various luminance values ​​that have statistically different behaviors of output fluctuations are identified by a simple calculation.

[0068] In the first aspect, in the identifying step, the processor may determine whether the specific pixel is an out-of-range pixel based on a comparison result between an integrated value of differences between the luminance value of the specific pixel in the image and the luminance values ​​of pixels surrounding the specific pixel and a statistical value of the luminance value of the specific pixel. In the second aspect, the processor may determine whether the specific pixel is an out-of-range pixel based on a comparison result between an integrated value of differences between the luminance value of the specific pixel in the image and the luminance values ​​of pixels surrounding the specific pixel and a statistical value of the luminance value of the specific pixel. According to the above configuration, pixels having various luminance values ​​that have statistically different behaviors of output fluctuations are identified by a simple calculation.

[0069] In the first aspect, the identifying step may calculate an evaluation value related to the probability from the luminance value of each pixel of the image and a statistical value of the luminance values. Also, in the second aspect, the processor may calculate an evaluation value related to the probability from the luminance value of each pixel of the image and a statistical value of the luminance values. According to the above configuration, pixels having various luminance values ​​that have statistically different behaviors of output fluctuations are identified with high accuracy.

[0070] Furthermore, in the first aspect, the acquiring step may acquire a plurality of images successively in time, and the identifying step may calculate a plurality of evaluation values ​​for a specific pixel in the plurality of images and determine whether or not the specific pixel is an out-of-range pixel based on the plurality of evaluation values. Furthermore, in the second aspect, the processor may acquire a plurality of images successively in time, calculate a plurality of evaluation values ​​for a specific pixel in the plurality of images, and determine whether or not the specific pixel is an out-of-range pixel based on the plurality of evaluation values. With this configuration, pixels having various luminance values ​​that have statistically different output fluctuation behaviors can be stably identified without omission.

[0071] In the first aspect, the identifying step may calculate multiple evaluation values ​​for multiple pixels included in a pixel group in the image, and determine whether the pixel group includes an out-of-range pixel based on the multiple evaluation values. In the second aspect, the processor may calculate multiple evaluation values ​​for multiple pixels included in a pixel group in the image, and determine whether the pixel group includes an out-of-range pixel based on the multiple evaluation values. In this case, pixel groups with statistically different output fluctuation behaviors are identified. As a result, stable noise removal can be achieved for images in which output fluctuations are likely to occur on a pixel group-by-pixel basis.

[0072] In the first aspect or the second aspect, the statistical value may be a standard deviation or a variance. In the second aspect, the method may further include a detector that detects an image of the object.

[0073] The image processing method of the embodiment is [1] "an image processing method comprising: a calculation step of calculating a relationship between a luminance value in an image and a statistical value of the luminance value; an acquisition step of acquiring an image to be processed; an identification step of identifying outlier pixels that deviate from the trend of output fluctuations indicated by the statistical value, using an evaluation value related to the fluctuation in luminance value calculated for each pixel of the image based on the luminance value of each pixel of the image and the relationship; and a processing step of processing the image using a filter that reduces the fluctuation in luminance value for the outlier pixels in the image."

[0074] The image processing method of the embodiment may be [2] "the image processing method described in [1] above, in which, in the identification step, the outlying pixels are identified based on the comparison result between the evaluation value calculated based on the luminance value of each pixel of the image and the statistical value of the luminance values."

[0075] The image processing method of the embodiment may be [3] "the image processing method described in the above [2], in which in the identification step, it is determined whether or not the specific pixel is an outlier pixel depending on the result of comparing the integrated value of the difference between the luminance value of the specific pixel in the image and the luminance values ​​of pixels surrounding the specific pixel with the statistical value of the luminance value of the specific pixel."

[0076] The image processing method of the embodiment may be [4] "the image processing method described in [1] above, in which, in the identification step, the evaluation value related to probability is calculated from the brightness value of each pixel of the image and a statistical value of the brightness value."

[0077] The image processing method of the embodiment may be [5] "the image processing method described in [4] above, in which, in the acquisition step, a plurality of the images are acquired consecutively in time, and in the identification step, a plurality of the evaluation values ​​are calculated for specific pixels in the plurality of the images, and whether or not the specific pixel is an out-of-range pixel is determined based on the plurality of the evaluation values."

[0078] The image processing method of the embodiment may be [6] "the image processing method described in [4] or [5] above, in which, in the identification step, a plurality of evaluation values ​​are calculated for a plurality of pixels included in a pixel group in the image, and whether or not the pixel group includes an out-of-range pixel is determined based on the plurality of evaluation values."

[0079] The image processing method of the embodiment may be [7] "the image processing method according to any one of the above [1] to [6], wherein the statistical value is a standard deviation or a variance."

[0080] The image processing device of the embodiment is [8] "an image processing device comprising a processor, which calculates the relationship between the luminance values ​​in an image and the statistical values ​​of the luminance values, acquires an image to be processed, identifies outlying pixels that deviate from the trend of output fluctuations indicated by the statistical values ​​using an evaluation value regarding the fluctuations in luminance values ​​calculated for each pixel of the image based on the luminance values ​​of each pixel of the image and the relationship, and processes the image using a filter that reduces the fluctuations in luminance values ​​for the outlying pixels in the image."

[0081] The image processing device of the embodiment may be [9] "the image processing device described in [8] above, in which the processor identifies the outlying pixels based on the comparison result between the evaluation value calculated based on the brightness value of each pixel of the image and the statistical value of the brightness value."

[0082] The image processing device of the embodiment may be

[10] "the image processing device described in [9] above, in which the processor determines whether the specific pixel is an outlier pixel based on a comparison result between an accumulated value of the difference between the luminance value of the specific pixel in the image and the luminance values ​​of pixels surrounding the specific pixel and a statistical value of the luminance value of the specific pixel."

[0083] The image processing device of the embodiment may be

[11] "an image processing device described in [8] above, in which the processor calculates the evaluation value related to probability from the brightness value of each pixel of the image and a statistical value of the brightness value."

[0084] The image processing device of the embodiment may be

[12] "the image processing device described in

[11] above, in which the processor acquires a plurality of the images successively in time, calculates a plurality of the evaluation values ​​for specific pixels in the plurality of the images, and determines whether the specific pixel is an off-target pixel based on the plurality of the evaluation values."

[0085] The image processing device of the embodiment may be

[13] "an image processing device as described in

[11] or

[12] above, in which the processor calculates a plurality of evaluation values ​​for a plurality of pixels included in a pixel group in the image, and determines whether the pixel group includes an outlier pixel based on the plurality of evaluation values."

[0086] The image processing device of the embodiment may be

[14] "the image processing device according to any one of [8] to

[13] above, wherein the statistical value is a standard deviation or a variance."

[0087] The image processing device of the embodiment may be

[15] "an image processing device according to any one of [8] to

[14] above, further comprising a detector for detecting the image of an object."

[0088] 100...X-ray imaging system, 131...CPU (processor), 135...GPU (processor), 107...X-ray imaging device (detector), 106...X-ray generator, 121, 121A...controller, 201...image acquisition unit, 204, 204A...pixel identification unit, 202...relationship calculation unit, 203...noise map generation unit, 205...processing unit.

Claims

1. A calculation step to calculate the relationship between the brightness value in the image and the statistical value of the brightness value, The acquisition step involves obtaining the image to be processed, A selection step to identify outlier pixels that deviate from the trend of output fluctuations shown by the statistical value, using evaluation values ​​related to the variation in brightness values ​​calculated for each pixel of the image based on the brightness value of each pixel of the image and the relationship, A processing step of processing the image by using a filter that reduces fluctuations in brightness values ​​for the outlier pixels in the image, An image processing method comprising:

2. In the aforementioned specific step, the outlier pixels are identified according to the result of comparing the evaluation value calculated based on the brightness value of each pixel in the image with the statistical value of the brightness value. The image processing method according to claim 1.

3. In the aforementioned specific step, it is determined whether the specific pixel is an outlier pixel based on the result of comparing the integrated difference between the brightness value of the specific pixel in the image and the brightness values ​​of the surrounding pixels of the specific pixel with the statistical value of the brightness value of the specific pixel. The image processing method according to claim 2.

4. In the aforementioned specific step, the evaluation value relating to probability is calculated from the brightness value of each pixel of the image and the statistical value of said brightness value. The image processing method according to claim 1.

5. In the acquisition step, multiple images are acquired sequentially in time. In the specified step, a plurality of evaluation values ​​are calculated for a specific pixel in a plurality of images, and based on the plurality of evaluation values, it is determined whether or not the specific pixel is an outlier pixel. The image processing method according to claim 4.

6. In the aforementioned specific step, a plurality of evaluation values ​​are calculated for a plurality of pixels included in the pixel group in the image, and it is determined whether or not the pixel group includes an outlier pixel based on the plurality of evaluation values. The image processing method according to claim 4 or 5.

7. The aforementioned statistical value is the standard deviation or variance. The image processing method according to any one of claims 1 to 5.

8. Equipped with a processor, The aforementioned processor, The relationship between the brightness value in the image and the statistical value of the brightness value is calculated. Obtain the image to be processed, Based on the brightness value of each pixel in the aforementioned image and the relationship described above, an outlier pixel that deviates from the trend of output fluctuations shown in the statistical value is identified using an evaluation value regarding the variation in brightness value calculated for each pixel in the aforementioned image. The image is processed using a filter that reduces fluctuations in brightness values ​​for the outlier pixels in the image. Image processing device.

9. The processor identifies the outlier pixels according to the result of comparing the evaluation value calculated based on the brightness value of each pixel in the image with the statistical value of the brightness value. The image processing apparatus according to claim 8.

10. The processor determines whether a specific pixel is an outlier pixel based on the result of comparing the integrated difference between the brightness value of a specific pixel in the image and the brightness values ​​of pixels surrounding the specific pixel with a statistical value of the brightness value of the specific pixel. The image processing apparatus according to claim 9.

11. The processor calculates the evaluation value relating to probability from the brightness value of each pixel of the image and the statistical value of said brightness value. The image processing apparatus according to claim 8.

12. The aforementioned processor, Multiple images are acquired sequentially over time. A plurality of evaluation values ​​are calculated for a specific pixel in a plurality of images, and based on the plurality of evaluation values, it is determined whether or not the specific pixel is an outlier pixel. The image processing apparatus according to claim 11.

13. The processor calculates a plurality of evaluation values ​​for a plurality of pixels included in the pixel group in the image, and determines whether or not the pixel group includes an outlier pixel based on the plurality of evaluation values. The image processing apparatus according to claim 11 or 12.

14. The aforementioned statistical value is the standard deviation or variance. The image processing apparatus according to any one of claims 8 to 12.

15. The detector further comprises a detector that detects the image of an object. The image processing apparatus according to any one of claims 8 to 12.

16. In the processor, A calculation step to calculate the relationship between the brightness value in the image and the statistical value of the brightness value, The acquisition step involves obtaining the image to be processed, A selection step to identify outlier pixels that deviate from the trend of output fluctuations shown by the statistical value, using evaluation values ​​related to the variation in brightness values ​​calculated for each pixel of the image based on the brightness value of each pixel of the image and the relationship, A processing step of processing the image by using a filter that reduces fluctuations in brightness values ​​for the outlier pixels in the image, An image processing program that processes images.