Image processing apparatus

Abstract

An image processing apparatus includes: a bit length determiner configured to determine a bit length indicating a number of at least one bit among all bits of image data; a high bit extractor configured to extract at least one high bit corresponding to a bit length from the image data; and a noise processing unit configured to perform noise reduction processing for the target pixel using at least one high bit of the image data.

Description

Technical Field

[0001] The disclosed techniques and embodiments generally relate to an image processing device capable of performing noise reduction processing. Background Art

[0002] Image sensing devices are devices that capture optical images by converting light into electrical signals using photosensitive semiconductor materials that react to light. With the development of the automotive, medical, computer, and communication industries, the demand for high-performance image sensing devices is increasing in various fields such as smartphones, digital cameras, game consoles, the Internet of Things (IoT), robots, surveillance cameras, and medical micro cameras.

[0003] An optical image captured by an image sensing device may contain noise caused by various factors generated internally and externally to the image sensing device. When processing an optical image, a noise removal operation may be performed to minimize the amount of noise. In order to perform the noise removal operation, image data of neighboring pixels should be adequately stored and cross-referenced. Summary of the invention

[0004] According to an embodiment of the present disclosure, an image processing device may include: a bit length determiner, which is configured to determine a bit length indicating the number of at least one bit among all bits of image data; a high bit extractor, which is configured to extract at least one high bit corresponding to the bit length from the image data; and a noise processing unit, which is configured to perform noise reduction processing on a target pixel using at least one high bit of the image data.

[0005] According to another embodiment of the present disclosure, an image processing device may include: a bit length determiner, which is configured to determine a bit length indicating the number of at least one bit among all bits of the image data based on characteristic information of an image sensing device that generates the image data or characteristic information of the image data; and a noise processing unit, which is configured to perform noise reduction processing on a target pixel using at least one high bit corresponding to the bit length, and the at least one high bit is extracted from the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The above and other features and advantageous aspects of the disclosed technology will become readily apparent with reference to the following detailed description when considered in conjunction with the accompanying drawings.

[0007] Figure 1 is a block diagram showing an image processing apparatus according to some embodiments of the present disclosure.

[0008] Figure 2 is a schematic diagram showing a comparison result between image data and reference image data based on some embodiments of the present disclosure.

[0009] Figure 3 Some embodiments of the present disclosure are shown Figure 1 A schematic diagram of the operation method of the bit length determiner shown.

[0010] Figure 4 Some embodiments of the present disclosure are shown Figure 1 A schematic diagram of another operating method of a bit length determiner is shown.

[0011] Figure 5 Some embodiments of the present disclosure are shown Figure 1 A schematic diagram of another operating method of a bit length determiner is shown.

[0012] Figure 6 Some embodiments of the present disclosure are shown Figure 1 A schematic diagram of another operating method of a bit length determiner is shown.

[0013] Figure 7 The present invention is based on some embodiments of the present invention. Figure 1 A block diagram of a computing device corresponding to an image processing device. DETAILED DESCRIPTION

[0014] The present disclosure provides embodiments and examples of image processing devices capable of performing noise reduction processing, which can be used in a configuration that substantially solves one or more technical or engineering problems and alleviates limitations or shortcomings encountered in some other image processing devices. Some embodiments of the present disclosure relate to an image processing device capable of effectively performing noise reduction processing. Recognizing the above problems, an image processing device based on some embodiments of the present disclosure can reduce the size of image data of reference pixels to be used in noise reduction processing, can store the resulting image data, and can maximize noise reduction performance by maximizing the number of reference pixels to be stored.

[0015] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals will be used throughout the accompanying drawings to represent the same or similar parts. Although the embodiments of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof are illustrated as examples in the accompanying drawings. However, the present disclosure should not be construed as being limited to the embodiments set forth herein.

[0016] Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that the disclosed technology is not limited to a specific embodiment, but includes various modifications, equivalents and / or substitutions of the embodiments. The embodiments of the present disclosure can provide various effects that can be directly or indirectly recognized by the disclosed technology.

[0017] Various embodiments of the present disclosure are directed to an image processing apparatus capable of efficiently performing noise reduction processing.

[0018] It is to be understood that both the foregoing general description and the following detailed description of the disclosed technology are exemplary and descriptive and are intended to provide further description of embodiments of the disclosure as claimed.

[0019] Figure 1 is a block diagram showing an image processing apparatus according to some embodiments of the present disclosure.

[0020] Reference Figure 1 , the image processing apparatus 100 may perform at least one image signal processing on the image data IDATA to generate processed image data IDATA_P.

[0021] The image processing device 100 can reduce the noise of the image data IDATA, and can perform various types of image signal processing for image quality improvement of the image data (e.g., demosaicing, defective pixel correction, gamma correction, color filter array interpolation, color matrix, color correction, color enhancement, lens distortion correction, etc.). The image processing device 100 can compress the image data created by performing the image signal processing for image quality improvement, so that the image processing device 100 can create an image file using the compressed image data. Alternatively, the image processing device 100 can restore the image data from the image file. In this case, the scheme for compressing such image data can be a reversible format or an irreversible format. As a representative example of such a compression format, in the case of using a still image, a Joint Photographic Experts Group (JPEG) format, a JPEG2000 format, etc. can be used. In addition, in the case of using a moving image, a plurality of frames can be compressed according to the Moving Picture Experts Group (MPEG) standard so that a moving image file can be created.

[0022] The image data IDATA may be generated by an image sensing device that captures an optical image of a scene, but the scope of the disclosed technology is not limited thereto. The image sensing device may include: a pixel array including a plurality of pixels configured to sense incident light received from a scene; a control circuit configured to control the pixel array; and a readout circuit configured to output digital image data IDATA by converting an analog pixel signal received from the pixel array into digital image data IDATA. In some embodiments of the present disclosure, the image data IDATA may be generated by an image sensing device.

[0023] The pixel array may include a color filter array (CFA) in which color filters are arranged according to a predetermined pattern (e.g., a Bayer pattern, a quaternary Bayer pattern, a nine-element Bayer pattern, an RGBW pattern, etc.) so that each color filter can sense light of a predetermined wavelength band. The pattern of the image data IDATA may be determined according to the type of the pattern of the CFA.

[0024] The image processing device 100 may be a computing device mounted on a chip independent of a chip on which an image sensing device is mounted. The chip on which the image sensing device is mounted and the chip on which the image processing device 100 is mounted may communicate with each other through a predetermined interface. In some embodiments, the chip on which the image sensing device is mounted and the chip on which the computing device is mounted may be implemented in one package (e.g., a multi-chip package (MCP)), but the scope of the disclosed technology is not limited thereto.

[0025] The image processing device 100 may include a noise reduction unit 200 that performs a noise reduction process to reduce a noise component included in the image data IDATA. The image processing device 100 may include: a processor configured to process data and / or instructions required to perform the function of the noise reduction unit 200; and a memory configured to store data and / or instructions required to perform the operation of the processor. In some embodiments, the image processing device 100 may perform at least one image signal process on the image data IDATA received from the image sensing device to generate processed image data IDATA_P. In addition, the image processing device 100 may store the processed image data IDATA_P in the memory, or may output the processed image data IDATA_P to an external device (e.g., an application processor, a flash memory, a display, etc.).

[0026] The noise component included in the image data IDATA may include fixed pattern noise and / or temporal noise.

[0027] Fixed pattern noise may refer to noise having a predetermined pattern due to hardware characteristics of an image sensing device (eg, lens shading).

[0028] Temporal noise may refer to noise that occurs randomly without a predetermined pattern, and may include shot noise, dark current noise, and the like.

[0029] The noise reduction unit 200 may reduce the noise component included in the image data IDATA using the image data of at least one reference pixel associated with the target pixel subjected to noise processing. Here, the reference pixel may be a neighboring pixel physically close to the target pixel, or may be a pixel belonging to the same row or column as the target pixel. In some embodiments, the reference pixel may be a pixel included in a kernel (e.g., a 5×5 kernel) including the target pixel located at the center of the kernel.

[0030] The noise reduction unit 200 may include a bit length determiner 210 , a high bit extractor 220 , a reference data storage unit 230 , and a noise processing unit 240 .

[0031] The bit length determiner 210 may determine a bit length (BL) indicating the number of at least one bit to be used for a noise reduction process of a target pixel among all bits of the image data IDATA corresponding to each pixel.

[0032] Reference Figure 2 , the image data IDATA corresponding to each pixel may include a total of (N+M) bits, wherein each of N and M is an integer of 1 or greater. That is, the image data IDATA may include N high bits and M low bits. Each high bit may represent a bit indicating a relatively high digit of the image data IDATA. Therefore, the high bit may include a most significant bit (MSB). Each low bit may represent a bit indicating a relatively low digit of the image data (IDATA). Therefore, the low bit may include a least significant bit (LSB).

[0033] The bit length BL may represent the number of bits to be used for noise processing. In one embodiment, the bit length BL may mean N bits, which is the number of bits of the high bits in the image data. That is, only the reference image data IDATA_R corresponding to the high bits in the image data IDATA may be used for noise processing, and the low bits may not be used for noise processing.

[0034] like Figure 2 As shown, the size of the image data IDATA corresponding to a total of 25 pixels of the (5×5) kernel can be represented by "(N+M)×25" bits, while the size of the reference image data IDATA_R corresponding to a total of 25 pixels of the (5×5) kernel can be reduced to (N×25) bits.

[0035] For example, when the image data IDATA corresponding to one pixel includes 10 bits and the reference data storage unit 230 has a storage capacity of 240 bits, in principle, the reference data storage unit 230 can store only the image data of the pixels corresponding to the (5×5) kernel including the target pixel (i.e., 24 pixels excluding the target pixel from 25 pixels). However, when the bit length BL is set to 5, the reference data storage unit 230 can store only the image data including five bits per pixel, so that the image data of the pixels corresponding to the (7×7) kernel including the target pixel (i.e., 48 pixels excluding the target pixel from 49 pixels) can be stored. Therefore, the number of reference pixels that can be stored in the reference data storage unit 230 and used for the noise reduction processing of the target pixel can be increased.

[0036] That is, the bit length determiner 210 may determine the bit length BL, which is the number of bits to be used for noise processing among all bits of the image data IDATA corresponding to each pixel, so that the size of the per-pixel image data required for such noise processing can be reduced. The bit length determiner 210 may determine the bit length BL using at least one of characteristic information of an image sensing device configured to generate image data (e.g., gain, exposure time, etc. of the image sensing device) or other characteristic information of the image data (e.g., noise value of the image data, frame difference of the image data, etc.).

[0037] Will refer to it later Figures 3 to 6 The embodiment of determining the bit length BL by the bit length determiner 210 is described. Figures 3 to 6 The shown embodiments are not mutually exclusive, and at least two or more embodiments may be combined with each other.

[0038] The upper bit extractor 220 may extract at least one upper bit corresponding to the bit length BL from the image data IDATA based on the bit length BL provided from the bit length determiner 210. For example, when the image data IDATA corresponding to each pixel consists of 10 bits and the bit length BL is 5, the upper bit extractor 220 may extract five upper bits from the image data IDATA, and may output the reference image data IDATA_R including the five upper bits.

[0039] The reference data storage unit 230 may store the reference image data IDATA_R, and may provide the reference image data IDATA_R upon receiving a request from the noise processing unit 240. In addition, the reference data storage unit 230 may delete some data from the currently stored reference image data IDATA_R under the control of the noise processing unit 240. For example, the reference data storage unit 230 may store the reference image data IDATA_R of a plurality of reference pixels required for noise processing of a target pixel. When such noise processing of a target pixel is completed, the noise processing unit 240 may control the reference data storage unit 230 to delete the reference image data IDATA_R of at least one reference pixel not required for noise processing of a next target pixel. Therefore, the reference data storage unit 230 may store the reference image data IDATA_R of at least one reference pixel required for noise processing of a next target pixel. The purpose of the above operation is to enable the reference image data IDATA_R of the reference pixels required for noise processing to be stored even if the reference data storage unit 230 has a limited storage capacity.

[0040] The reference data storage unit 230 may refer to a memory device having a predetermined storage capacity. For example, the memory device may refer to a volatile memory device (eg, dynamic random access memory DRAM) or a nonvolatile memory device (eg, flash memory) capable of temporarily storing data.

[0041] The noise processing unit 240 may perform noise reduction processing to reduce noise components included in the image data IDATA of the target pixel. The noise components may include fixed pattern noise and / or temporal noise. The noise reduction processing may be performed using statistical information of the image, pre-learned dictionary information, a neural network, etc.

[0042] In some embodiments, the noise processing unit 240 may perform filtering using at least one of a Gaussian filter, an ε filter, a median filter, an average filter, a non-local mean (NLM) filter, a block matching and 3D (BM3D) filter, an edge reconstruction filter, an error compensation filter, a bilateral filter, a trilateral filter, a joint color filter, a joint bilateral filter, and an offset compensation filter, and thus may perform noise reduction processing based on the result of such filtering.

[0043] In other embodiments, the noise processing unit 240 may use a denoising convolutional neural network (DnCNN) to perform noise reduction processing.

[0044] The noise processing unit 240 may perform noise reduction processing of the target pixel by referring to the reference image data IDATA_R of at least one reference pixel stored in the reference data storage unit 230. At this time, as the number of reference pixels used for the noise reduction processing increases, the noise reduction performance may increase. However, as the number of reference pixels used for the noise reduction processing of the target pixel increases, the storage capacity required for the reference data storage unit 230 and the power consumed by driving the reference data storage unit 230 may increase. Therefore, since the number of reference pixels is inevitably limited to a certain level or less, as many reference pixels as possible must be stored in the reference data storage unit 230 within the allowed storage capacity.

[0045] In some embodiments, the size of the image data of the reference pixels used in the noise reduction process is reduced, and then the resulting image data of the reference pixels is stored, so that the noise reduction performance can be maximized by increasing the number of reference pixels to be stored as much as possible. In addition, by reducing the size of the image data of each reference pixel, the number of calculations required for the noise reduction process required per reference pixel can be reduced, resulting in an increase in data processing speed.

[0046] Figure 3 Some embodiments of the present disclosure are shown Figure 1 A schematic diagram of the operation method of the bit length determiner 210 is shown.

[0047] Reference Figure 3 , the bit length determiner 210a is Figure 1 An example of a bit length determiner 210 is shown. The bit length determiner 210a may determine the bit length based on a gain of an image sensing device. The gain may be received from an external device (eg, an image sensing device).

[0048] In some embodiments, the gain may be a conversion gain of a pixel circuit corresponding to a target pixel, may be an analog gain of an analog-to-digital converter (ADC) corresponding to a target pixel, or may be the product of a conversion gain and an analog gain. The pixel circuit and the ADC may be included in an image sensing device. Here, the conversion gain may refer to the ratio of the amount of photocharge (i.e., e) to the voltage (i.e., mV) of an electrical signal generated in response to the amount of photocharge (e.g., represented by "mV / e"). The amount of photocharge may be generated by sensing the incident light of the pixel circuit. The analog gain may refer to the ratio of the voltage (i.e., mV) of the electrical signal received from the pixel circuit to the digital value (i.e., DN) obtained by conversion of the voltage of the electrical signal in the ADC (e.g., represented by "DN / mV").

[0049] The bit length determiner 210a may store a first lookup table 300 in which the gain and the bit length are matched to each other. Figure 3 As shown, the first lookup table 300 may refer to a table in which first to W-th gains G1 to Gw (where “W” is an integer of 2 or greater) are matched with first to W-th bit lengths BL1 to BLw, respectively.

[0050] In the first lookup table 300, as the gain increases, the bit length corresponding to the gain may decrease. This is because the number of bits greatly affected by the noise component of the image data increases in proportion to the increased gain.

[0051] In some embodiments, the first lookup table 300 may be generated by experimentally determining a specific gain and a bit length suitable for the gain. For example, the noise standard deviation of the image data generated by the image sensing device having a specific gain may be calculated. In addition, the reduced bits required to represent the numerical values ​​within multiple ranges (e.g., 1 / 10 to 1 times) of the noise standard deviation may be calculated. In one embodiment, the reduced bits may be subtracted from the total number of bits of the image data IDATA so that the number of high bits may be calculated. The number of high bits may be greatly affected by the noise component of the image data. Here, the noise standard deviation may be obtained by calculating the standard deviation of the image data generated in a state where the light incident on the image sensing device is completely blocked during the test process of the image sensing device.

[0052] For example, when the first gain G1 corresponds to the reference gain, the second gain G2 is twice the first gain G1, the third gain G3 is four times the first gain G1, and the fourth gain G4 is 8 times the first gain G1, the first bit length BL1 can be represented by 10 bits, the second bit length BL2 can be represented by 9 bits, the third bit length BL3 can be represented by 8 bits, and the fourth bit length BL4 can be represented by 7 bits.

[0053] The bit length determiner 210a may determine a bit length BL matching (or most similar to) a gain provided from the outside by referring to the first lookup table 300. In addition, the bit length determiner 210a may provide the determined bit length BL to the high bit extractor 220.

[0054] Figure 4 Some embodiments of the present disclosure are shown Figure 1 FIG. 2 is a schematic diagram of another operation method of the bit length determiner 210 shown in FIG.

[0055] Reference Figure 4 , the bit length determiner 210b may be Figure 1 Another example of the bit length determiner 210 is shown. The bit length determiner 210b may determine the bit length based on an exposure time provided from the outside (eg, an image sensing device).

[0056] The exposure time may refer to the exposure time of the pixel circuit corresponding to the target pixel of the image sensing device. Here, the exposure time may mean the time taken for the pixel circuit to detect incident light (ie, the time taken to generate photocharges corresponding to the incident light).

[0057] The bit length determiner 210b may store a second lookup table 400 in which the exposure time and the bit length are matched to each other. Figure 4 As shown, the second lookup table 400 may be a table that matches the first to X-th exposure times T1 to Tx (where “X” is an integer of 2 or greater) with the first to X-th bit lengths BL1 to BLx, respectively.

[0058] In the second lookup table 400, as the exposure time increases, the bit length corresponding to the exposure time may decrease. This is because the number of bits greatly affected by the noise component of the image data increases in proportion to the increased exposure time.

[0059] In some embodiments, the second lookup table 400 may be generated by experimentally determining a specific exposure time and a bit length suitable for the exposure time. For example, the noise standard deviation of the image data generated by the image sensing device having a specific exposure time may be calculated. In addition, the reduced bits required to represent the numerical values ​​within multiple ranges (e.g., 1 / 10 to 1 times) of the noise standard deviation may be calculated. In one embodiment, the reduced bits may be subtracted from the total number of bits of the image data IDATA so that the number of high bits may be calculated. The number of high bits may be greatly affected by the noise component of the image data. Here, the noise standard deviation may be obtained by calculating the standard deviation of the image data generated in a state where the light incident on the image sensing device is completely blocked during the test process of the image sensing device.

[0060] For example, when the first exposure time T1 corresponds to the reference exposure time, the second exposure time T2 is 1.2 times the first exposure time T1, the third exposure time T3 is 1.4 times the first exposure time T1, and the fourth exposure time T4 is 1.6 times the first exposure time T1, the first bit length BL1 can be represented by 10 bits, the second bit length BL2 can be represented by 9 bits, the third bit length BL3 can be represented by 8 bits, and the fourth bit length BL4 can be represented by 7 bits.

[0061] The bit length determiner 210b may determine a bit length BL matching (or most similar to) the exposure time provided from the outside by referring to the second lookup table 400. In addition, the bit length determiner 210b may provide the determined bit length BL to the high bit extractor 220.

[0062] Figure 5 Some embodiments of the present disclosure are shown Figure 1 FIG. 2 is a schematic diagram of another operation method of the bit length determiner 210 shown in FIG.

[0063] Reference Figure 5 , the bit length determiner 210c may be Figure 1 Another example of the bit length determiner 210 is shown. The bit length determiner 210c may calculate a noise value of the image data IDATA based on the image data IDATA, and may determine a bit length BL corresponding to the calculated noise value.

[0064] The pixel array of the image sensing device may include active pixels and optical black pixels. The active pixels may detect incident light and generate an electrical signal corresponding to the intensity of the incident light. The optical black pixels may block the incident light and generate an electrical signal corresponding to a dark current (regardless of the incident light). Here, the dark current is a current generated in the optical black pixels when the incident light is blocked. Therefore, the image data IDATA may include image data generated by the active pixels and image data generated by the optical black pixels.

[0065] In some embodiments, the bit length determiner 210c may calculate a specific noise value of image data generated by an optical black pixel corresponding to a target pixel (eg, belonging to the same row as the target pixel), and may determine a bit length BL corresponding to the calculated noise value.

[0066] In this case, the noise value may be a standard deviation of image data generated by an optical black pixel corresponding to the target pixel.

[0067] As an example, a rounding bit reduction method may be used to determine a bit length BL corresponding to a calculated noise value. The rounding bit reduction method may refer to a method for determining a specific value obtained by subtracting the reduced bits from the total number of bits of the image data IDATA. Here, the reduced bits indicate a value obtained by multiplying the standard deviation of the image data generated by the optical black pixel by a predetermined reduction ratio. The reduction ratio may refer to a ratio of the relationship between the standard deviation of the image data and the number of low bits that are not affected by the noise component of the image data, which is determined experimentally. When the standard deviation of the image data generated by the optical black pixel is multiplied by a predetermined reduction ratio (e.g., 3 / threshold), the reduced bits corresponding to the number of low bits that are not affected by the noise component of the image data may be calculated in response to the resulting standard deviation. Subsequently, when the reduced bits are subtracted from the total number of bits of the image data IDATA, the number of high bits that are greatly affected by the noise component of the image data may be calculated.

[0068] As another example, the third lookup table 500 may be used to determine the bit length BL corresponding to the calculated noise value, wherein the pre-stored noise value and the bit length match each other. Figure 5 As shown, the third lookup table 500 may refer to a table matching the first noise value N1 to the Yth noise value Ny (where "Y" is an integer of 2 or more) with the first bit length BL1 to the Yth bit length BLy, respectively. The third lookup table 500 may be stored in the bit length determiner 210c.

[0069] In the third lookup table 500, as the noise value increases, the bit length corresponding to the noise value may increase. This is because the number of bits greatly affected by the noise component of the image data increases in proportion to the increased noise value.

[0070] For example, when the first noise value N1 corresponds to the reference noise value, the second noise value N2 is twice the first noise value N1, the third noise value N3 is three times the first noise value N1, and the fourth noise value N4 is four times the first noise value N1, the first bit length BL1 can be represented by 10 bits, the second bit length BL2 can be represented by 9 bits, the third bit length BL3 can be represented by 8 bits, and the fourth bit length BL4 can be represented by 7 bits.

[0071] The bit length determiner 210c may determine a bit length BL matching (or most similar to) the noise value of the image data IDATA by referring to the third lookup table 500. In addition, the bit length determiner 210c may provide the determined bit length BL to the high bit extractor 220.

[0072] Figure 6 Some embodiments of the present disclosure are shown Figure 1 FIG. 2 is a schematic diagram of another operation method of the bit length determiner 210 shown in FIG.

[0073] Reference Figure 6 , the bit length determiner 210d may be Figure 1 Another example of the bit length determiner 210 is shown. The bit length determiner 210d may calculate a frame difference based on the image data IDATA of the current frame and the image data IDATA_PF of the previous frame, and may determine a bit length BL corresponding to the calculated frame difference.

[0074] The image sensing device may sequentially generate image data IDATA in units of frames (i.e., on a frame basis), may generate image data IDATA_PF of a previous frame, and may generate image data IDATA of a current frame. Therefore, the image processing device 100 may sequentially perform image processing of the image data IDATA_PF of the previous frame and image processing of the image data IDATA_PF of the previous frame. In this case, even if the image processing of the image data IDATA_PF of the previous frame is completed, the image data IDATA_PF of the previous frame may be stored and maintained in an external memory (e.g., a frame memory) accessible to the image processing device 100.

[0075] The bit length determiner 210d may compare the image data IDATA of the current frame corresponding to a target kernel (e.g., a (5×5) kernel) having a predetermined size with respect to the target pixel with the image data (IDATA_PF) of the previous frame corresponding to the target kernel, and may calculate a frame difference based on a result of such comparison. In more detail, the expression “corresponding to the target kernel” may mean the expression “corresponding to each pixel included in the target kernel”.

[0076] In some embodiments, the frame difference may be obtained by: i) calculating an absolute value (or square value) of a value obtained by subtracting image data IDATA of a current frame from image data IDATA_PF of a previous frame for each pixel included in a target kernel including the target pixel; and ii) performing summation of the absolute values ​​calculated for the pixels included in the target kernel, respectively. The frame difference may refer to a value obtained by performing summation of the absolute values ​​(or square values) calculated for the pixels included in the target kernel, respectively.

[0077] The bit length determiner 210d may use the fourth lookup table 600 to determine the bit length BL corresponding to the calculated frame difference, wherein the frame difference and the bit length match each other. Figure 6 As shown, the fourth lookup table 600 may refer to a table matching the first frame difference F1 to the Zth frame difference Fz (where Z is an integer of 2 or greater) with the first bit length BL1 to the Zth bit length BLz, respectively, and may be stored in the bit length determiner 210d.

[0078] In the fourth lookup table 600, as the frame difference decreases, the bit length corresponding to the frame difference may decrease. This is because the number of bits greatly affected by the noise component of the image data increases as the frame difference decreases.

[0079] For example, when the first frame difference F1 corresponds to the reference frame difference, the second frame difference F2 is 0.9 times the first frame difference F1, the third frame difference F3 is 0.8 times the first frame difference F1, and the fourth frame difference F4 is 0.7 times the first frame difference F1, the first bit length BL1 can be represented by 10 bits, the second bit length BL2 can be represented by 9 bits, the third bit length BL3 can be represented by 8 bits, and the fourth bit length BL4 can be represented by 7 bits.

[0080] The bit length determiner 210d may determine a bit length BL matching (or most similar to) the frame difference of the image data IDATA by referring to the fourth lookup table 600. In addition, the bit length determiner 210d may provide the determined bit length BL to the high bit extractor 220.

[0081] Figure 7 is shown with Figure 1 A block diagram of a computing device 1000 corresponding to the image processing device 100.

[0082] Reference Figure 7 , the computing device 1000 may represent a computer program product for executing Figure 1 An embodiment of a hardware configuration of an operation of the image processing apparatus 100.

[0083] The computing device 1000 may be mounted on a chip independent of a chip on which an image sensing device is mounted. According to one embodiment, a chip on which an image sensing device is mounted and a chip on which the computing device 1000 is mounted may be implemented in one package (e.g., a multi-chip package (MCP)), but the scope of the disclosed technology is not limited thereto.

[0084] Computing device 1000 may include a processor 1010 , a memory 1020 , an input and output (input / output) interface 1030 , and a communication interface 1040 .

[0085] Processor 1010 can process execution Figure 1The processor 1010 may be data and / or instructions required for the operation of the components (200-240) of the image processing apparatus 100 described in the specification. That is, the processor 1010 may refer to the image processing apparatus 100, but the scope of the disclosed technology is not limited thereto.

[0086] The memory 1020 may store data and / or instructions required to perform operations of the components 200 to 240 of the image processing apparatus 100, and may be accessed by the processor 1010. For example, the memory 1020 may be a volatile memory (e.g., a dynamic random access memory (DRAM), a static random access memory (SRAM), etc.) or a nonvolatile memory (e.g., a programmable read-only memory (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, etc.).

[0087] That is, a computer program for executing the operation of the image processing apparatus 100 disclosed in the present disclosure is recorded in the memory 1020 and executed and processed by the processor 1010 , thereby realizing the operation of the image processing apparatus 100 .

[0088] The input / output interface 1030 is an interface that connects an external input device (eg, keyboard, mouse, touch panel, etc.) and / or an external output device (eg, display) to the processor 1010 to allow transmission and reception of data.

[0089] The communication interface 1040 is a component that can transmit and receive various data with an external device (eg, an application processor, an external memory, etc.), and may be a device that supports wired or wireless communication.

[0090] It is obvious from the above description that the image processing device based on some embodiments of the present disclosure can reduce the size of image data of reference pixels to be used in noise reduction processing, can store the resulting image data, and can maximize the noise reduction performance by maximizing the number of reference pixels to be stored.

[0091] The embodiments of the present disclosure can provide various effects that can be recognized directly or indirectly.

[0092] Although a number of exemplary embodiments have been described, it should be understood that modifications and enhancements to the disclosed embodiments and other embodiments may be conceived based on what is described and / or shown in this disclosure. In addition, these embodiments may be combined to form additional embodiments.

[0093] CROSS-REFERENCE TO RELATED APPLICATIONS

[0094] This patent application claims priority to and the benefit of Korean Patent Application No. 10-2023-0123668, filed on September 18, 2023, the disclosure of which is incorporated herein by reference in its entirety.

Claims

1. An image processing device, comprising: a bit length determiner that determines a bit length indicating the number of at least one bit among all bits of the image data; a high bit extractor that extracts at least one high bit corresponding to the bit length from the image data; as well as A noise processing unit performs noise reduction processing on a target pixel using the at least one high bit of the image data.

2. The image processing device according to claim 1, wherein: The bit length determiner: The bit length is determined based on a gain of an image sensing device that generates the image data.

3. The image processing device according to claim 2, wherein: The gain includes a conversion gain of a pixel circuit corresponding to the target pixel, an analog gain of an analog-to-digital converter ADC corresponding to the target pixel, or a product of the conversion gain and the analog gain, and wherein the pixel circuit and the ADC are included in the image sensing device.

4. The image processing device according to claim 2, wherein: The bit length determiner: A bit length corresponding to the gain of the image sensing device is determined by referring to a first lookup table in which a plurality of gains and a plurality of bit lengths are matched with each other.

5. The image processing device according to claim 4, wherein: In the first lookup table, as the gain increases, the bit length corresponding to the gain decreases.

6. The image processing device according to claim 1, wherein: The bit length determiner: The bit length is determined based on an exposure time of an image sensing device that generates the image data.

7. The image processing device according to claim 6, wherein: The exposure time is the time it takes for a pixel circuit of the image sensing device corresponding to the target pixel to detect incident light.

8. The image processing device according to claim 6, wherein: The bit length determiner: The bit length corresponding to the exposure time is determined by referring to a second lookup table in which a plurality of exposure times and bit lengths are matched with each other.

9. The image processing apparatus according to claim 8, wherein: In the second lookup table, as the exposure time increases, the bit length corresponding to the exposure time decreases.

10. The image processing apparatus according to claim 1, wherein: The bit length determiner: calculating a noise value of the image data; and The bit length corresponding to the noise value is determined.

11. The image processing apparatus according to claim 10, wherein: The noise value includes a standard deviation of image data generated by an optically black pixel corresponding to the target pixel.

12. The image processing apparatus according to claim 10, wherein: The bit length determiner: The bit length corresponding to the noise value is determined by referring to a third lookup table in which a plurality of noise values ​​and a plurality of bit lengths are matched with each other.

13. The image processing apparatus according to claim 12, wherein: In the third lookup table, as the noise value increases, the bit length corresponding to the noise value decreases.

14. The image processing apparatus according to claim 1, wherein: The bit length determiner: calculating a frame difference between image data of a current frame and image data of a previous frame; and The bit length corresponding to the frame difference is determined.

15. The image processing apparatus according to claim 14, wherein: The frame difference is obtained by the following operations: calculating, for each pixel included in a target kernel including the target pixel, an absolute value of a value obtained by subtracting the image data of the current frame from the image data of the previous frame; and A summation is performed on the absolute values ​​respectively calculated for the pixels included in the target kernel.

16. The image processing apparatus according to claim 14, wherein: The bit length determiner: The bit length corresponding to the frame difference is determined by referring to a fourth lookup table in which a plurality of frame differences and bit lengths match each other.

17. The image processing apparatus according to claim 16, wherein: In the fourth lookup table, as the frame difference decreases, the bit length corresponding to the frame difference decreases.

18. The image processing apparatus according to claim 1, further comprising: A reference data storage unit stores reference data including the at least one upper bit and provides the reference data to the noise processing unit.

19. The image processing apparatus according to claim 1, wherein: The noise processing unit: The denoising process is performed on the target pixel using at least one filter or a denoising convolutional neural network.

20. An image processing device, comprising: a bit length determiner that determines a bit length indicating the number of at least one bit among all bits of the image data based on characteristic information of an image sensing device that generates image data or characteristic information of the image data; as well as A noise processing unit performs noise reduction processing on a target pixel using at least one high bit corresponding to the bit length, the at least one high bit being extracted from the image data.

Images