Image signal processor

Abstract

An image signal processor includes: a resolution converter configured to generate at least two images based on image data including color pixel values and white pixel values; an element processor configured to extract texture information using the converted color pixel values and the converted white pixel values included in each of the at least two images, and generate at least two processed images based on the extracted texture information; a synthesizer configured to generate a composite chroma image and a composite white image by synthesizing the at least two processed images; and an image acquisition circuit configured to generate an output image based on the image data, the composite chroma image, and the composite white image.

Description

Technical Field

[0001] The techniques and embodiments disclosed in this patent document generally relate to an image signal processor capable of processing an image signal. Background Art

[0002] Recently, as the number of functions of electronic devices has rapidly increased, the need for improving image capture functions using these electronic devices has also increased. Therefore, there is a need for technology for improving the quality of images obtained by electronic devices.

[0003] The image sensor used in the imaging device has a configuration in which color filters that transmit red (R) light, green (G) light, and blue (B) light of specific wavelength bands for each pixel are attached to the surface of the sensor. Although there are various types of color arrays for color filters, Bayer arrays have been widely used, each of which includes three types of filters that transmit only light of specific wavelengths of red, green, and blue.

[0004] Recently, as the size of pixels of image sensors is miniaturized, the amount of light incident on each pixel is reduced, thereby increasing the signal-to-noise ratio (SNR), and thus the performance of the image sensor may deteriorate rapidly. In order to increase the transmittance of the filter located inside the pixel and realize a high-sensitivity pixel in response to a low-illuminance environment, the utilization of image sensors including a white (W) filter and other filters that transmit only specific wavelengths of light such as red, green, and blue (RGB) is rapidly increasing.

[0005] That is, a color pattern including a white channel having a characteristic that is relatively robust to signal noise compared to the RGB channels has been proposed. Since the white channel has been designed to receive light of a full band compared to the RGB channels that receive light of a specific wavelength band, the signal-to-noise ratio (SNR) is relatively low even when acquiring an image in a low illumination environment.

[0006] However, the RGBW arrangement is a single-plate type image sensor in which R, G, B, and W filters are arranged in a mosaic pattern. Therefore, when generating a color image, a heavy mosaic process is required as a color encoding for generating red, green, and blue (RGB) pixel values ​​corresponding to each pixel.

[0007] However, in an RGBW arrangement including white pixels, the sampling rates of the R, G, and B components may be reduced compared to the RGB arrangement. As a result, when data for acquiring the RGBW arrangement is used in the process of generating a color image, false colors may appear. Therefore, white pixels are discarded in the re-mosaic processing stage, and the discarded white pixels cannot be used in the subsequent stage. Summary of the invention

[0008] According to an embodiment of the disclosed technology, an image signal processor may include: a resolution converter configured to generate at least two images based on image data including color pixel values ​​and white pixel values; an element processor configured to extract texture information using converted color pixel values ​​and converted white pixel values ​​included in each of the at least two images, and generate at least two processed images based on the extracted texture information; a synthesizer configured to generate a composite chrominance image and a composite white image by synthesizing the at least two processed images; and an image acquisition circuit configured to generate an output image based on the image data, the composite chrominance image, and the composite white image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features and advantageous aspects of the disclosed technology will become readily apparent with reference to the following detailed description when considered in conjunction with the accompanying drawings.

[0010] Figure 1 is a schematic diagram showing an example of an RGBW arrangement based on some implementations of the disclosed technology.

[0011] Figure 2 is a schematic diagram showing an example of image data input to an image signal processor based on some embodiments of the disclosed technology.

[0012] Figure 3 is a block diagram illustrating an example of an image signal processor based on some implementations of the disclosed technology.

[0013] Figure 4 It shows that according to Figure 3 A flowchart of an example of a method for reducing noise included in image data and interpolating color difference (chrominance) of the image data according to an embodiment of the present invention.

[0014] Figure 5 Some embodiments based on the disclosed technology are shown Figure 3 A schematic diagram showing an example of a detailed configuration of an image signal processor is shown.

[0015] Figure 6 and Figure 7 Some embodiments based on the disclosed technology are shown Figure 5 A schematic diagram showing an example of the detailed configuration and operation of the resolution converter is shown.

[0016] Figure 8 and Fig. 9 Some embodiments based on the disclosed technology are shown Figure 5 A schematic diagram of an example of the detailed structure of an element processor is shown.

[0017] Fig.10Some embodiments based on the disclosed technology are shown Figure 5 A schematic diagram showing an example of a detailed configuration of a synthesizer is shown.

[0018] Fig.11 Some embodiments based on the disclosed technology are shown Figure 5 A schematic diagram of an example of an image acquisition circuit is shown.

[0019] Fig. 12A and Fig. 12B Some embodiments based on the disclosed technology are shown Fig.11 A graph showing an example of the operation of a chrominance signal interpolator is shown. DETAILED DESCRIPTION

[0020] It will be understood that both the above general description and the following detailed description of the disclosed technology are exemplary and illustrative, and are intended to provide further description of the claimed disclosure. It will be understood that although the terms "first", "second", "third", etc. are used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element, and are not intended to imply the order or quantity of the elements. Therefore, without departing from the teachings of the present disclosure, the first element in some embodiments may be referred to as the second element in other embodiments.

[0021] This patent document provides embodiments and examples of image signal processors for processing image signals, which can be used in a configuration that substantially solves one or more technical or engineering problems and alleviates limitations or shortcomings encountered in some image signal processors in the art. Some embodiments of the disclosed technology relate to an image signal processor that can improve the quality of an image by utilizing white pixels in a pattern including white pixels in a noise reduction process or a demosaic process. Recognizing the above problems, an image signal processor based on some embodiments of the disclosed technology can obtain an image with higher sharpness and less noise, thereby improving the quality of the image signal.

[0022] Reference will now be made in detail to some embodiments of the disclosed technology, examples of which are shown in the accompanying drawings. Whenever possible, the same reference numerals will be used throughout the accompanying drawings to represent the same or similar parts. Although the present disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown as examples in the accompanying drawings. However, the present disclosure should not be construed as being limited to the embodiments set forth herein.

[0023] Hereinafter, various embodiments will be described with reference to the accompanying drawings. However, it should be understood that the disclosed technology is not limited to a specific embodiment, but includes various modifications, equivalents and / or substitutions of the embodiments. The embodiments of the disclosed technology can provide various effects that can be directly or indirectly recognized by the disclosed technology.

[0024] Various embodiments of the disclosed technology relate to an image signal processor capable of improving image quality using white pixels among a pattern including white pixels in a noise reduction process or a demosaic process.

[0025] An image signal processor (ISP) may perform at least one image signal processing on the image data to generate processed image data. The image signal processor may reduce noise of the image data, and may perform various types of image signal processing (e.g., demosaicing, defective pixel correction, gamma correction, color filter array interpolation, color matrix, color correction, color enhancement, lens distortion correction, etc.) for image quality improvement of the image data.

[0026] In addition, in an embodiment, the image signal processor may compress image data created by performing image signal processing for image quality improvement, so that the image signal processor can create an image file using the compressed image data. Alternatively, the image signal processor may restore the image data from the image file. In this case, the scheme for compressing such image data may be a reversible format or an irreversible format. As a representative example of such a compression format, in the case of using a still image, a Joint Photographic Experts Group (JPEG) format, a JPEG 2000 format, or the like may be used. In addition, in an embodiment, in the case of using a moving image, a plurality of frames may be compressed according to the Moving Picture Experts Group (MPEG) standard so that a moving image file may be created.

[0027] The image data may be generated by an image sensing device that captures an optical image of a scene, but the scope of the disclosed technology is not limited thereto. The image sensing device may include: a pixel array including a plurality of pixels configured to sense incident light received from a scene; a control circuit configured to control the pixel array; and a readout circuit configured to output digital image data by converting analog pixel signals received from the pixel array into digital image data. In some embodiments of the disclosed technology, it is assumed that the image data is generated by the image sensing device.

[0028] Figure 1 is a schematic diagram showing an example of an RGBW arrangement based on some implementations of the disclosed technology.

[0029] Reference Figure 1 , for convenience of description, 16 pixels arranged in a matrix including four rows and four columns are shown. For example, 16 pixels may be repeated in the row direction and the column direction while being used as the minimum unit of the pixel array (10, 20, 30), but the scope of the disclosed technology is not limited thereto.

[0030] An image sensing device based on some embodiments of the disclosed technology may include color pixels such as red (R), green (G), and blue (B) pixels, as well as white (W) pixels. Figure 1 As shown, RGB color pixels and white (W) pixels can be arranged in various forms. In some embodiments, "R" may represent a red pixel, "G" may represent a green pixel, "B" may represent a blue pixel, and "W" may represent a white pixel.

[0031] Each of the color pixels R, G, and B may be a pixel on which light of a specific wavelength is incident through a corresponding color filter. Each white pixel (W) may not have a corresponding color filter and may also be referred to as a texture pixel, a black and white pixel, or a monochrome pixel. For example, the color pixels R, G, and B may obtain a color pixel value according to incident light of a specific color (e.g., red, green, blue, etc.) through a color filter that transmits only incident light of a specific wavelength. In contrast, a white pixel (W) may obtain a white pixel value according to incident light (e.g., visible light) that is not limited to a specific color.

[0032] The pixel array 10 may include unit pixels (PX1), each of which includes pixels arranged in a (2×2) matrix, and each of which may include one color pixel (eg, R, G, or B) and three white pixels (W).

[0033] The pixel array 20 may include unit pixels (PX2), each unit pixel (PX2) includes pixels arranged in a (2×2) matrix, and each unit pixel (PX2) may include two color pixels (e.g., R, G, or B) and two white pixels (W). In the pixel array 20, the two color pixels included in each unit pixel (PX2) may be color pixels corresponding to the same color (i.e., Figure 1 G and G shown).

[0034] The pixel array 30 may include unit pixels (PX3), each of which includes pixels arranged in a (2×2) matrix, and each of which may include three color pixels (e.g., R, G, or B) and one white pixel (W). In the pixel array 30, the three color pixels included in each of the unit pixels (PX3) may be color pixels corresponding to the same color (i.e., Figure 1 G, G and G shown).

[0035] although Figure 1Three examples are shown, each of which includes color pixels (R, G, B) and white pixels (W), but the scope of the disclosed technology is not limited thereto. For example, the pixel arrays 10, 20, and 30 of the image sensor may include not only RGB pixels, but also other pixels such as magenta, yellow, cyan, and orange. As another example, the pixel arrays (10, 20, 30) may be configured so that for each unit pixel (PX1, PX2, or PX3), the color pixels are arranged at other possible positions (e.g., the lower right position) rather than the upper right position (or the upper left and lower left positions).

[0036] As another example, the pixel arrays (10, 20, 30) may also be configured so that the unit pixels (PX1, PX2, PX3) have different numbers of white pixels (W) or white pixels (W) arranged at different positions. That is, the number of white pixels and the positions of white pixels may be different from those of the unit pixels (PX1, PX2, PX3). Therefore, the pixel arrangements of the pixel arrays (10, 20, 30) are not limited to those shown, and may be configured in various forms to include white pixels (W) and color pixels (R, G, B).

[0037] Figure 2 is a schematic diagram showing an example of image data input to an image signal processor based on some embodiments of the disclosed technology.

[0038] Figure 2 FIG. 4 shows an example format of image data 40 obtained and output by a binning circuit 50 of an image signal processor. Here, the binning circuit 50 may perform binning of color pixels (R, G, B) and white pixels (W). Figure 2 It can be seen from the embodiment that the pixel array 20 among the pixel arrays (10, 20, 30) can be used to generate the image data 40. Figure 2 An example is shown in which the binning circuit 50 generates the image data 40 in the half-resolution mode, but the scope of the disclosed technology is not limited thereto.

[0039] The binning circuit 50 may output the image data 40 having a smaller number of pixels (e.g., half the number of pixels) than the total number of pixels of the pixel array 20. For each unit pixel (PX2), the binning circuit 50 may output one white pixel value 41 and one color pixel value (42, 43, or 44).

[0040] For example, the binning circuit 50 may output a white pixel value 41 obtained by adding or averaging the values ​​obtained from two white pixels (W) included in each unit pixel (PX2). For another example, the binning circuit 50 may output a red pixel value 42 obtained by adding or averaging the values ​​obtained from two color pixels (R) included in each unit pixel (PX2). The binning circuit 50 may output a green pixel value 43 obtained by adding or averaging the values ​​obtained from two color pixels (G) included in each unit pixel (PX2). The binning circuit 50 may output a blue pixel value 44 obtained by adding or averaging the values ​​obtained from two color pixels (B) included in each unit pixel (PX2).

[0041] although Figure 2 An example is shown in which the pixel array 20 includes an RGBW arrangement and generates image data 40 in half-resolution mode, but other embodiments are possible, and it should be noted that the format of image data generated by another pixel array including another arrangement may be determined differently according to the half-resolution mode or the full-resolution mode.

[0042] Figure 3 is a block diagram illustrating an example of an image signal processor (ISP) based on some implementations of the disclosed technology.

[0043] Reference Figure 3 , an image signal processor (ISP) may include a binning circuit 50 , a resolution converter 100 , an element processor 200 , a synthesizer 300 , and an image acquisition circuit 400 .

[0044] The binning circuit 50 may perform binning of color pixels (R, G, B) and white pixels (W) among image data received from an image sensor (not shown), and thus may generate image data 40 including color pixel values ​​and white pixel values. Here, Figure 3 The binning circuit 50 may correspond to Figure 2 The binning circuit 50.

[0045] The resolution converter 100 may generate two or more images having different sizes based on the image data 40. In some embodiments, the resolution converter 100 may include a first resolution converter that obtains a first image having a size smaller than the image data 40 by converting the resolution of the image data 40. In addition, the resolution converter 100 may include a second resolution converter that obtains a second image having a size smaller than the first image by converting the resolution of the first image. For example, the first image may have a size reduced by 1 / 2 compared to the image data 40, and the second image may have a size reduced by 1 / 4 compared to the image data 40.

[0046] The element processor 200 may extract texture information using the converted color pixel values ​​and the converted white pixel values ​​included in each of the two or more images. Here, texture may mean the shape or outline of an object on the image. In addition, the element processor 200 may generate two or more composite images and two or more chrominance (color difference) values ​​based on the texture information, and may perform noise reduction processing on the converted white pixel values.

[0047] In some embodiments, the element processor 200 may include a first element processor that generates a first processed image corresponding to the first image by using the first converted color pixel values ​​and the first converted white pixel values ​​included in the first image. In addition, the element processor 200 may include a second element processor that generates a second processed image corresponding to the second image by using the second converted color pixel values ​​and the second converted white pixel values ​​included in the second image.

[0048] The synthesizer 300 may generate a composite chrominance image and a composite white image by combining two or more processed images. For example, the synthesizer 300 may generate a composite chrominance image and a composite white image based on a first processed image and a second processed image.

[0049] The image acquisition circuit 400 may acquire an output image based on the composite chrominance image, the composite white image, the image processed by the element processor 200, and the white pixel values ​​41 included in the image data 40. The white pixel values ​​and chrominance values ​​that have completed the noise reduction process in the element processor 200 may be synthesized by the synthesizer 300, and the synthesized resultant values ​​may be input to the image acquisition circuit 400, so that the image acquisition circuit 400 may output an output image having less noise and more corrected chrominance than the image data 40.

[0050] Will refer to later Figure 4 Flowchart and Figures 5 to 12B Detailed methods of some embodiments are described with reference to an example in which an image signal processor (ISP) uses a resolution converter 100, an element processor 200, a synthesizer 300, and an image acquisition circuit 400 to reduce noise included in image data and interpolate chrominance (color difference).

[0051] Figure 4 It shows that according to Figure 3 A flowchart of an example of a method for reducing noise included in image data and interpolating color difference (chrominance) of the image data according to an embodiment of the present invention.

[0052] In operation S10, the binning circuit 50 may generate a pixel array including color pixel values ​​(eg, RGB pixels) and white pixels (W) by using color pixels (eg, RGB pixels) and white pixels (W) of the pixel array 20. Figure 2 42 to 44) and white pixel values ​​(e.g. Figure 2 41) of image data (for example, Figure 2 As used herein, the tilde "~" indicates a range of components. For example, "42-44" indicates Figure 2 Color pixel values ​​42, 43 and 44 are shown.

[0053] In operation S20, the image signal processor ISP (e.g., the resolution converter 100) may generate two or more images having different sizes based on the image data (e.g., 40). The resolution converter 100 may acquire a plurality of images by reducing the resolution of the image data. As the resolution converter 100 converts the resolution of the image data, each of the two or more images may include a converted color pixel value and a converted white pixel value.

[0054] In operation S30, the image signal processor ISP (eg, the element processor 200) may extract texture information using the converted color pixel values ​​and the converted white pixel values ​​included in each of the two or more images.

[0055] In operation S40, the image signal processor ISP (e.g., the element processor 200) may generate an image for synthesis based on the extracted texture information, may calculate a chrominance (color difference) value, and may perform a noise reduction process on the converted white pixel value. Thus, the element processor 200 may generate two or more processed images respectively corresponding to the two or more images.

[0056] In operation S50, the image signal processor ISP (eg, the synthesizer 300) may synthesize the two or more processed images to obtain a composite chrominance image and a composite white image.

[0057] The synthesizer 300 may generate a first processed image for a partial area having a texture higher than a certain level (e.g., an image area having an edge or a thinline), and may generate a composite chrominance image and a composite white image for another partial area (e.g., a flat image area or a gradient image area) based on a second processed image. For example, the synthesizer 300 may generate a composite chrominance image and a composite white image by performing multi-scale processing on two or more processed images. Here, the expression "multi-scale processing" may refer to an operation of selecting / processing a low-resolution processed image (i.e., a second processed image) in a flat image area without texture based on a plurality of processed images having different sizes, and may also refer to an operation of selecting / processing a high-resolution processed image (i.e., a first processed image) in a textured image area.

[0058] In operation S60, the image signal processor ISP (eg, the image acquisition circuit 400) may generate a composite chrominance image based on the composite white image and the white pixel value (eg, Figure 2 41) to acquire the output image. For example, the image acquisition circuit 400 may interpolate the chrominance signal based on the composite chrominance image, the composite white image, and the white pixel value. In addition, in an embodiment, the image acquisition circuit 400 may generate an edge enhancement signal based on the composite chrominance image, the composite white image, and the white pixel value, thereby increasing the sharpness of the over-blurred image area.

[0059] In an embodiment, an image signal processor (ISP) may obtain an output image with less noise and not excessively blurred by using white pixel values ​​and a composite image obtained based on images having different sizes. Unlike the disclosed technology, when multi-scale processing is performed on an image (e.g., a color image) that does not use white pixel values, a specific area of ​​the composite image may be excessively blurred. For example, multi-scale processing of an image may refer to an operation of selecting a non-reduced image in an area with texture (hereinafter referred to as a texture area), selecting a reduced image (or a simple enlarged image with a low resolution) in an area without texture, and synthesizing the non-reduced image and the reduced image. However, when noise is included in an image, it is determined that the image has no texture even if the image actually has texture, so that a reduced image may be selected and synthesized. That is, when noise is included in an image, it is difficult to determine whether the image has texture due to the presence of such noise, so there are some cases where excessive blurring is performed even in a texture area.

[0060] However, according to an embodiment of the disclosed technology, an image signal processor (ISP) may generate an edge enhancement signal based on the original white pixel value without noise reduction. Therefore, in an embodiment, the degree of blur included in the output image may be minimized by restoring the over-blurred area (or the unnecessarily blurred area) from the composite image.

[0061] Hereinafter, a specific example in which constituent elements included in an image signal processor (ISP) generate an output image will be described in detail.

[0062] Figure 5 Some embodiments based on the disclosed technology are shown Figure 3 Schematic diagram of an example of a detailed configuration of an image signal processor (ISP) shown.

[0063] Reference Figure 5 , the image signal processor (ISP) may receive the binned image data 40 through the binning circuit 50. Here, the image data 40 may correspond to Figure 2 The image data 40 is shown.

[0064] The image signal processor (ISP) may convert the resolution of the image data 40 through the resolution converter 110, and thus may generate a first image (HRI1) having a size smaller than that of the image data 40. Here, the first image (HRI1) may include R, G, B, and W pixel values ​​each having a size smaller than that of the image data 40. For example, the first image (HRI1) may have a size corresponding to half the size of the image data 40.

[0065] The image signal processor (ISP) may convert the resolution of the first image (HRI1) through the resolution converter 120 to generate a second image (HRI2) having a smaller size than the first image (HRI1). The second image (HRI2) may include R, G, B, and W pixel values ​​each having a smaller size than the first image (HRI1). For example, the second image (HRI2) may have a size corresponding to 1 / 4 of the size of the image data 40.

[0066] Similarly, the image signal processor (ISP) may acquire a third image (HRI3) having a smaller size than the second image (HRI2) through the resolution converter 130, may acquire a fourth image (HRI4) having a smaller size than the third image (HRI3) through the resolution converter 140, may acquire a fifth image (HRI5) having a smaller size than the fourth image (HRI4) through the resolution converter 150, and may acquire a sixth image (HRI6) having a smaller size than the fifth image (HRI5) through the resolution converter 160. The third image (HRI3) may have a size corresponding to 1 / 8 of the size of the image data 40, the fourth image (HRI4) may have a size corresponding to 1 / 16 of the size of the image data 40, the fifth image (HR15) may have a size corresponding to 1 / 32 of the size of the image data 40, and the sixth image (HRI6) may have a size corresponding to 1 / 64 of the size of the image data 40.

[0067] The image signal processor (ISP) may generate a processed image (PI1) using the color pixel values ​​and white pixel values ​​included in the image data 40 through the element processor 210. Here, the element processor 210 may extract texture information based on the color pixel values ​​and white pixel values ​​included in the image data 40, and thus may output an image for synthesis (i.e., a synthesized image). In addition, the element processor 210 may perform noise reduction on the white pixel values ​​based on the extracted texture information. In addition, the element processor 210 may calculate the chrominance (color difference) value between the color pixel values ​​and the white pixel values ​​based on the extracted texture information. An image set including a composite image output from the element processor 210, a white image processed for noise reduction, and chrominance values ​​may be understood as a processed image (PI1). For example, the processed image (PI1) may have the same size as the image data 40.

[0068] The image signal processor (ISP) may generate a processed image (PI2) using the converted color pixel values ​​and the converted white pixel values ​​included in the first image (HRI1) through the element processor 220. The element processor 220 may extract texture information based on the converted color pixel values ​​and the converted white pixel values ​​included in the first image (HRI1), and thus may output an image for synthesis (i.e., a composite image). In addition, the element processor 220 may perform noise reduction on the white pixel values ​​based on the extracted texture information. In addition, the element processor 220 may calculate a chrominance (color difference) value between the color pixel values ​​and the white pixel values ​​based on the extracted texture information. An image set including the composite image output from the element processor 220, the white image processed for noise reduction, and the chrominance value may be understood as a processed image (PI2). For example, the processed image (PI2) may have a size corresponding to half the size of the image data 40.

[0069] The image signal processor (ISP) may generate a processed image (PI3) using the converted color pixel values ​​and the converted white pixel values ​​included in the second image (HRI2) through the element processor 230. The element processor 230 may extract texture information based on the converted color pixel values ​​and the converted white pixel values ​​included in the second image (HRI2), and thus may output an image for synthesis (i.e., a composite image). In addition, the element processor 230 may perform noise reduction on the white pixel values ​​based on the extracted texture information. In addition, the element processor 230 may calculate a chrominance (color difference) value between the color pixel values ​​and the white pixel values ​​based on the extracted texture information. An image set including the composite image output from the element processor 230, the white image processed for noise reduction, and the chrominance value may be understood as a processed image (PI3). For example, the processed image (PI3) may have a size corresponding to 1 / 4 of the size of the image data 40.

[0070] Likewise, the image signal processor (ISP) may generate a processed image (PI4) corresponding to the third image (HRI3) through the element processor 240, may generate a processed image (PI5) corresponding to the fourth image (HRI4) through the element processor 250, may generate a processed image (PI6) corresponding to the fifth image (HRI5) through the element processor 260, and may generate a processed image (PI7) corresponding to the sixth image (HRI6) through the element processor 270. Here, the processed image (PI4) may have a size corresponding to 1 / 8 of the size of the image data 40, the processed image (PI5) may have a size corresponding to 1 / 16 of the size of the image data 40, the processed image (PI6) may have a size corresponding to 1 / 32 of the size of the image data 40, and the processed image (PI7) may have a size corresponding to 1 / 64 of the size of the image data 40.

[0071] The image signal processor (ISP) may synthesize the processed image (PI2) and the composite image (SI2) through the synthesizer 310, and thus may generate a composite image (SI1). Here, an image set including the composite chrominance image and the composite white image output from the synthesizer 310 may be understood as a composite image (SI1). The composite image (SI1) may have a size corresponding to half the size of the image data 40.

[0072] The image signal processor (ISP) may synthesize the processed image (PI3) and the composite image (SI3) through the synthesizer 320, and thus may generate a composite image (SI2). Here, an image set including the composite chrominance image and the composite white image output from the synthesizer 320 may be understood as a composite image (SI2). The composite image (SI2) may have a size corresponding to 1 / 4 of the size of the image data 40.

[0073] Likewise, the image signal processor (ISP) may generate a composite image (SI3) by synthesizing the processed image (PI4) and the composite image (SI4) via the synthesizer 330, may generate a composite image (SI4) by synthesizing the processed image (PI5) and the composite image (SI5) via the synthesizer 340, and may generate a composite image (SI5) by synthesizing the processed image (PI6) and the processed image (PI7) via the synthesizer 350. Here, the composite image (SI3) may have a size corresponding to 1 / 8 of the size of the image data 40, the composite image (SI4) may have a size corresponding to 1 / 16 of the size of the image data 40, and the composite image (SI5) may have a size corresponding to 1 / 32 of the size of the image data 40.

[0074] The image signal processor (ISP) may generate an RGB image through the image acquisition circuit 400 using the white pixel value 41 included in the image data 40, the processed image (PI1), and the composite image (SI1). The image signal processor (ISP) may acquire an output image 500 by converting the RGBW image into an RGB image through the image acquisition circuit 400.

[0075] Although for the sake of convenience, Figure 5 The image signal processor (ISP) is shown to include six resolution converters (110-160), seven element processors (210-270), and five synthesizers (310-350), but other implementations are possible.

[0076] The following will refer to Figures 6 to 12B Detailed Description Figure 5 The above-mentioned constituent elements shown.

[0077] Figure 6Some embodiments based on the disclosed technology are shown Figure 5 FIG. 1 is a schematic diagram of an example of a detailed configuration and operation of the resolution converter 110 shown.

[0078] Reference Figure 6 , the resolution converter 110 may include a Bayer resolution converter 113 and a white resolution converter 114 .

[0079] The Bayer resolution converter 113 may convert the resolution of the Bayer image 111 of the image data 40 to output color images (R, G, and B images) each of which has a smaller size than the image data 40. For example, each of the R, G, and B images (corresponding to the first image HRI1) output from the Bayer resolution converter 113 may have a size corresponding to half the size of the Bayer image 111. For example, “half the size” may mean reducing the number of pixels by half, thus reducing the data storage capacity by half.

[0080] The white resolution converter 114 may convert the resolution of the white image 112 of the image data 40 to output a white image having a size smaller than that of the image data 40. For example, the white image output from the white resolution converter 114 may have a size corresponding to half the size of the image data 40.

[0081] Here, the positions of the Bayer image 111 and the white image 112 of the image data 40 input to the resolution converter 110 may be misaligned. Therefore, the Bayer resolution converter 113 may perform interpolation so that the position of the center pixel of the reduced (downscaled) white image coincides with the center of gravity of each of the reduced (downscaled) R, G, and B images. That is, in order to prevent or reduce the defocus phenomenon that occurs in the white image 112 for obtaining texture information when downscaling the Bayer image 111 and the white image 112, the Bayer resolution converter 113 may calculate the interpolation coefficient of each of the R, G, and B pixels based on the white image 112. For example, the Bayer resolution converter 113 may perform interpolation using various interpolation methods such as bilateral interpolation, bicubic interpolation, bilinear interpolation, etc. so that the center of gravity of the R, G, and B images coincides with the center of gravity of the white image. Here, the method of interpolating the R, G, and B images to match the center of gravity is not limited to this.

[0082] Figure 7 Some embodiments based on the disclosed technology are shown Figure 5 2 is a schematic diagram of an example of a detailed configuration of the resolution converter 120 shown in FIG.

[0083] Reference Figure 7, the resolution converter 120 may include a red (R) resolution converter 121 , a green (G) resolution converter 122 , a blue (B) resolution converter 123 , and a white (W) resolution converter 124 .

[0084] Here, the red resolution converter 121 may convert the resolution of the first image (HRI1) of the resolution converter 110 (i.e., the resolution of the red image) to output a red image having a size smaller than that of the image data 40. For example, the red resolution converter 121 may average pixel values ​​obtained from four red pixels (R) to output a red image having a smaller size (i.e., reduced size).

[0085] The green resolution converter 122 may convert the resolution of the first image (HRI1) of the resolution converter 110 (i.e., the resolution of the green image) to output a green image having a size smaller than that of the image data 40. For example, the green resolution converter 122 may average pixel values ​​obtained from four green pixels (G) to output a green image having a smaller size (i.e., reduced size).

[0086] The blue resolution converter 123 may convert the resolution of the first image (HRI1) of the resolution converter 110 (i.e., the resolution of the blue image) to output a blue image having a size smaller than that of the image data 40. For example, the blue resolution converter 123 may average pixel values ​​obtained from four blue pixels (B) to output a blue image having a smaller size (i.e., reduced size).

[0087] The white resolution converter 124 may convert the resolution of the first image (HRI1) of the resolution converter 110 (i.e., the resolution of the white image) to output a white image smaller in size than the image data 40. For example, the white resolution converter 124 may average pixel values ​​obtained from four white pixels (W) to output a white image smaller in size (i.e., reduced in size).

[0088] For example, the R, G, B, and W image (ie, the second image HRI2 ) output from the resolution converter 120 may have a size corresponding to a half size of the input image (ie, the first image HRI1 ).

[0089] Compared to the resolution converter 120, except for the size of the input / output (I / O) image, the remaining resolution converters (130~160) have the same configuration as the resolution converter 120, so for the sake of conciseness, the redundant description of the detailed configuration and operation of the resolution converters (130~160) will be omitted in this article.

[0090] Figure 8 It is shown Figure 5 2 is a schematic diagram of an example of a detailed configuration of the element processor 210 shown.

[0091] Reference Figure 8 , the element processor 210 may include a texture determiner 213 , a noise reduction circuit 214 , and a chroma (color difference) calculator 215 .

[0092] The texture determiner 213 may determine the texture based on the Bayer image 211 and the white image 212 of the image data 40, and thus may generate texture information (TI) and an image for synthesis (T). Figure 8 The Bayer image 211 and the white image 212 of the texture determiner 213 shown can be respectively Figure 6 The Bayer image 111 and white image 112 shown are identical.

[0093] Here, the texture information (TI) may include information about whether the Bayer image (BY) 211 and the white image 212 belong to a flat image region or a gradient image region, information about whether there is an edge or a boundary, and information about the direction of the boundary when there is a boundary. For example, the texture determiner 213 may preferentially determine the texture information (TI) of the white image (W) 212. For example, the texture determiner 213 may first determine the texture information (TI) in the white image 212. When there is no texture information in the white image 212, the texture determiner 213 may determine the texture information (TI) in the Bayer image 211.

[0094] In addition, when Figure 4 and Figure 5 When multi-scale processing is performed in the image acquisition circuit 400 shown, the ratio of two images to be synthesized with each other may be adjusted based on the texture information (TI). Therefore, the texture determiner 213 may convert the texture information (TI) into a synthesized image (T) for the synthesis operation of the image acquisition circuit 400, and may provide the synthesized image (T) to the image acquisition circuit 400.

[0095] Here, the synthesized image (T) may be an image used when two images having different resolutions are synthesized with each other. For example, the two images may be synthesized using an alpha (α) blending method. For example, an interpolation process may be performed on a low-resolution image using linear interpolation, and an output image whose resolution matches that of a high-resolution image may be generated. Although the two images are synthesized using an alpha (α) blending method, the scope of the disclosed technology is not limited thereto, and various methods for synthesizing the two images may be used. The synthesized image (T) may be defined as an image (or image value) obtained when data representing the amplitude or synthesis ratio of a texture is calculated for each pixel. For example, the synthesized image (T) may be used when synthesizing two images by increasing the ratio of a high-resolution image in a boundary area having an edge or texture, or may be used when synthesizing two images by increasing the ratio of a low-resolution image in a flat area having no edge or no texture.

[0096] The noise reduction circuit 214 may perform noise reduction processing on the pixel values ​​of the white image 212 based on the texture information (TI), and may generate a noise-reduced white image (W_NR1). The noise reduction circuit 214 may perform different noise reduction processing not only in some areas where the texture is greater than or equal to a specific level based on the texture information (TI), but also in other areas where the texture is less than a specific level based on the texture information (TI).

[0097] For example, the noise reduction circuit 214 may perform noise reduction by averaging the weights of neighboring pixels located around a target pixel (i.e., a center pixel located at the center of an input image unit) in a direction of the texture (e.g., the direction of the edge or boundary) in some areas (e.g., image areas with edges or boundaries) where the texture is equal to or higher than a certain level based on the texture information (TI). In addition, the noise reduction circuit 214 may perform noise reduction by averaging the pixel values ​​of neighboring pixels located around the target pixel in some areas (e.g., flat image areas or gradient image areas) where the texture is less than a certain level based on the texture information (TI).

[0098] In addition, the chrominance calculator 215 may calculate a chrominance value (B_CD1) between the Bayer image 211 and the white image 212 based on the texture information (TI).

[0099] For example, the chrominance calculator 215 may calculate chrominance values ​​in the direction of the texture (i.e., the direction of the edge or boundary) in some areas where the texture is equal to or higher than a certain level based on the texture information TI. In addition, the chrominance calculator 215 may calculate chrominance (color difference) values ​​using neighboring pixels located around the target pixel in other areas where the texture is less than a certain level (e.g., a flat image area or a gradient image area) based on the texture information (TI).

[0100] The composite image (T), the noise-reduced white image (W_NR1), and the chrominance (color difference) values ​​(B_CD1) generated by the element processor 210 may be sent to the image acquisition circuit 400 as a processed image (PI1).

[0101] Fig. 9 It is shown Figure 5 FIG. 2 is a schematic diagram of an example of a detailed configuration of the element processor 220 shown in FIG.

[0102] Reference Fig. 9 , the element processor 220 may include a texture determiner 223 , a noise reduction circuit 224 , and a chroma calculator 225 .

[0103] Here, the texture determiner 223 may determine a texture based on the first image (HRI1) 221 (ie, red, green, and blue images, RGB) and the white image (W) 222 of the resolution converter 110, and may generate a synthesized image (T) and texture information (TI).

[0104] When Figure 4 and Figure 5 When the synthesizer 310 shown performs an operation of synthesizing two input images, the ratio of the composite image may be adjusted based on the texture information (TI). Therefore, the texture determiner 223 may convert the texture information (TI) into a synthesized image (T) for the synthesizing operation of the synthesizer 310, and may provide the synthesizer 310 with the synthesized image (T).

[0105] In addition, the noise reduction circuit 224 may perform noise reduction processing on the pixel values ​​of the white image 222 based on the texture information (TI), and may generate a noise-reduced white image (W_NR2). The noise reduction circuit 224 may perform different noise reduction processing not only in some areas where the texture is equal to or higher than a specific level, but also in other areas where the texture is less than a specific level based on the texture information (TI).

[0106] For example, the noise reduction circuit 224 may perform noise reduction by averaging the weights of neighboring pixels located around the target pixel in the direction of the texture (i.e., the direction of the edge or boundary) in some areas where the texture is equal to or higher than a certain level (e.g., image areas where edges or boundaries exist) based on the texture information (TI). In addition, the noise reduction circuit 224 may perform noise reduction by averaging the pixel values ​​of neighboring pixels located around the target pixel in other areas where the texture is less than a certain level (e.g., a flat image area or a gradient image area) based on the texture information (TI).

[0107] In addition, the chromaticity calculator 225 may calculate a chromaticity value (RGB_CD1) between the RGB image and the white image 222 based on the texture information (TI).

[0108] For example, the chromaticity calculator 225 may calculate the chromaticity value in the direction of the texture (i.e., the direction of the edge or boundary) in some areas (e.g., image areas with edges or boundaries) where the texture is equal to or higher than a certain level based on the texture information (TI). In addition, the chromaticity calculator 225 may calculate the chromaticity value using neighboring pixels located around the target pixel in some areas (e.g., flat image areas or gradient image areas) where the texture is less than a certain level based on the texture information (TI). For example, the chromaticity value may be calculated by dividing the average of the red R pixels (or the green G pixels or the blue B pixels) by the average of the white W pixels.

[0109] The synthesized image (T), the noise-reduced white image (W_NR2), and the chrominance values ​​(RGB_CD1) generated by the element processor 220 may be sent to the synthesizer 310 as a processed image (PI2).

[0110] Compared to the element processor 220, except for the size of the input / output (I / O) image, the remaining element processors (230~270) have the same configuration as the element processor 220, so for the sake of conciseness, the redundant description of the detailed configuration and operation of the element processors (230~270) will be omitted in this article.

[0111] Fig.10 Some embodiments based on the disclosed technology are shown Figure 5 A schematic diagram of an example of a detailed configuration of the synthesizer 310 is shown.

[0112] Reference Fig.10 , the synthesizer 310 may include a chrominance synthesizer 311 and a white (W) image synthesizer 312.

[0113] The chrominance synthesizer 311 may synthesize the chrominance value (RGB_CD1) and the chrominance value (RGB_CD2) based on the synthesized image (T) received from the element processor 220, and thus may generate a composite chrominance image (RGB_CD3). Here, the chrominance value (RGB_CD1) may refer to the chrominance (color difference) value of the RGB image (corresponding to the processed image PI2) received from the element processor 220. In addition, the chrominance value (RGB_CD2) may refer to the chrominance value of the low-resolution RGB image (corresponding to the composite image SI2) received from the synthesizer 320.

[0114] The chroma synthesizer 312 may synthesize the white image (W_NR2) and the white image (W_NR3) based on the synthesized image (T) received from the element processor 220, and thus may generate a composite white image (W_NR4). Here, the white image (W_NR2) may refer to a low-noise white image (corresponding to the processed image PI2) received from the element processor 220. In addition, the white image (W_NR3) may refer to a low-resolution white image (corresponding to the composite image SI2) received from the synthesizer 320.

[0115] For example, the chroma synthesizer 311 and the white image synthesizer 312 may increase the ratio of the reduced image (e.g., the composite image SI2) in the flat image region or the gradient image region based on the image (T) for synthesis, and may increase the ratio of the non-reduced input image (e.g., the processed image PI2) in the image region having an edge or boundary, so that the two input images may be synthesized. Therefore, in an area where the texture is equal to or higher than a certain level, the sharpness may be maintained, and in another area where the texture is less than a certain level, the noise may be reduced.

[0116] Compared to synthesizer 310, except for the size of the input / output (I / O) image, the remaining synthesizers (320~350) have the same configuration as synthesizer 310, so for the sake of simplicity, the redundant description of the detailed configuration and operation of the synthesizers (300~350) will be omitted in this article.

[0117] Fig.11 Some embodiments based on the disclosed technology are shown Figure 5 A schematic diagram of an example of an image acquisition circuit 400 is shown. Fig. 12A and Fig. 12B Some embodiments based on the disclosed technology are shown Fig.11 A graph showing an example of the operation of a chrominance signal interpolator is shown.

[0118] Reference Fig.11 , the image acquisition circuit 400 may include a chrominance synthesizer 401, a white (W) image synthesizer 402, an edge enhancement signal generator 403, a moiré region determiner 404, a chrominance signal interpolator 405, and an image generator 406.

[0119] The chroma synthesizer 401 may synthesize the chroma value (B_CD1) received from the element processor 210 and the composite chroma image (RGB_CD3) received from the synthesizer 310 based on the synthesized image (T) generated by the element processor 210, and thus may generate a composite chroma image (RGB_CD4). Here, as one input of the chroma synthesizer 401, the chroma value (B_CD1) may be a Bayer image, so that the composite chroma image (RGB_CD4) may also be a Bayer image.

[0120] White image synthesizer 402 may synthesize the low noise white image (W_NR1) received from element processor 210 and the composite white image (W_NR4) received from synthesizer 310 based on the composite image (T) generated by element processor 210, and may thus generate a composite white image (W_NR5).

[0121] For example, the chrominance synthesizer 401 and the white image synthesizer 402 can increase the ratio of the reduced image (e.g., the composite image SI1) in the flat image area or the gradient image area based on the synthesized image (T), and can increase the ratio of the non-reduced image (e.g., the processed image PI1) in the image area having edges or boundaries, thereby synthesizing the two input images.

[0122] The edge enhancement signal generator 403 may perform an edge enhancement on the white image W1 (corresponding to Figure 6The edge enhancement signal generator 403 performs edge enhancement processing on the composite white image (W_NR5) received from the white image synthesizer 402. As a result, a white image (W2) obtained by completing such edge enhancement processing can be output. That is, before the image generator 406 generates the output RGB image, the edge enhancement signal generator 403 performs edge enhancement processing to increase the contrast, thereby making the image texture of the output RGB image clearer.

[0123] For example, the edge enhancement signal generator 403 may first extract high-frequency components from the white image (W1) of the original image data 40 from which noise has not been reduced, and may filter frequencies greater than a preset frequency, thereby generating a texture enhancement component. Then, the edge enhancement signal generator 403 may generate a white image W2 that has undergone edge enhancement processing by adding the texture enhancement component to the composite white image (W_NR5) that has undergone noise reduction processing. In addition, since the edge enhancement signal generator 403 may perform edge enhancement processing by combining the composite white image (W_NR5) that has undergone noise reduction processing and the original white image (W1) from which noise has not been reduced, the sharpness of an area that is excessively blurred due to such noise reduction processing may be increased.

[0124] The moiré region determiner 404 may determine the moiré region based on the white image (W1) of the image data 40 and the composite white image (W_NR5) received from the image synthesizer 402, and thus may generate a moiré determination signal (MDS). Here, the term "moiré" may refer to an uneven image generated when interference of high-frequency components occurs in a region where the pattern is similar to the spatial frequency of the image sensor when photographing an object having a repetitive pattern. The moiré region determiner 404 may detect the moiré region in the white image (W1) and the composite white image (W_NR5), and may generate a moiré determination signal (MDS) corresponding to whether the moiré region is detected.

[0125] For example, an area where color moiré (false color) occurs may be an area where there is a texture having a frequency higher than the Nyquist frequency corresponding to the sampling interval of the color pixels. Since the number of white pixels is less than the number of output pixels, there may be a frequency domain where it is impossible to confirm whether there is a texture having a high-frequency component. In an embodiment of the disclosed technology, since the white image to be input has the same resolution as the output image (i.e., the same number of pixels), it is not necessary to interpolate the white image. Therefore, in an embodiment, it can be seen from the white image shown in the figure that it can be determined whether there is a high-frequency texture up to the Nyquist frequency of the output image. That is, in an embodiment, the moiré area determiner 404 can determine whether there is a high-frequency component around each pixel of the white image, and therefore can determine whether the corresponding area is an area where color moiré occurs. Here, "high-frequency component" may mean a frequency having a frequency higher than a predetermined frequency (e.g., the above-mentioned Nyquist frequency).

[0126] Here, the method for determining the color moiré region by the moiré region determiner 404 may also use a method for calculating the intensity of the amplitude of the high-frequency component using Fourier transform, Hadamard transform, high-pass filter, etc. As another example, a method for adding the absolute difference of the pixel values ​​adjacent to each other in the horizontal or vertical direction based on the center pixel and determining the size of the added value may be used. As another example, a method for comparing the absolute difference of the upper and lower pixels of the center pixel with the absolute difference of the left and right pixels of the center pixel, thereby calculating the average value of a pair of pixels with a smaller value may be used. Then, the calculated average value and the absolute difference of the pixel value of the center pixel are calculated, and the calculated value is compared with the noise amount. In this case, if the calculated value in the corresponding area is greater than the noise amount, the corresponding area can be determined as a color moiré region. The method for determining the moiré region is not limited to this, and other methods may also be used as needed.

[0127] The chrominance signal interpolator 405 may interpolate the composite white image (W_NR5) and the composite chrominance image (RGB_CD4) based on the moiré determination signal (MDS) and the synthesized image (T), and may output an interpolated RGB image (IRGB). Here, as an input signal of the chrominance signal interpolator 405, a composite white image (W_NR5) having the same resolution as the output image and not requiring interpolation may be used. The chrominance signal interpolator 405 may interpolate the color difference between the red image (or the green image or the blue image) and the white image based on the composite white image (W_NR5), and may generate an interpolated RGB image (IRGB).

[0128] For example, refer to Fig. 12A , when the corresponding area is not a color moiré area based on the moiré determination signal (MDS), the chrominance signal interpolator 405 may use the white image to determine the direction, and may interpolate the red (R) chrominance, the green (G) chrominance, and the blue (B) chrominance based on the determined direction. In some embodiments, the interpolation of the R chrominance, the interpolation of the G chrominance, and the interpolation of the B chrominance may be linear interpolation, but the scope of the disclosed technology is not limited thereto.

[0129] On the other hand, referring to Fig. 12B, when the corresponding area is determined to be a color moiré area, the chrominance signal interpolator 405 may interpolate the chrominance in the color moiré area based on the moiré determination signal (MDS). In some embodiments, such chrominance interpolation may be performed using filtering (e.g., guided filtering) that takes edge information into account. That is, it may be assumed that the texture obtained by linearly transforming the white image W(x) using arbitrary coefficients (a, b) is each chrominance image (represented by "chrominance (x)"). The coefficients (a, b) of such linear transformation may be calculated by linear regression to calculate the pixel value of the center pixel so that interpolation may be performed. For example, in order to calculate such linear regression, the pixel values ​​of the white image and the pixel values ​​of the chrominance image of a plurality of neighboring pixels located around the center pixel may be used. The chrominance image (represented by "chrominance (x)") may be determined using the coefficients (a, b) calculated by linear regression.

[0130] In addition, the image generator 406 may generate an output RGB image based on the interpolated RGB image (IRGB) and the white image (W2) that has completed the edge enhancement process. For example, the image generator 406 may generate an output RGB image by multiplying the interpolated RGB image (IRGB) by the pixel value of the white image (W2) that has completed the edge enhancement process. Therefore, in an embodiment, the RGB image generator 406 may generate an output RGB image with reduced color distortion and reduced noise.

[0131] As apparent from the above description, an image signal processor based on some embodiments of the disclosed technology can obtain an image with higher sharpness and less noise, and thus can improve the quality of an image signal.

[0132] The embodiments of the disclosed technology can provide various effects that can be directly or indirectly recognized through the above-mentioned patent documents.

[0133] Although several exemplary embodiments are described, it should be understood that modifications and enhancements to the disclosed embodiments, as well as other embodiments, are contemplated based on what is described and / or illustrated in this patent document.

[0134] CROSS-REFERENCE TO RELATED APPLICATIONS

[0135] This patent document claims priority to and the benefit of Korean Patent Application No. 10-2023-0091672, filed on Jul. 14, 2023, which is hereby incorporated by reference in its entirety as a part of the disclosure of this patent document.

Claims

1. An image signal processor, comprising: a resolution converter that generates at least two images based on image data including color pixel values ​​and white pixel values; an element processor that extracts texture information using the converted color pixel values ​​and the converted white pixel values ​​included in each of the at least two images and generates at least two processed images based on the extracted texture information; a synthesizer that generates a composite chrominance image and a composite white image by synthesizing the at least two processed images; as well as An image acquisition circuit generates an output image based on the image data, the composite chrominance image, and the composite white image.

2. The image signal processor according to claim 1, wherein: The resolution converter comprises: a first resolution converter that converts the resolution of the image data to generate a first image that is smaller in size than the image data; and A second resolution converter converts the resolution of the first image to generate a second image smaller in size than the image data.

3. The image signal processor according to claim 2, wherein: The first resolution converter comprises: a Bayer resolution converter that converts the resolution of the Bayer image of the image data and outputs a first red image, a first green image, and a first blue image each having a size smaller than the Bayer image; and A white resolution converter converts the resolution of a white image of the image data and outputs a first white image which is smaller in size than the white image.

4. The image signal processor according to claim 3, wherein: The Bayer resolution converter performs interpolation so that a center of gravity of a central pixel of each of the first red image, the first green image, and the first blue image coincides with a center of gravity of a central pixel of the first white image.

5. The image signal processor according to claim 3, wherein: The second resolution converter comprises: a red resolution converter converting a resolution of the first red image to output a second red image having a size smaller than that of the first red image; a green resolution converter that converts a resolution of the first green image to output a second green image having a size smaller than that of the first green image; a blue resolution converter converting a resolution of the first blue image to output a second blue image having a size smaller than that of the first blue image; and A white resolution converter converts the resolution of the first white image to output a second white image smaller in size than the first white image.

6. The image signal processor according to claim 2, wherein: The element processor comprises: a first element processor that extracts texture information using the image data and generates a first processed image based on the extracted texture information; a second element processor that generates a second processed image corresponding to the first image by using first converted color pixel values ​​and first converted white pixel values ​​included in the first image; and A third element processor generates a third processed image corresponding to the second image by using second converted color pixel values ​​and second converted white pixel values ​​included in the second image.

7. The image signal processor according to claim 6, wherein: The first element processor comprises: a first texture determiner that determines a texture based on the Bayer image and the white image of the image data, and outputs texture information and a composite image obtained by conversion of the texture information; a first noise reduction circuit that performs noise reduction processing on pixel values ​​of the white image based on the texture information and generates a third white image with less noise; and A first chromaticity calculator is configured to calculate a first chromaticity value between the Bayer image and the white image based on the texture information.

8. The image signal processor according to claim 7, wherein: The texture information includes: At least one of information on whether the Bayer image and the white image belong to a flat image region or a gradation image region, information on whether there is an edge or a boundary, and information on directionality of the boundary.

9. The image signal processor according to claim 7, wherein: The first noise reduction circuit: Based on the texture information, different noise reduction processes are performed not only in some areas where the texture is equal to or higher than a certain level but also in other areas where the texture is smaller than the certain level.

10. The image signal processor according to claim 6, wherein: The second element processor comprises: a second texture determiner that determines a texture based on the first converted color pixel value and the first converted white pixel value, and outputs texture information and a synthesized image obtained by conversion of the texture information; a second noise reduction circuit that performs noise reduction processing on the first converted white pixel value based on the texture information and generates a fourth white image with less noise; and A second chromaticity calculator calculates a second chromaticity value between the first converted color pixel value and the first converted white pixel value based on the texture information.

11. The image signal processor according to claim 10, wherein: The synthesizer comprises: a first synthesizer that generates a first composite image by synthesizing the second composite image and a second processed image among the at least two processed images; and A second synthesizer generates the second composite image by synthesizing the third composite image and a third processed image among the at least two processed images.

12. The image signal processor according to claim 11, wherein: The first synthesizer comprises: a first chrominance synthesizer that generates a first composite chrominance image by synthesizing the second chrominance values ​​and third chrominance values ​​of the second image based on the synthesized image; and A first white image synthesizer generates a first composite white image by synthesizing the fourth white image and a fifth white image included in the second composite image based on the composite image.

13. The image signal processor according to claim 12, wherein: The first synthesizer enables: A ratio of the second chromaticity value to the third chromaticity value is adjusted and synthesized, and a ratio of the fourth white image to the fifth white image is adjusted and synthesized.

14. The image signal processor according to claim 12, wherein: The image acquisition circuit comprises: a second chroma synthesizer that generates a second composite chroma image by synthesizing first chroma values ​​and the first composite chroma image based on the composite image; a second white image synthesizer that generates a second composite white image by synthesizing a third white image received from the first element processor and the first composite white image based on the composite image; an edge enhancement signal generator for generating a sixth white image that has undergone edge enhancement processing by combining the white image of the image data and the second composite white image; a moiré region determiner that determines a moiré region based on the white image and the second composite white image of the image data and generates a moiré determination signal; a chrominance signal interpolator that outputs interpolated red, green, and blue RGB images by interpolating the second composite chrominance image and the second composite white image based on the moiré determination signal and the synthesized image; and An image generator generates an output RGB image based on the sixth white image and the interpolated RGB image.

15. The image signal processor according to claim 14, wherein: The image acquisition circuit enables: A ratio of the first chromaticity value to the first composite chromaticity image is adjusted and synthesized, and a ratio of the third white image to the first composite white image is adjusted and synthesized.

16. The image signal processor according to claim 14, wherein: The edge enhancement signal generator: A texture enhancement component is generated by extracting a high frequency component from the white image, and the sixth white image is generated by adding the generated texture enhancement component to the second composite white image.

17. The image signal processor according to claim 14, wherein: The moiré region determiner: It is determined whether there is a high-frequency component around each pixel of the white image and the second composite white image, and whether the corresponding area is an area where color moiré occurs.

18. The image signal processor according to claim 14, wherein: The chroma signal interpolator: When the corresponding area is determined to be a color moiré area based on the moiré determination signal, performing interpolation using a filtering process; and When the corresponding area is determined to be an area outside the color moiré area, directivity is determined using the second composite white image, and interpolation of red R chromaticity, green G chromaticity, and blue B chromaticity is performed based on the determined directivity.

19. The image signal processor according to claim 14, wherein: The image generator: The output RGB image is generated by multiplying the interpolated RGB image by the pixel values ​​of the sixth white image.

20. The image signal processor according to claim 1, further comprising: A binning circuit generates the color pixel value and the white pixel value by binning the color pixels and the white pixels in the image data.

Images