Image sensor and image processing method and apparatus

Abstract

The present application provides an image sensor and an image processing method and apparatus. The image sensor of the present application comprises: a color filter array (CFA), the CFA comprising a first pixel, a second pixel, a third pixel, and pixels W. A first light-splitting structure is provided on the surface of the first pixel, a second light-splitting structure is provided on the surface of the second pixel, and a third light-splitting structure is provided on the surface of the third pixel. The first light-splitting structure is used for splitting and distributing a first optical signal, which is not received by the first pixel, to pixels W adjacent to the first pixel; the second light-splitting structure is used for splitting and distributing a second optical signal, which is not received by the second pixel, to pixels W adjacent to the second pixel; and the third light-splitting structure is used for splitting and distributing a third optical signal, which is not received by the third pixel, to pixels W adjacent to the third pixel. The present application can increase the amount of light entering a pixel, thereby improving the imaging quality in low-illumination environments.

Description

Image sensor and image processing method and device TECHNICAL FIELD

[0001] The present application relates to the imaging technology field, and in particular to an image sensor and an image processing method and device. BACKGROUND

[0002] With the rapid development of image applications, the imaging quality of image acquisition devices in low-illumination environments is increasingly required. The imaging quality in low-illumination environments is lower than that in normal-illumination environments, mainly manifested as reduced image brightness and increased noise. The biggest factor leading to the decline in imaging quality is the weakening of the light input of the image sensor, which leads to a reduction in the signal-to-noise ratio of the output signal. Therefore, improving the light input of the image sensor in low-illumination environments is the key to improving the imaging quality in low-illumination environments. SUMMARY

[0003] The present application provides an image sensor and an image processing method and device, which can increase the light input of the pixels and achieve the purpose of improving the imaging quality in low-illumination environments.

[0004] In a first aspect, the present application provides an image sensor, comprising: a color filter array (CFA), the CFA comprising a first pixel, a second pixel, a third pixel and a W pixel; wherein a surface of the first pixel is provided with a first light splitting structure, a surface of the second pixel is provided with a second light splitting structure, and a surface of the third pixel is provided with a third light splitting structure; the first light splitting structure is configured to split a first light signal not received by the first pixel to a W pixel adjacent to the first pixel; the second light splitting structure is configured to split a second light signal not received by the second pixel to a W pixel adjacent to the second pixel; and the third light splitting structure is configured to split a third light signal not received by the third pixel to a W pixel adjacent to the third pixel.

[0005] Optionally, the first light splitting structure, the second light splitting structure and the third light splitting structure are respectively configured to implement light signal distribution in at least one of the upward, downward, leftward or rightward directions. For example, the first light splitting structure splits the first light signal to W pixels adjacent to the first pixel in the upward, downward, leftward or rightward directions; the second light splitting structure splits the second light signal to W pixels adjacent to the second pixel in the upward or downward directions or in the leftward or rightward directions; and the third light splitting structure splits the third light signal to W pixels adjacent to the third pixel in the upward, downward, leftward or rightward directions.

[0006] Based on the above light splitting structure, the light signal not received by the pixel can be split and distributed to the pixels adjacent in the upward and / or downward directions and / or the leftward and / or rightward directions, and the direction of the light splitting can be determined by adjusting the design of the light splitting structure, so as to purposefully increase the light input of the adjacent pixels and achieve the purpose of improving the imaging quality in low-illumination environments.

[0007] In a possible implementation, a surface of the W pixel is provided with a fourth light splitting structure; the fourth light splitting structure is configured to perform at least one of the following light splitting operations: splitting a fourth light signal that is not received by the W pixel to a first pixel adjacent to the W pixel; or splitting a fifth light signal that is not received by the W pixel to a second pixel adjacent to the W pixel; or splitting a sixth light signal that is not received by the W pixel to a third pixel adjacent to the W pixel.

[0008] Optionally, the fourth light splitting structure can be similar to the first light splitting structure, the second light splitting structure, and the third light splitting structure, and is configured to perform light signal distribution in at least one of the up, down, left, or right directions.

[0009] In addition to receiving the light signal from the W pixel, the amount of light received by the visible light pixel can also be increased, and the color in the image obtained based on the visible light pixel can be enhanced, thereby further improving the imaging quality in a low-illumination environment.

[0010] Optionally, an infrared light cutoff filter layer (which can adopt a stop Infra-red, SIR, structure) is further arranged in the visible light pixel. The infrared light cutoff filter layer can block light rays exceeding 650 nm or block infrared light of a certain specific waveband (for example, 850 nm light rays), so that infrared light cannot pass through the infrared light cutoff filter layer.

[0011] The infrared light cutoff filter layer arranged in the visible light pixel in the present application cuts off the infrared light reaching the visible light pixel, removes the infrared light (Infra-red, IR) signal in the photosensitive result of the visible light pixel, makes the color of the photosensitive result more accurate, and improves the photosensitive effect of the sensor.

[0012] In a possible implementation, the CFA further includes a supervision pixel; a surface of the supervision pixel is provided with a fifth light splitting structure; the fifth light splitting structure is configured to split infrared light that is not received by the supervision pixel to a W pixel adjacent to the supervision pixel.

[0013] In a possible implementation, the fourth light splitter is further configured to split a seventh light signal that is not received by the W pixel to a supervision pixel adjacent to the W pixel.

[0014] In a possible implementation, the first pixel is an R pixel; the second pixel is a G pixel; and the third pixel is a B pixel.

[0015] In a possible implementation, the first light signal includes green light and blue light; the second light signal includes red light and blue light; and the third light signal includes red light and green light.

[0016] In a possible implementation, the first pixel is a C pixel; the second pixel is an M pixel; and the third pixel is a Y pixel.

[0017] In a possible implementation, the first light signal comprises magenta light and yellow light; the second light signal comprises cyan light and yellow light; and the third light signal comprises cyan light and magenta light.

[0018] In a possible implementation, the first light signal, the second light signal, and the third light signal each comprise infrared light.

[0019] In a possible implementation, the first pixel is an R pixel, the fourth light signal comprises red light; the second pixel is a G pixel, the fifth light signal comprises green light; and the third pixel is a B pixel, the sixth light signal comprises blue light.

[0020] In a possible implementation, the first pixel is a C pixel, the fourth light signal comprises cyan light; the second pixel is an M pixel, the fifth light signal comprises magenta light; and the third pixel is a Y pixel, the sixth light signal comprises yellow light.

[0021] In a possible implementation, the seventh light signal comprises red light, green light, or blue light.

[0022] In a possible implementation, the CFA comprises RGBW, RGB2W, RGB3W, MYCW, MYC2W, or MYC3W.

[0023] In a possible implementation, a surface of the W pixel is provided with a fourth light splitting structure; the fourth light splitting structure is configured to perform at least one of the following light splitting operations: splitting a fourth light signal, which is not received by the W pixel, to a first pixel adjacent to the W pixel; or splitting a fifth light signal, which is not received by the W pixel, to a second pixel adjacent to the W pixel; or splitting a sixth light signal, which is not received by the W pixel, to a third pixel adjacent to the W pixel.

[0024] In a second aspect, the present application provides an image processing method, which is applicable to an imaging system comprising the image sensor according to any one of claims 1-7, and the method comprises: obtaining first image data, which is obtained based on the W pixels in the image sensor; obtaining second image data, which is obtained based on the R pixels, the G pixels, and the B pixels in the image sensor; and fusing the first image data and the second image data to obtain a target image.

[0025] In a possible implementation, before the fusing of the first image data and the second image data to obtain the target image, the method further includes: performing interpolation processing and noise reduction processing according to the first image data; and performing color noise removal and pseudo-color reduction processing according to the second image data.

[0026] In a possible implementation, when the second image data further includes infrared light image data, the method further includes: performing infrared light removal processing according to the second image data.

[0027] In a third aspect, the present application provides an image acquisition device, including: the image sensor according to any one of the first aspect; one or more image signal processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more image signal processors, the one or more image signal processors implement the method according to any one of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0028] FIG. 1 shows a pattern diagram of a typical Bayer RGB CFA;

[0029] FIG. 2 shows a pattern diagram of an RGBW CFA;

[0030] FIG. 3a and FIG. 3b respectively show a pattern diagram of an RGB2W CFA;

[0031] FIG. 4 shows a pattern diagram of an RGB3W CFA;

[0032] FIG. 5 shows a pattern diagram of an MYCW CFA;

[0033] FIG. 6a and FIG. 6b respectively show a pattern diagram of an MYC2W CFA;

[0034] FIG. 7 shows a pattern diagram of an MYC3W CFA;

[0035] FIG. 8a and FIG. 8b respectively show a pattern diagram of an RGB2W CFA containing supervised pixels;

[0036] FIG. 9a is a schematic structural diagram of a longitudinal section of an image sensor 900 according to the present application;

[0037] FIG. 9b is a schematic structural diagram of a longitudinal section of an image sensor 900 according to the present application;

[0038] Fig. 9c is a schematic structural view of a longitudinal section of the image sensor 900 according to the present application;

[0039] Fig. 9d is a schematic structural view of a longitudinal section of the image sensor 900 according to the present application;

[0040] Figs. 10a-10d are schematic spectral splitting views of the spectral splitting structure according to the present application;

[0041] Fig. 11 is a schematic spectral splitting view of the RGB2W sensor according to the present application;

[0042] Fig. 12 shows a graph of a light-sensing characteristic of an image sensor;

[0043] Figs. 13a-13e are schematic spectral splitting views of the RGB2W sensor according to the present application;

[0044] Figs. 14a-14f are schematic spectral splitting views of the RGB3W sensor according to the present application;

[0045] Figs. 15a-15e are schematic spectral splitting views of the RGB2W sensor according to the present application;

[0046] Figs. 16a-16c are schematic spectral splitting views of the MYC2W sensor according to the present application;

[0047] Figs. 17a-17c are schematic spectral splitting views of the MYC3W sensor according to the present application;

[0048] Figs. 18a-18d are schematic spectral splitting views of the MYC2W sensor according to the present application;

[0049] Fig. 19 is a schematic structural view of an image acquisition apparatus according to the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0051] The terms "first", "second", and the like in the description and in the claims of the present application and the drawings merely mean differentiating description purposes and cannot be understood as indicating or implying relative importance or indicating or implying sequence. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, comprising a series of steps or units. The method, system, product or device is not necessarily limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases of only A, only B and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0053] The color filter array (CFA) of a conventional Bayer Red Green Blue (RGB) sensor includes three types of pixels, R, G, and B, wherein one type of pixel is only sensitive to one color of light (i.e., one type of pixel only receives one color of light) among R, G, and B, for example, an R pixel is only sensitive to red light (red light band) (i.e., the R pixel only receives red light), a G pixel is only sensitive to green light (green light band) (i.e., the G pixel only receives green light), and a B pixel is only sensitive to blue light (blue light band) (i.e., the B pixel only receives blue light). A plurality of R, G, and B pixels are arranged to form a mosaic color pattern, for example, FIG. 1 shows a pattern diagram of a typical Bayer RGB CFA, as shown in FIG. 1, the pattern adopts a 2x2 array order, the first row of a 2x2 array (the smallest repeating unit) includes an R pixel and a G pixel, and the second row includes a G pixel and a B pixel.

[0054] In order to improve the light amount of the CFA, a white (White, W) pixel (also called a full-pass pixel) is added on the basis of the Bayer RGB CFA. In addition, a W pixel can also be added on the basis of a Magenta, Yellow, Cyan (MYC) CFA. The W pixel can receive visible light (visible light band) and can also receive infrared light (Infra-red, IR) (infrared light band). In this application, visible light can include red light, green light and blue light, and can also include magenta light, yellow light and cyan light.

[0055] In this application, the color pattern of the CFA in the image sensor can include the following:

[0056] 1. Red, Green, Blue and White (RGBW)

[0057] On the basis of the RGB CFA, part of the R pixels, G pixels or B pixels are reduced, and one W pixel is added to form a mosaic pattern, for example, Figure 2 shows a pattern schematic diagram of an RGBW CFA, as shown in Figure 2, the pattern adopts a 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes one W pixel, and the other three pixels are R pixels, G pixels and B pixels respectively.

[0058] 2. RGB2W

[0059] On the basis of the RGB CFA, part of the R pixels, G pixels or B pixels are reduced, and two W pixels are added to form a mosaic pattern, for example, Figures 3a and 3b respectively show a pattern schematic diagram of an RGB2W CFA, as shown in Figure 3a, the pattern adopts a 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes two W pixels, and the other two pixels are R pixels and G pixels or B pixels and G pixels; as shown in Figure 3b, the pattern adopts a 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes two W pixels, and the other two pixels are R pixels, G pixels or B pixels.

[0060] 3. RGB3W

[0061] On the basis of the RGB CFA, part of the R pixels, G pixels or B pixels are reduced, and three W pixels are added to form a mosaic pattern, for example, Figure 4 shows a pattern schematic diagram of an RGB3W CFA, as shown in Figure 4, the pattern adopts a 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes three W pixels, and the other one pixel is an R pixel, a G pixel or a B pixel.

[0062] 4. MYCW

[0063] On the basis of MYC CFA, some C pixels, M pixels or Y pixels are reduced, and one W pixel is added to form a mosaic pattern. For example, Fig. 5 shows a pattern diagram of MYCW CFA. As shown in Fig. 5, the pattern adopts 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes one W pixel, and the other three pixels are C pixel, M pixel and Y pixel respectively.

[0064] 5. MYC2W

[0065] On the basis of MYC CFA, some C pixels, M pixels or Y pixels are reduced, and two W pixels are added to form a mosaic pattern. For example, Fig. 6a and Fig. 6b respectively show a pattern diagram of MYC2W CFA. As shown in Fig. 6a, the pattern adopts 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes two W pixels, and the other two pixels are C pixel and M pixel respectively. As shown in Fig. 6b, the pattern adopts 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes two W pixels, and the other two pixels are C pixel, M pixel or Y pixel.

[0066] 6. MYC3W

[0067] On the basis of MYC CFA, some C pixels, M pixels or Y pixels are reduced, and three W pixels are added to form a mosaic pattern. For example, Fig. 7 shows a pattern diagram of MYC3W CFA. As shown in Fig. 7, the pattern adopts 2x2 array arrangement, and one 2x2 array (the smallest repeating unit) includes three W pixels, and the other one pixel is C pixel, M pixel or Y pixel.

[0068] On the basis of the pattern of the CFA in the above-mentioned image sensors, the visible light pixels (including R pixel, G pixel, B pixel, C pixel, M pixel or Y pixel) or W pixel can be replaced by a supervisory pixel.

[0069] The supervising pixel is similar to the visible light pixel, and is internally provided with a stop Infra-red (SIR) structure, so that it can receive part or all of the visible light, but does not receive the Infra-red (IR) light. For example, the R pixel can receive the red light, but does not receive the IR light; the G pixel can receive the green light, but does not receive the IR light; the B pixel can receive the blue light, but does not receive the IR light; the C pixel can receive the cyan light, but does not receive the IR light; the M pixel can receive the magenta light, but does not receive the IR light; the Y pixel can receive the yellow light, but does not receive the IR light; and the supervising pixel can receive all of the visible light, but does not receive the IR light.

[0070] For example, FIGS. 8a and 8b respectively show a pattern diagram of an RGB2W CFA containing a supervising pixel. As shown in FIG. 8a, the G pixel located on the top is replaced by the supervising pixel on the basis of the pattern shown in FIG. 3b; and as shown in FIG. 8b, the G pixel located on the bottom is replaced by the supervising pixel on the basis of the pattern shown in FIG. 3b.

[0071] It should be noted that the above embodiment exemplarily shows several color patterns of the CFA in the image sensor, but this does not constitute a limitation on the pattern, and other patterns can also be applied to the present application, and the present application does not specifically limit the arrangement mode of the pattern.

[0072] Based on the color pattern of the CFA in the image sensor, the present application provides an image sensor, which can split the light (including the un-received visible light or IR light) not received by the visible light pixel to the adjacent W pixel, so as to increase the light amount of the W pixel, and improve the imaging quality in the low-illumination environment; and / or split the light (including the un-received visible light) not received by the W pixel to the adjacent visible light pixel, so as to increase the light amount of the visible light pixel, and improve the imaging quality in the low-illumination environment.

[0073] The image sensor can be applied to mobile phones, monitoring cameras, security access control systems and other devices, and fields that require the use of color images and IR or W black and white images for image processing at the same time, such as photography, video shooting, monitoring, etc. Typical applications can include visible light and / or infrared light-based living body detection, night video monitoring, color-black and white dynamic fusion, etc. The terminal device to which the image sensor of the present application is applicable can also be referred to as user equipment (UE), which can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; can also be deployed on the water surface (such as ships, etc.); can also be deployed in the air (such as airplanes, balloons and satellites, etc.). The terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) device, an augmented reality (AR) device, a monitoring device, a smart large screen, a smart television, a wireless device in remote medical treatment or a wireless device in a smart home, etc., which are not limited in the present application.

[0074] FIG. 9a is a schematic structural diagram of a longitudinal section of an image sensor 900 of the present application, as shown in FIG. 9a, the CFA of the image sensor 900 includes four pixels: a first pixel, a second pixel, a third pixel and a W pixel. The present embodiment takes the first pixel as an R pixel, the second pixel as a G pixel, and the third pixel as a B pixel as an example for illustration. It should be understood that the aforementioned first pixel, second pixel and third pixel can also be a C pixel (referring to a B pixel), an M pixel (referring to an R pixel), and a Y pixel (referring to a G pixel), respectively, and the technical principles in the two cases are similar.

[0075] A first light splitting structure 901A is arranged on the surface of the first pixel, a second light splitting structure 901B is arranged on the surface of the second pixel, and a third light splitting structure 901C is arranged on the surface of the third pixel. Among them,

[0076] The first light splitting structure 901A is used to split the first light signal not received by the first pixel to the W pixel adjacent to the first pixel;

[0077] The second light splitting structure 901B is used to split the second light signal not received by the second pixel to the W pixel adjacent to the second pixel;

[0078] The third light splitting structure 901C is used to split the third light signal not received by the third pixel to the W pixel adjacent to the third pixel.

[0079] Optionally, the first light splitting structure 901A, the second light splitting structure 901B and the third light splitting structure 901C are respectively used to realize light signal distribution in at least one of the upward, downward, leftward or rightward directions. For example, the first light splitting structure 901A distributes the first light signal to the W pixels adjacent to the first pixel in the up, down, left and right directions; the second light splitting structure 901B distributes the second light signal to the W pixels adjacent to the second pixel in the up-down direction or the left-right direction; and the third light splitting structure 901C distributes the third light signal to the W pixels adjacent to the third pixel in the up, down, left and right directions.

[0080] For example, FIGS. 10a-10d are light splitting diagrams of the light splitting structure of the present application. As shown in FIG. 10a, taking the CFA pattern of an RGB2W sensor as an example, a light splitting structure is designed in a 4x4 pixel area (four regions), which can adopt a meta surface design scheme, for example, to improve the light intake of the W pixels in the 4x4 pixel area. Taking the pixels in the "cross" area shown by the thick arrows as an example, the middle of the area is a W pixel, the upper side of the W pixel is a G pixel, the lower side of the W pixel is an R pixel, the left side of the W pixel is a B pixel, and the right side of the W pixel is a G pixel.

[0081] As shown in FIG. 10b, from a top-down perspective, each pixel in the above-mentioned "cross" area is divided into four sub-areas. Among them, the surfaces of the four sub-areas of the middle W pixel are not provided with a meta surface (indicated by No St.); among the four sub-areas of the upper G pixel, the surfaces of the upper and lower two sub-areas in the vertical direction are provided with a meta surface C (indicated by St. C), and the surfaces of the left and right two sub-areas in the horizontal direction are provided with a meta surface B (indicated by St. B); the surfaces of the four sub-areas of the lower R pixel are provided with a meta surface D (indicated by St. D); the surfaces of the four sub-areas of the left B pixel are provided with a meta surface A (indicated by St. A); and among the four sub-areas of the right G pixel, the surfaces of the upper and lower two sub-areas in the vertical direction are provided with a meta surface C (indicated by St. C), and the surfaces of the left and right two sub-areas in the horizontal direction are provided with a meta surface B (indicated by St. B).

[0082] As shown in FIG. 10c, from a side view along the horizontal direction of the above-mentioned "cross" area, including the left B pixel, the middle W pixel and the right G pixel, from top to bottom, there are a meta surface, a color filter (CF) and a photodiode (PD); among them,

[0083] The super surface of the B pixel is St. A, which can divide the visible light signal (including red light, green light and blue light) irradiated thereon into two paths, one of which is blue light, which is vertically incident on the CF of the B pixel without being divided, and then is incident on the PD of the B pixel after being transmitted through the CF; and the other path includes red light and green light, which are divided into the left and right adjacent W pixels, and are incident on the CF of the left and right adjacent W pixels, and then are incident on the PD of the W pixel after being transmitted through the CF.

[0084] The W pixel is not provided with a super surface, and the visible light signal (including red light, green light and blue light) irradiated thereon is directly incident on the CF of the W pixel, and then is incident on the PD of the W pixel after being transmitted through the CF.

[0085] The super surface of the G pixel is St. B in the lateral direction, which can divide the visible light signal (including red light, green light and blue light) irradiated thereon into three paths, one of which is green light, which is vertically incident on the CF of the G pixel without being divided, and then is incident on the PD of the G pixel after being transmitted through the CF; another path is blue light, which is divided into the left and right adjacent W pixels, and is incident on the CF of the left and right adjacent W pixels, and then is incident on the PD of the W pixel after being transmitted through the CF; and the other path is red light, which is vertically incident on the CF of the G pixel without being divided, and is filtered by the CF.

[0086] As shown in FIG. 10d, in the side view along the longitudinal direction of the above-mentioned "cross" area, the G pixel above, the W pixel in the middle and the R pixel below are included from top to bottom, including the super surface, the CF and the PD; wherein,

[0087] The super surface of the G pixel is St. C in the longitudinal direction, which can divide the visible light signal (including red light, green light and blue light) irradiated thereon into three paths, one of which is green light, which is vertically incident on the CF of the G pixel without being divided, and then is incident on the PD of the G pixel after being transmitted through the CF; another path is red light, which is divided into the upper and lower adjacent W pixels, and is incident on the CF of the upper and lower adjacent W pixels, and then is incident on the PD of the W pixel after being transmitted through the CF; and the other path is blue light, which is vertically incident on the CF of the G pixel without being divided, and is filtered by the CF.

[0088] The W pixel is not provided with a super surface, and the visible light signal (including red light, green light and blue light) irradiated thereon is directly incident on the CF of the W pixel, and then is incident on the PD of the W pixel after being transmitted through the CF.

[0089] The super surface of the R pixel is St. D, which can divide the visible light signal (including red light, green light and blue light) irradiated thereon into two paths, one of which is red light, which is vertically incident on the CF of the R pixel without being divided, and then is incident on the PD of the R pixel after being transmitted through the CF; and the other path includes green light and blue light, which are divided into the upper and lower adjacent W pixels, and are incident on the CF of the upper and lower adjacent W pixels, and then are incident on the PD of the W pixel after being transmitted through the CF.

[0090] Therefore, based on the above light splitting structure, the light signal not received by the pixel can be split and distributed to the upper and lower adjacent and / or left and right adjacent pixels, and the direction of light splitting can be determined by adjusting the design of the light splitting structure, so as to purposefully increase the light amount of the adjacent pixels, thereby improving the imaging quality in the low-illumination environment.

[0091] For example, FIG. 11 is a light splitting schematic diagram of the RGB2W sensor of the present application. As shown in FIG. 11, the R pixel does not receive green light and blue light, and the green light and blue light can be distributed to the upper and lower and left and right adjacent W pixels through the first light splitting structure 901A; the G pixel does not receive red light and blue light, and the red light and blue light can be distributed to the upper and lower adjacent or left and right adjacent W pixels through the second light splitting structure 901B; the B pixel does not receive red light and green light, and the red light and green light can be distributed to the upper and lower and left and right adjacent W pixels through the third light splitting structure 901C. In this way, one W pixel can receive 2 portions of red light, 2 portions of green light and 2 portions of blue light. One portion of light can increase the light amount by 25% (i.e., the light is split into 4 portions, and one portion of the light amount accounts for one fourth of the original light), so that the light amount of the red light received by the W pixel is increased by 50%, the light amount of the green light is increased by 50%, and the light amount of the blue light is increased by 50%.

[0092] It can be seen that through the above image sensor, the light amount of the W pixel can be greatly increased, the details in the image obtained based on the W pixel are improved, and thus the imaging quality in the low-illumination environment is improved.

[0093] In a possible implementation, FIG. 9b is an exemplary structural schematic diagram of a longitudinal section of the image sensor 900 of the present application. As shown in FIG. 9b, based on the image sensor 900 shown in FIG. 9a, the W pixel further includes a fourth light splitting structure 901D; the fourth light splitting structure is used to implement at least one of the following light splitting operations:

[0094] distributing the fourth light signal not received by the W pixel to a first pixel adjacent to the W pixel;

[0095] or, distributing the fifth light signal not received by the W pixel to a second pixel adjacent to the W pixel;

[0096] or, distributing the sixth light signal not received by the W pixel to a third pixel adjacent to the W pixel.

[0097] Optionally, the fourth light splitting structure 901D can be similar to the first light splitting structure 901A, the second light splitting structure 901B and the third light splitting structure 901C, and is used to implement the light signal distribution in at least one of the upward, downward, leftward or rightward directions.

[0098] Based on the image sensor shown in FIG. 9a, the light receiving amount of the R / G / B pixels can be increased due to the received light signal from the W pixel, and the color in the image obtained based on the R / G / B pixels can be enhanced, thereby further improving the imaging quality in a low-illumination environment.

[0099] In a possible implementation, the image sensor 900 described above further includes the following structure:

[0100] The red filter layer 902R is arranged on the R pixel, the green filter layer 902G is arranged on the G pixel, and the blue filter layer 902B is arranged on the B pixel. Each pixel includes the filter layer and the charge reading module 903.

[0101] For example, FIG. 12 shows a graph of a light sensing characteristic of an image sensor. As shown in FIG. 12, in the image sensor, the R pixel has a light sensing intensity peak in the wavelength range of red light (around 650 nm), the G pixel has a light sensing intensity peak in the wavelength range of green light (around 550 nm), and the B pixel has a light sensing intensity peak in the wavelength range of blue light (around 450 nm). In addition, the R pixel, the G pixel, and the B pixel each have another light sensing intensity peak in the wavelength range of infrared light (around 850 nm or 910 nm). Therefore, the red filter layer 902R, the green filter layer 902G, and the blue filter layer 902B each allow the light signal at the above two light sensing intensity peaks corresponding to the respective color to pass through.

[0102] As shown in FIG. 12, the light sensing characteristic of the visible light and the infrared light passing through the red filter layer 902R in the R pixel is shown by the thin black solid line R, and the R pixel has two light sensing intensity peaks in the wavelength range of red light 650 nm and the wavelength range of infrared light 850 nm. The light sensing characteristic of the visible light and the infrared light passing through the green filter layer 902G in the G pixel is shown by the short dashed line G, and the G pixel has two light sensing intensity peaks in the wavelength range of green light 550 nm and the wavelength range of infrared light 850 nm. The light sensing characteristic of the visible light and the infrared light passing through the blue filter layer 902B in the B pixel is shown by the dotted line B, and the B pixel has two light sensing intensity peaks in the wavelength range of blue light 450 nm and the wavelength range of infrared light 850 nm. Therefore, the red filter layer 902R can simultaneously pass through the red light and the infrared light, the green filter layer 902G can simultaneously pass through the green light and the infrared light, and the blue filter layer 902B can simultaneously pass through the blue light and the infrared light.

[0103] When only the visible light is present, the red filter layer 902R can pass through the red light, the green filter layer 902G can pass through the green light, and the blue filter layer 902B can pass through the blue light.

[0104] When only the infrared light is present, the red filter layer 902R, the green filter layer 902G, and the blue filter layer 902B can pass through the infrared light, respectively.

[0105] Optionally, an infrared light cut-off filter layer 904 (which can adopt the SIR structure described above) is arranged in the R pixel, the G pixel and the B pixel. The infrared light cut-off filter layer 904 can block light rays exceeding 650 nm or block infrared light of a certain specific waveband (for example, 850 nm light rays), so that infrared light cannot pass through the infrared light cut-off filter layer 904.

[0106] The present application arranges the infrared light cut-off filter layer 904 in the R pixel, the G pixel and the B pixel, cuts off the infrared light reaching the R pixel, the G pixel and the B pixel, removes the IR signal in the photosensitive result of the visible light pixel, makes the color of the photosensitive result more accurate, and improves the photosensitive effect of the sensor.

[0107] When only visible light is present, the red filter layer 902R can transmit red light, the green filter layer 902G can transmit green light, and the blue filter layer 902B can transmit blue light. At this time, the R pixel receives red light, and green light and blue light (first light signal) that are not received are divided by the first light splitting structure 901A to the adjacent W pixel; the G pixel receives green light, and red light and blue light (second light signal) that are not received are divided by the second light splitting structure 901B to the adjacent W pixel; and the B pixel receives blue light, and red light and green light (third light signal) that are not received are divided by the third light splitting structure 901C to the adjacent W pixel. Correspondingly, the W pixel can receive all the visible light irradiated thereon, and can also receive the visible light from the foregoing light splitting. In this case, the infrared light cut-off filter layer 904 can not be arranged.

[0108] When only infrared light is present, the red filter layer 902R, the green filter layer 902G and the blue filter layer 902B cannot transmit infrared light under the action of the infrared light cut-off filter layer 904. At this time, the R pixel, the G pixel and the B pixel cannot receive infrared light, the R pixel can divide infrared light (first light signal) by the first light splitting structure 901A to the adjacent W pixel, the G pixel can divide infrared light (second light signal) by the second light splitting structure 901B to the adjacent W pixel, and the B pixel can divide infrared light (third light signal) by the third light splitting structure 901C to the adjacent W pixel. Correspondingly, the W pixel can receive infrared light irradiated thereon, and can also receive the infrared light from the foregoing light splitting, which can improve the amount of incoming infrared light of the W pixel and reduce the power of the infrared light supplement lamp.

[0109] When there is visible light and infrared light at the same time, under the action of the infrared light cut-off filter layer 904, the red filter layer 902R only transmits red light, the green filter layer 902G only transmits green light, and the blue filter layer 902B only transmits blue light. Because of the addition of the infrared light cut-off filter layer 904, the red filter layer 902R, the green filter layer 902G, and the blue filter layer 902B cannot transmit infrared light. At this time, in one case, the R pixel receives red light, and the green light, blue light, and infrared light (first light signal) that are not received are divided by the first light splitting structure 901A to the adjacent W pixel; the G pixel receives green light, and the red light, blue light, and infrared light (second light signal) that are not received are divided by the second light splitting structure 901B to the adjacent W pixel; and the B pixel receives blue light, and the red light, green light, and infrared light (third light signal) that are not received are divided by the third light splitting structure 901C to the adjacent W pixel. In another case, the R pixel receives red light, and the infrared light (first light signal) that is not received is divided by the first light splitting structure 901A to the adjacent W pixel, and the green light and blue light that are not received are filtered out by the filter in the R pixel; the G pixel receives green light, and the infrared light (second light signal) that is not received is divided by the second light splitting structure 901B to the adjacent W pixel, and the red light and blue light that are not received are filtered out by the filter in the G pixel; and the B pixel receives blue light, and the infrared light (third light signal) that is not received is divided by the third light splitting structure 901C to the adjacent W pixel, and the red light and green light that are not received are filtered out by the filter in the B pixel. Correspondingly, the W pixel can receive the visible light and infrared light from the foregoing light splitting, or receive the infrared light from the foregoing light splitting, which can improve the amount of infrared light entering the W pixel, reduce the power of the infrared light supplement lamp, and even improve the amount of visible light entering the W pixel.

[0110] In addition, when there is visible light and infrared light at the same time, the W pixel can receive the infrared light irradiated thereon, and the visible light irradiated thereon is divided by the fourth light splitting structure 901D to the adjacent R pixel, G pixel, or B pixel. That is, the red light (fourth light signal) is divided to the adjacent R pixel, the green light (fifth light signal) is divided to the adjacent G pixel, and the blue light (sixth light signal) is divided to the adjacent B pixel. In this case, the amount of visible light entering the R / G / B pixel can be improved, and in combination with the improvement of the amount of light entering the W pixel as described above, the best imaging effect can be achieved.

[0111] Each pixel in the image sensor includes a light sensing device, which can be a photodiode, for example, for converting a light signal into an electrical signal or converting a light signal into an electric charge.

[0112] The charge readout module 903 is configured to read out the charge accumulated by the photosensitive device and output to a subsequent image processing circuit or image processor. The charge readout module is similar to a buffer area, and the charge accumulated by the photosensitive device is transferred and temporarily buffered in the charge readout module, and the charge signal of the corresponding pixel is output under the control of the readout signal. The charge readout module 903 can output visible light image data and / or infrared light image data.

[0113] In a possible implementation, FIG. 9c is a schematic structural diagram of a longitudinal section of the image sensor 900 of the present application. As shown in FIG. 9c, based on the image sensor 900 shown in FIG. 9a, further comprising a supervisory pixel, and the surface of the supervisory pixel is provided with a fifth light splitting structure 901E. The fifth light splitting structure 901E is configured to split the infrared light not received by the supervisory pixel to the W pixel adjacent to the supervisory pixel. At this time, the fourth light splitter 901D is further configured to split the seventh light signal not received by the W pixel to the supervisory pixel adjacent to the W pixel.

[0114] The infrared light cutoff filter layer 904 (which can adopt the SIR structure described above) is also arranged in the supervisory pixel. The infrared light cutoff filter layer 904 can block light exceeding 650 nm, so that the infrared light cannot pass through the infrared light cutoff filter layer 904.

[0115] When there is visible light and infrared light at the same time, under the action of the infrared light cutoff filter layer 904, the supervisory pixel only receives the visible light irradiated thereon, and the infrared light (the seventh light signal) not received is split to the adjacent W pixel through the fifth light splitting structure 901E.

[0116] Based on the image sensor shown in FIG. 9b, the amount of light entering the W pixel is further increased, the details in the image obtained based on the W pixel are improved, and thus the imaging quality in a low-illumination environment is improved.

[0117] In a possible implementation, FIG. 9d is a schematic structural diagram of a longitudinal section of the image sensor 900 of the present application. As shown in FIG. 9d, the difference from the image sensor 900 shown in FIG. 9a is that only the W pixel is provided with the fourth light splitting structure 901D, and the R / G / B pixel is not provided with the light splitting structure. The fourth light splitting structure is configured to perform at least one of the following light splitting operations:

[0118] Splitting the fourth light signal not received by the W pixel to the first pixel adjacent to the W pixel;

[0119] Or, splitting the fifth light signal not received by the W pixel to the second pixel adjacent to the W pixel;

[0120] Or, splitting the sixth light signal not received by the W pixel to the third pixel adjacent to the W pixel.

[0121] Optionally, the fourth light splitting structure 901D can be similar to the first light splitting structure 901A, the second light splitting structure 901B and the third light splitting structure 901C, and is used to realize light signal distribution in at least one of the upward, downward, leftward or rightward directions.

[0122] The image sensor described above can increase the light quantity of the R / G / B pixels, enhance the color in the image obtained based on the R / G / B pixels, and thus improve the imaging quality in a low-illumination environment.

[0123] It should be understood that the image sensor shown in FIGS. 9a-9d is only an example, and the design and position of the structures such as the red filter layer 902R, the green filter layer 902G, the blue filter layer 902B, the charge reading module 903 and the infrared light cutoff filter layer 904 can also be implemented in other ways, which are not specifically limited.

[0124] The image sensor described above is further described below through specific embodiments.

[0125] FIGS. 13a-13e are schematic diagrams of light splitting of the RGB2W sensor of the present application. In the present embodiment, a 4x4 pixel region (four regions) in the middle of the diagram is taken as an example, and the light splitting from the R / G / B pixels to the W pixels is realized in combination with the meta surface design in the region. In the present embodiment, the light signal on the RGB2W sensor is visible light including red light, green light and blue light.

[0126] As shown in FIG. 13a, for the blue light (blue waveband), the R pixel can split the blue light to the adjacent W pixels in the upward, downward, leftward and rightward directions, and the blue light not received by the R pixel is distributed to the aforementioned four W pixels to improve the light quantity of the W pixels; the G pixel splits the blue light to the adjacent W pixels in the upward and downward directions or the leftward and rightward directions to improve the light quantity of the W pixels, so as to ensure the response balance of the W channel; and the B pixel can receive the blue light. In this way, each W pixel can obtain the blue light distributed from the RG two pixels. Optionally, as shown in FIG. 13d, the G pixel can also split the blue light to the adjacent W pixels in the upward, downward, leftward and rightward directions, i.e., some W pixels receive two blue light distributions, some W pixels receive three blue light distributions, and some W pixels receive four blue light distributions.

[0127] As shown in FIG. 13b, for green light (green waveband), the R pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the green light not received by the R pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; the B pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the green light not received by the B pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; the G pixel can receive green light. In this way, each W pixel can obtain green light split from the R and B pixels.

[0128] As shown in FIG. 13c, for red light (red waveband), the B pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the red light not received by the B pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; the G pixel splits light to the adjacent W pixels in the two directions of up and down or left and right only to ensure the response balance of the W channel, and the red light not received by the G pixel is split to the aforementioned two W pixels to improve the light quantity of the W pixels; the R pixel can receive red light. In this way, each W pixel can obtain red light split from the G and B pixels. Optionally, as shown in FIG. 13e, the G pixel can also split light to the adjacent W pixels in the four directions of up, down, left and right, that is, some W pixels receive 2 red light split, some W pixels receive 3 red light split, and some W pixels receive 4 red light split.

[0129] As shown in FIG. 11, when the G pixel splits light to the adjacent W pixels in the two directions of up and down or left and right only, the light quantity of the W pixels can be improved by 50%, wherein the red light is improved by 25%*2, the green light is improved by 25%*2, and the blue light is improved by 25%*2.

[0130] When the G pixel splits light to the adjacent W pixels in the four directions of up, down, left and right, the light quantity of the W pixels can be improved, wherein the red light is improved by 25%*2-25%*4, the green light is improved by 25%*2, and the blue light is improved by 25%*2-25%*4.

[0131] FIGS. 14a-14f are schematic diagrams of light splitting of the RGB3W sensor of the present application. The present embodiment takes the 4*4 pixel region (four regions) in the middle of the diagram as an example, and realizes light splitting from the R / G / B pixels to the W pixels in combination with the meta surface design in the region. The light signal on the RGB2W sensor in the present embodiment is visible light including red light, green light and blue light.

[0132] As shown in FIG. 14a, for blue light (blue waveband), the R pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the blue light not received by the R pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; the G pixel splits light to the adjacent W pixels in the two directions of up and down or left and right to ensure the response balance of the W channel, and the blue light not received by the G pixel is split to the aforementioned two W pixels to improve the light quantity of the W pixels; and the B pixel can receive blue light. In this way, each W pixel can obtain blue light split from the RG two pixels. Alternatively, as shown in FIG. 14e, the G pixel can also split light to the adjacent W pixels in the four directions of up, down, left and right, that is, some W pixels receive one blue light split, and some W pixels receive two blue light splits.

[0133] As shown in FIG. 14b, for green light (green waveband), the R pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the green light not received by the R pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; the B pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the green light not received by the B pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; and the G pixel can receive green light. In this way, each W pixel can obtain green light split from the RB two pixels.

[0134] As shown in FIG. 14c, for red light (red waveband), the B pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, and the red light not received by the B pixel is split to the aforementioned four W pixels to improve the light quantity of the W pixels; the G pixel splits light to the adjacent W pixels in the two directions of up and down or left and right to ensure the response balance of the W channel, and the red light not received by the G pixel is split to the aforementioned two W pixels to improve the light quantity of the W pixels; and the R pixel can receive red light. In this way, each W pixel can obtain red light split from the GB two pixels. Alternatively, as shown in FIG. 14f, the G pixel can also split light to the adjacent W pixels in the four directions of up, down, left and right, that is, some W pixels receive one red light split, and some W pixels receive two red light splits.

[0135] As shown in FIG. 14d, when the G pixel splits light to the adjacent W pixels in the two directions of up and down or left and right, there is one B pixel, one R pixel and two G pixels in the 4x4 pixel area, and the light quantity of the W pixels can be improved by 12.5%, wherein the red light is improved by 25% / 50%, the green light is improved by 25% / 50%, and the blue light is improved by 25% / 50%.

[0136] When the G pixel splits light to the adjacent W pixels in the four directions of up, down, left and right, the light quantity of the W pixels can be improved, wherein the red light is improved by 25%, the green light is improved by 16.7%, and the blue light is improved by 25%.

[0137] Fig. 15a-15e are schematic diagrams of light splitting of RGB2W sensor of the present application, the present embodiment takes the 4x4 pixel area (four regions) in the middle of the diagram as an example, and realizes light splitting from R / G / B pixels to W pixels in combination with the meta surface design in the area. The light signal hitting on the RGB2W sensor in the present embodiment is visible light including red light, green light and blue light, and infrared light (IR).

[0138] As shown in Fig. 15a, for blue light (blue band), B pixels can receive blue light; W pixels receive IR (IR band), which can split light to the adjacent B pixels in the up, down, left and right four directions, i.e. the blue light of the W pixels adjacent to the B pixels in the up, down, left and right directions is divided to the B pixels to improve the light intake of the B pixels.

[0139] As shown in Fig. 15b, for green light (green band), G pixels can receive green light; W pixels receive IR (IR band), which can split light to the adjacent G pixels in the up, down, left and right four directions, i.e. the green light of the W pixels adjacent to the G pixels in the up, down, left and right directions is divided to the G pixels to improve the light intake of the G pixels.

[0140] In addition, W pixels receive IR, which can split light to the adjacent supervisory pixels in the up, down, left and right four directions, i.e. the red light, green light or blue light of the W pixels adjacent to the supervisory pixels in the up, down, left and right directions is divided to the supervisory pixels to improve the light intake of the supervisory pixels.

[0141] As shown in Fig. 15c, for red light (red band), R pixels can receive red light; W pixels receive IR (IR band), which can split light to the adjacent R pixels in the up, down, left and right four directions, i.e. the red light of the W pixels adjacent to the R pixels in the up, down, left and right directions is divided to the R pixels to improve the light intake of the R pixels.

[0142] As shown in Fig. 15d, for IR, since R / G / B pixels and supervisory pixels only receive visible light, IR will be filtered, therefore, in addition to that W pixels receive IR, IR hitting on R / G / B pixels and supervisory pixels can also be split to the adjacent W pixels in the up, down, left and right four directions, i.e. the IR of the W pixels adjacent to the R / G / B pixels and supervisory pixels in the up, down, left and right directions is divided to the W pixels to improve the light intake of the W pixels.

[0143] Optionally, IR hitting on R / G / B pixels and supervisory pixels can also not be split to W pixels.

[0144] As shown in Fig. 15e, when visible light and infrared light hit on the CFA of RGB2W Sensor, the light intake of R pixels in the 4x4 pixel area can be improved by 100%, the light intake of G pixels can be improved by 100%, the light intake of B pixels can be improved by 100%, and the IR light intake of W pixels can be improved by 100% or 0%.

[0145] Fig. 16a-16c are light splitting schematic diagrams of the MYC2W sensor of the present application, the present embodiment takes the 4x4 pixel area (four areas) in the middle of the diagram as an example, and realizes light splitting from M / Y / C pixels to W pixels in combination with the meta surface design in the area. The light signal hitting the MYC2W sensor in the present embodiment is visible light including red light, green light and blue light.

[0146] As shown in Fig. 16a, for blue light (blue band), Y pixels can split light to the adjacent W pixels in the up, down, left and right four directions, and the blue light not received by the Y pixels is divided into the aforementioned four W pixels to improve the light amount of the W pixels; C pixels and M pixels can receive blue light. In this way, each W pixel can obtain blue light from two Y pixels.

[0147] As shown in Fig. 16b, for green light (green band), M pixels can split light to the adjacent W pixels in the up, down, left and right four directions, and the green light not received by the M pixels is divided into the aforementioned four W pixels to improve the light amount of the W pixels; C pixels and Y pixels can receive green light. In this way, each W pixel can obtain green light from two M pixels.

[0148] As shown in Fig. 16c, for red light (red band), C pixels can split light to the adjacent W pixels in the up, down, left and right four directions, and the red light not received by the C pixels is divided into the aforementioned four W pixels to improve the light amount of the W pixels; M pixels and Y pixels can receive red light. In this way, each W pixel can obtain red light from two C pixels.

[0149] In this way, the light amount of the W pixels can be improved, wherein the red light is improved by 25%, the green light is improved by 25%, and the blue light is improved by 50%.

[0150] Fig. 17a-17c are light splitting schematic diagrams of the MYC3W sensor of the present application, the present embodiment takes the 4x4 pixel area (four areas) in the middle of the diagram as an example, and realizes light splitting from M / Y / C pixels to W pixels in combination with the meta surface design in the area. The light signal hitting the MYC3W sensor in the present embodiment is visible light including red light, green light and blue light.

[0151] As shown in Fig. 17a, for blue light (blue band), Y pixels can split light to the adjacent W pixels in the up, down, left and right four directions, and the blue light not received by the Y pixels is divided into the aforementioned four W pixels to improve the light amount of the W pixels; C pixels and M pixels can receive blue light. In this way, each W pixel can obtain blue light from one Y pixel.

[0152] As shown in FIG. 17b, for green light (green waveband), the M pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, i.e., the green light not received by the M pixel is divided into the aforementioned four W pixels to improve the light amount of the W pixel.

[0153] As shown in FIG. 17c, for red light (red waveband), the C pixel can split light to the adjacent W pixels in the four directions of up, down, left and right, i.e., the red light not received by the C pixel is divided into the aforementioned four W pixels to improve the light amount of the W pixel; the M pixel and the Y pixel can receive red light. In this way, each W pixel can obtain red light from a C pixel.

[0154] In this way, the light amount of the W pixel can be improved, wherein the red light is improved by 12.5%, the green light is improved by 12.5%, and the blue light is improved by 25%.

[0155] FIGS. 18a-18d are schematic diagrams of light splitting of the MYC2W sensor of the present application. In the present embodiment, a 4x4 pixel region (four regions) in the middle of the diagram is taken as an example, and light splitting from the M / Y / C pixel to the W pixel is realized in combination with the meta surface design in the region. The light signal on the MYC2W sensor in the present embodiment is visible light including red light, green light and blue light, and infrared light (IR).

[0156] As shown in FIG. 18a, for blue light and green light, the C pixel can receive blue light and green light; the W pixel receives IR (IR waveband) and can split light to the adjacent C pixels in the four directions of up, down, left and right, i.e., the blue light and green light of the W pixel adjacent to the C pixel in the up, down, left and right directions is divided into the C pixel to improve the light amount of the C pixel.

[0157] As shown in FIG. 18b, for red light and green light, the Y pixel can receive red light and green light; the W pixel receives IR (IR waveband) and can split light to the adjacent Y pixels in the four directions of up, down, left and right, i.e., the red light and green light of the W pixel adjacent to the Y pixel in the up, down, left and right directions is divided into the Y pixel to improve the light amount of the Y pixel.

[0158] As shown in FIG. 18c, for red light and blue light, the M pixel can receive red light and blue light; the W pixel receives IR (IR waveband) and can split light to the adjacent M pixels in the four directions of up, down, left and right, i.e., the red light and blue light of the W pixel adjacent to the M pixel in the up, down, left and right directions is divided into the M pixel to improve the light amount of the M pixel.

[0159] As shown in FIG. 18d, for IR, since M / Y / C pixels only receive visible light, IR will be filtered, therefore, in addition to W pixels receiving IR, IR directed to M / Y / C pixels can be split to adjacent W pixels in up, down, left and right directions, i.e., IR of M / Y / C pixels adjacent to W pixels in up, down, left and right directions is split to W pixels to improve the light intake of W pixels.

[0160] Optionally, IR directed to M / Y / C pixels can not be split to W pixels.

[0161] When both visible light and infrared light hit the CFA of MYC2W Sensor, the light intake of M pixels can be improved by 100% in a 4x4 pixel area, the light intake of C pixels can be improved by 100%, the light intake of Y pixels can be improved by 100%, and the IR light intake of W pixels can be improved by 100% or 0%.

[0162] FIG. 19 is a schematic structural diagram of an image acquisition device according to an embodiment of the present application. As shown in FIG. 19, the image acquisition device includes a lens 1901, an image sensor 1902, an image signal processor (ISP) module 1903, an image fusion module 1904, an infrared light driving control module 1905, and an infrared light supplement lamp 1906.

[0163] The lens 1901 is configured to capture a still image or a video, collect light signals reflected by an object to be photographed, and transmit the collected light signals to the image sensor 1902.

[0164] The image sensor 1902 can be the image sensor provided in the above embodiments, which can generate raw image data (including visible light image data and / or infrared light image data) of the object to be photographed according to the light signals.

[0165] The ISP module 1903 is configured to obtain first image data, which is obtained based on W pixels in the image sensor, for example, the above-mentioned infrared light image data. The W pixels receive IR, and full resolution is completed by an interpolation algorithm, and then the W pixels are subjected to noise reduction and other processing. The W pixels have no color information, and provide detail information for a final image. The greater the light intake of the W pixels, the better the details. The ISP module 1903 is also configured to obtain second image data, which is obtained based on R pixels, G pixels and B pixels in the image sensor, for example, the above-mentioned visible light image data. Color noise and pseudo-color of the color image collected by the R / G / B pixels are removed by DM, and the problem of color noise and pseudo-color is reduced. When there is infrared light, IR needs to be removed to eliminate the interference of IR. The greater the light intake of the R / G / B pixels, the smaller the color noise, and the better the color accuracy after fusion.

[0166] The image fusion module 1904 is configured to fuse the first image data and the second image data to obtain a target image.

[0167] The infrared lamp driving control module 1905 is configured to control the infrared light compensation lamp 1906 according to the light intensity of the infrared light compensation lamp configured by the ISP module 1903.

[0168] The infrared light compensation lamp 1906 is configured to provide infrared light.

[0169] Optionally, the image acquisition device can adopt a single-lens plus single-image sensor structure, or a double-lens plus double-image sensor structure, or a single-lens plus light splitting plate and double-image sensor structure. The single-lens structure can save cost, and the single-image sensor structure can simplify the structure of the camera. The present application does not make specific limitations thereto.

[0170] In the implementation process, each step of the above method embodiments can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware coding executed by the processor to complete, or executed by a combination of hardware and software modules in the coding processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the storage memory, and the processor reads the information in the storage memory, and combines the hardware to complete the steps of the above method.

[0171] The memory mentioned in each of the above embodiments can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the system and method described herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0172] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0174] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0175] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0176] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically separate unit, or two or more units can be integrated into a unit.

[0177] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various program codes that can be stored in the medium.

[0178] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image sensor, characterized by, include: A color filter array (CFA), wherein the CFA includes a first pixel, a second pixel, a third pixel, and a W pixel; wherein, The surface of the first pixel is provided with a first beam-splitting structure, the surface of the second pixel is provided with a second beam-splitting structure, and the surface of the third pixel is provided with a third beam-splitting structure; The first beam splitting structure is used to split the first optical signal that the first pixel does not receive to the W pixel adjacent to the first pixel; The second beam splitting structure is used to split the second optical signal that the second pixel does not receive to the W pixel adjacent to the second pixel; The third beam splitting structure is used to split the third optical signal that the third pixel does not receive to the W pixel adjacent to the third pixel.

2. The image sensor of claim 1, wherein, The first beam splitting structure, the second beam splitting structure, and the third beam splitting structure are respectively used to distribute optical signals in at least one of the following directions: upward, downward, leftward, or rightward.

3. The image sensor according to claim 1 or 2, characterized by The surface of the W pixel is provided with a fourth beam-splitting structure; the fourth beam-splitting structure is used to perform at least one of the following beam-splitting operations: The fourth optical signal that the W pixel does not receive is split and given to the first pixel adjacent to the W pixel; Alternatively, the fifth optical signal that the W pixel does not receive can be split and given to the second pixel adjacent to the W pixel; Alternatively, the sixth optical signal that the W pixel does not receive can be split and given to the third pixel adjacent to the W pixel.

4. The image sensor of claim 3, wherein, The fourth beam splitting structure is used to distribute light signals in at least one of the following directions: upward, downward, leftward, or rightward.

5. The image sensor according to claim 3 or 4, characterized in that, The CFA also includes a supervisory pixel; the surface of the supervisory pixel is provided with a fifth beam-splitting structure; The fifth beam-splitting structure is used to split the infrared light that the supervisory pixel does not receive to the W pixel adjacent to the supervisory pixel.

6. The image sensor of claim 5, wherein, The fourth beam splitter is also used to split the seventh optical signal that the W pixel does not receive to the supervisory pixel adjacent to the W pixel.

7. The image sensor according to any one of claims 1 to 6, wherein The first pixel is an R pixel; the second pixel is a G pixel; and the third pixel is a B pixel.

8. The image sensor of claim 7, wherein, The first optical signal includes green light and blue light; the second optical signal includes red light and blue light; and the third optical signal includes red light and green light.

9. The image sensor according to any one of claims 1 to 6, wherein, The first pixel is a C pixel; the second pixel is an M pixel; and the third pixel is a Y pixel.

10. The image sensor of claim 9, wherein, The first optical signal includes magenta light and yellow light; the second optical signal includes cyan light and yellow light; and the third optical signal includes cyan light and magenta light.

11. The image sensor according to any one of claims 7-10, wherein, The first optical signal, the second optical signal, and the third optical signal each include infrared light.

12. The image sensor according to any one of claims 3-6, wherein, The first pixel is an R pixel, the fourth optical signal includes red light; the second pixel is a G pixel, the fifth optical signal includes green light; the third pixel is a B pixel, and the sixth optical signal includes blue light.

13. The image sensor of any of claims 3-6, wherein, The first pixel is a C pixel, the fourth light signal includes cyan light; the second pixel is an M pixel, the fifth light signal includes magenta light; the third pixel is a Y pixel, and the sixth light signal includes yellow light.

14. The image sensor of claim 6, wherein, The seventh optical signal includes red light, green light, or blue light.

15. The image sensor of any one of claims 1-14, wherein, The CFA includes RGBW, RGB2W, RGB3W, MYCW, MYC2W, or MYC3W.

16. An image sensor, comprising: include: A color filter array (CFA) includes a first pixel, a second pixel, a third pixel, and a W pixel; wherein a fourth light splitting structure is disposed on a surface of the W pixel; the fourth light splitting structure is configured to perform at least one of the following light splitting operations: splitting a fourth light signal, which is not received by the W pixel, to the first pixel adjacent to the W pixel; or splitting a fifth light signal, which is not received by the W pixel, to the second pixel adjacent to the W pixel; or splitting a sixth light signal, which is not received by the W pixel, to the third pixel adjacent to the W pixel.

17. The image sensor of claim 16, wherein, when the first pixel is an R pixel, the fourth light signal includes red light; when the second pixel is a G pixel, the fifth light signal includes green light; and when the third pixel is a B pixel, the sixth light signal includes blue light.

18. The image sensor of claim 16, wherein, when the first pixel is a C pixel, the fourth light signal includes cyan light; when the second pixel is an M pixel, the fifth light signal includes magenta light; and when the third pixel is a Y pixel, the sixth light signal includes yellow light.

19. The image sensor of any of claims 16-18, wherein, The CFA includes RGBW, RGB2W, RGB3W, MYCW, MYC2W, or MYC3W.

20. An image processing method, characterized by, The method is applicable to an imaging system including the image sensor of any one of claims 1-7, and the method includes: acquiring first image data based on the W pixel in the image sensor; acquiring second image data based on the R pixel, the G pixel, and the B pixel in the image sensor; fusing the first image data and the second image data to obtain a target image.

21. The method of claim 20, wherein, Before the fusing the first image data and the second image data to obtain the target image, the method further includes: performing interpolation processing and noise reduction processing according to the first image data; performing color noise removal and pseudo-color reduction processing according to the second image data.

22. The method of claim 21, wherein, When the second image data further includes infrared light image data, the method further includes: performing infrared light removal processing according to the second image data.

23. An image acquisition device, characterized in that The method includes: the image sensor of any one of claims 1-19; one or more image signal processors; a memory configured to store one or more programs; when the one or more programs are executed by the one or more image signal processors, the one or more image signal processors implement the method of any one of claims 20-22.

Images