Image sensor, image generation method, camera module and electronic device

Abstract

The present invention provides an image sensor, an image generating method, a camera module and an electronic device. The image sensor comprises: a photosensitive layer, a plurality of color filter layers, and a driving unit connected to the color filter layers; the plurality of color filter layers are superimposed on the light receiving surface of the photosensitive layer; and the driving unit is used for driving the relative positions of the plurality of color filter layers to change.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing, and in particular to an image sensor, an image generating method, a camera module and an electronic device. Background Art

[0002] With the continuous development of science and technology, people have higher and higher requirements for the quality of photo shooting. How to improve the quality of photo shooting has become a hot issue that various manufacturers are competing to study.

[0003] In the related art, in order to improve the shooting quality of photos, multiple cameras with different color filter arrays are usually set on the shooting equipment. The color filter array is used to adjust the color of light received by the photosensitive element. For example, a color camera with a color filter array and a monochrome camera with a monochrome filter array can be set. The color camera is used to collect color images with sufficient color information but poor light transmittance, and the monochrome camera is used to collect monochrome images with insufficient color information but good light transmittance. The pixels of the two images are then aligned and matched, and finally, photos of better quality are obtained through image synthesis.

[0004] In the process of pixel alignment and matching, due to the deviations in structure and process between different cameras, it is difficult to ensure a good pixel alignment effect, which easily leads to poor synthesis effect of multiple images. Summary of the invention

[0005] The embodiments of the present invention provide an image sensor, an image generation method, a camera module and an electronic device, which can perform image synthesis based on multiple images with different color filter arrays. There is no need to align pixels during the synthesis process, which avoids the pixel alignment process affecting the synthesis effect and improves the effect and efficiency of image synthesis.

[0006] In a first aspect, an embodiment of the present invention provides an image sensor, the image sensor comprising:

[0007] A photosensitive layer, a plurality of color filter layers, and a driving unit connected to the plurality of color filter layers;

[0008] The plurality of color filter layers are superimposed on the light receiving surface of the photosensitive layer;

[0009] The driving unit is used for driving the relative positions of the plurality of color filter layers to change.

[0010] In a second aspect, an embodiment of the present invention provides an image generation method, which is applied to an electronic device, wherein the electronic device includes the image sensor of the first aspect, and the method includes:

[0011] Acquire a first image captured by a photosensitive layer of the image sensor;

[0012] Controlling a driving unit of the image sensor to drive the relative positions of the plurality of color filter layers to change at least once;

[0013] After each change in the relative positions between the plurality of color filter layers, acquiring a second image captured by the photosensitive layer to obtain at least one second image;

[0014] A target image is generated based on the first image and the at least one second image.

[0015] In a third aspect, an embodiment of the present invention provides an image generating device, the device comprising:

[0016] A first acquisition module, used to acquire a first image captured by the photosensitive layer of the image sensor;

[0017] A control module, used for controlling a driving unit of the image sensor to drive the relative positions of the plurality of color filter layers to change at least once;

[0018] A second acquisition module, configured to acquire the second image captured by the photosensitive layer after each change in the relative positions between the plurality of color filter layers, to obtain at least one second image;

[0019] A generating module is used to generate a target image based on the first image and the at least one second image.

[0020] In a fourth aspect, an embodiment of the present invention provides a camera module, which includes the image sensor of the first aspect.

[0021] In a fifth aspect, an embodiment of the present invention further provides an electronic device, comprising the camera module of the fourth aspect.

[0022] In a sixth aspect, an embodiment of the present invention provides another image sensor, the image sensor comprising:

[0023] A photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer;

[0024] The target color filter layer is arranged on the light receiving surface of the photosensitive layer;

[0025] The driving unit is used for driving the relative position between the target color filter layer and the photosensitive layer to change.

[0026] In a seventh aspect, an embodiment of the present invention provides another image generation method, which is applied to an electronic device, wherein the electronic device includes the image sensor of the sixth aspect, and the method includes:

[0027] Acquire a first image captured by a photosensitive layer of the image sensor;

[0028] Controlling a driving unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once;

[0029] After each change in the relative position between the target color filter layer and the photosensitive layer, a second image captured by the photosensitive layer is acquired to obtain at least one second image;

[0030] A target image is generated based on the first image and the at least one second image.

[0031] In an eighth aspect, an embodiment of the present invention provides another image generating device, the device comprising:

[0032] A first acquisition module, used to acquire a first image captured by the photosensitive layer of the image sensor;

[0033] A control module, configured to control a driving unit of the image sensor to drive a relative position between the target color filter layer and the photosensitive layer to change at least once;

[0034] A second acquisition module, configured to acquire a second image captured by the photosensitive layer after each change in the relative position between the target color filter layer and the photosensitive layer, to obtain at least one second image;

[0035] A generating module is used to generate a target image based on the first image and the at least one second image.

[0036] In a ninth aspect, an embodiment of the present invention provides another camera module, which includes the image sensor of the sixth aspect.

[0037] In a tenth aspect, an embodiment of the present invention further provides another electronic device, comprising the camera module of the ninth aspect.

[0038] In the eleventh aspect, an embodiment of the present invention further provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image generation method provided by the present invention are implemented.

[0039] In an embodiment of the present invention, the image sensor may include: a photosensitive layer, a plurality of color filter layers, and a driving unit connected to the color filter layers; the plurality of color filter layers are superimposed and arranged on the light receiving surface of the photosensitive layer; under the drive of the driving unit, the relative position relationship between different color filter layers is changed, or the relative position relationship between the color filter layer and the photosensitive layer is changed. The image sensor can work at different color filter layer positions, the color filter array of the image sensor can be changed, and the electronic device can capture multiple images based on different color filter arrays through the image sensor, and then perform image synthesis. The color filter array of the image sensor can be mechanically switched to improve the sensitivity, so that an image sensor can take into account the advantages of different color filter arrays. In addition, since the synthesized image is captured by the same image sensor, the pixel alignment process is avoided, and the accuracy and efficiency of image synthesis are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is one of the structural diagrams of the image sensor provided by the embodiment of the present invention;

[0041] Figure 2 is a top view of a color filter layer provided by an embodiment of the present invention;

[0042] Figure 3 This is one of the schematic diagrams of the color filter layer provided in the embodiment of the present invention;

[0043] Figure 4 This is one of the schematic diagrams of the movement direction of the color filter layer provided in an embodiment of the present invention;

[0044] Figure 5 is a top view of a first color filter array provided by an embodiment of the present invention;

[0045] Figure 6 This is the second schematic diagram of the movement direction of the color filter layer provided by the embodiment of the present invention;

[0046] Figure 7 is a top view of a second color filter array provided by an embodiment of the present invention;

[0047] Figure 8 is a structural diagram of an electrostatic comb drive provided by an embodiment of the present invention;

[0048] Fig. 9 It is one of the step flow charts of the image generation method provided by an embodiment of the present invention;

[0049] Fig.10 is an image fusion flow chart based on multi-scale decomposition provided by an embodiment of the present invention;

[0050] Fig.11 This is the second structural diagram of the image sensor provided by the embodiment of the present invention;

[0051] Fig.12 This is the second schematic diagram of the color filter layer provided by the embodiment of the present invention;

[0052] Fig.13 is a first position relationship schematic diagram provided by an embodiment of the present invention;

[0053] Fig.14 is one of the second position relationship schematic diagrams provided in an embodiment of the present invention;

[0054] Fig.15 This is the second schematic diagram of the second position relationship provided by the embodiment of the present invention;

[0055] Fig.16 This is the second step flow chart of the image generation method provided by the embodiment of the present invention;

[0056] Fig.17 is one of the block diagrams of the image generating device provided by the embodiment of the present invention;

[0057] Fig.18 This is the second block diagram of the image generating device provided by the embodiment of the present invention;

[0058] Fig.19 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention.

[0059] Description of reference numerals:

[0060] 10. Photosensitive layer; 11. Photosensitive pixel; 20. Multiple color filter layers; 21. First color filter layer; 22. Second color filter layer; 23. Third color filter layer; 24. Target color filter layer; 30. Driving unit; 31. First driving unit; 32. Second driving unit; 33. Third driving unit; 34. Power supply; 35. Fixed frame; 36. First capacitor plate; 37. First comb tooth; 38. Second capacitor plate; 39. Second comb tooth; 40. Focusing layer; 50. Wiring layer; 61. Red color filter pixel; 62. Green color filter pixel; 63. Blue color filter pixel; 64. Fully transparent color filter pixel; 70. Color filter pixel array; 81. First color filter group; 82. Second color filter group; 83. Third color filter group; 84. Target color filter group. DETAILED DESCRIPTION

[0061] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0062] Figure 1is one of the image sensor structure diagrams provided by an embodiment of the present invention, such as Figure 1 As shown, the image sensor may include: a photosensitive layer, a plurality of color filter layers, and a driving unit connected to the plurality of color filter layers.

[0063] The photosensitive layer is also commonly referred to as a photoelectric sensor, which can sense the intensity of light and output a corresponding electrical signal according to the intensity of light. In the embodiment of the present invention, the form of the photosensitive layer included in the image sensor is not limited. For example, the photosensitive layer can be a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), or other forms. Among them, one side of the photosensitive layer is the light receiving surface, which is used to receive light signals.

[0064] Since the photosensitive layer can only sense the intensity of light but cannot distinguish the color of light (i.e. the wavelength of light), the image sensor also needs to set a color filter layer on the side of the photosensitive layer that receives light (light receiving surface). The color filter pixels of the color filter layer correspond to the photosensitive pixels of the photosensitive layer. The color filter pixels only allow light of a specific color (wavelength) to pass through, so that the photosensitive pixels of the photosensitive layer receive light of a specific color. Through the correspondence between the color filter pixels and the photosensitive pixels, the image sensor can obtain the color information of the light and thus capture a color image.

[0065] In the embodiment of the present invention, Figure 1 As shown, multiple color filter layers are stacked on the light-receiving surface of the photosensitive layer. In order to ensure a good color filtering effect, multiple color filter layers 20 can completely cover the photosensitive layer 10 and are arranged parallel to the photosensitive layer 10. Figure 1 As shown, it includes 4 layers of multiple color filter layers 20, all multiple color filter layers 20 can be stacked and arranged in parallel, all multiple color filter layers 20 can completely cover the photosensitive layer, and multiple color filter layers 20 can be arranged in parallel with the photosensitive layer. In addition, the multiple color filter layers 20 can also be covered with a focusing layer 40, which is composed of microlenses, each microlens covers a color filter pixel of the multiple color filter layers 20, and the microlenses of the focusing layer 40 can converge light, so that the light is more concentrated through each color filter pixel of the multiple color filter layers 20, thereby improving the utilization rate of light. The backlight surface of the photosensitive layer 10 can also be connected to the wiring layer 50, and the wiring layer 50 is arranged with signal lines connecting the photosensitive layer 10 and the processor.

[0066] The driving unit is used to drive the relative position between multiple color filter layers to change. The driving unit is connected to multiple color filter layers, and under the drive of the driving unit, the relative position relationship between different color filter layers can be changed, or the relative position relationship between multiple color filter layers and the photosensitive layer can be changed. It should be noted that a color filter layer can be connected to one or more corresponding driving units, so that the color filter layer can be independently controlled by the corresponding one or more driving units, and multiple color filter layers can also be connected to one or more corresponding driving units at the same time, so that multiple color filter layers can be controlled simultaneously by one or more driving units. It should be noted that there can be a color filter layer without a driving unit, and the other color filter layers with driving units can move relative to the color filter layer without the driving unit under the drive of the driving unit, thereby achieving the effect of changing the relative position relationship between multiple color filter layers and the photosensitive layer.

[0067] Reference Figure 2 , Figure 2 FIG. 4 shows a top view of a color filter layer provided by an embodiment of the present invention. Figure 2 As shown, the driving unit is connected to the side of the plurality of color filter layers, and the driving unit is used to drive the plurality of color filter layers to move in a direction perpendicular to the side. The driving unit includes a first driving unit connected to the first color filter layer, a second driving unit connected to the second color filter layer, and a third driving unit connected to the third color filter layer. Under the drive of the first driving unit 31, the first color filter layer 21 can move parallel to the first direction a relative to the second color filter layer 22 and the third color filter layer 23, under the drive of the second driving unit 32, the second color filter layer 22 can move parallel to the first direction a relative to the first color filter layer 21 and the third color filter layer 23, and under the drive of the third driving unit 33, the third color filter layer 23 can move parallel to the second direction b relative to the first color filter layer 21 and the second color filter layer 22. Wherein, the first direction a and the second direction b are perpendicular to each other. It should be noted that the movement directions of the above-mentioned color filter layers are only exemplary descriptions. According to actual needs, each color filter layer can be set to move in any direction of the plane where it is located. The distance of each movement is an integer multiple of a color filter pixel distance, where a color filter pixel distance represents the distance between the center points of two adjacent color filter pixels. In addition, the above-mentioned movement direction and movement distance can also be flexibly adjusted by technicians according to actual needs, and the embodiments of the present invention do not specifically limit this.

[0068] Each color filter layer is composed of a number of color filter pixels, which can form a color filter array. The color filter array can also be called a color filter array (CFA). After the light passes through the color filter array, a color light array composed of light of different colors can be projected onto the photosensitive layer behind it. After the photosensitive layer is sensitive to the color light array, a color image can be output.

[0069] In the embodiment of the present invention, by adjusting the relative position relationship between a plurality of different color filter layers, the color filter arrays of different color filter layers can be made to produce different superposition modes, thereby projecting color light arrays of different styles on the photosensitive layer. Alternatively, by adjusting the relative position relationship between a single color filter layer and the photosensitive layer, color light arrays at different positions can be projected on the photosensitive layer. Therefore, by adjusting the position of the color filter layer at the shooting interval, the same photosensitive pixel of the photosensitive layer can obtain a variety of different colors of light when shooting the same scene, thereby enriching the light information received by the photosensitive layer, and then an image with richer light information can be obtained by synthesizing multiple pictures, thereby improving the shooting effect.

[0070] Optionally, in an implementation of an embodiment of the present invention, a driving unit drives multiple color filter layers to switch between a first state and a second state. In the first state, the multiple color filter layers have a first relative position relationship, and the multiple color filter layers are stacked on each other to form a first color filter array; in the second state, the multiple color filter layers have a second relative position relationship, and the multiple color filter layers are stacked on each other to form a second color filter array.

[0071] There may be a first relative position relationship and a second relative position relationship between the multiple color filter layers, wherein, under the first relative position relationship, the multiple color filter layers in the image sensor are all at their corresponding first positions, that is, every two color filter layers are in the first relative position relationship, and under the second relative position relationship, the multiple color filter layers in the image sensor are all at their corresponding second positions, that is, every two color filter layers are in the second relative position relationship. It should be noted that each color filter layer corresponds to its own first position and second position, and the first position and second position corresponding to each color filter layer may be different positions or the same positions. The first drive control operation may be an operation of adjusting the multiple color filter layers to their corresponding first positions, so that every two color filter layers are in the first relative position relationship, so as to adjust the relative position relationship of the multiple color filter layers to the first relative position relationship, and the second drive control operation may be an operation of adjusting the multiple color filter layers to their corresponding second positions, so that every two color filter layers are in the first relative position relationship, so as to adjust the relative position relationship of the multiple color filter layers to the second relative position relationship.

[0072] In the embodiment of the present invention, different relative position relationships may correspond to different superimposed color filter arrays. When multiple color filter layers are in the first relative position relationship, each color filter layer overlaps and forms a first color filter array; when multiple color filter layers are in the second relative position relationship, each color filter layer overlaps and forms a second color filter array; wherein the transmittance of the second color filter array is greater than the transmittance of the first color filter array. It should be noted that the transmittance of the color filter array refers to the ratio of the light intensity before passing through the color filter array to the light intensity after passing through the color filter array. A color filter array with a smaller transmittance can usually provide more accurate color information for the photosensitive layer, while a color filter array with a larger transmittance can usually provide more accurate brightness information for the photosensitive layer. In the embodiment of the present invention, a first image with accurate color can be captured when multiple color filter layers form the first color filter array, and a second image with sufficient and accurate brightness can be captured when multiple color filter layers form the second color filter array. Then, the first image and the second image are synthesized to form a target image with accurate color and brightness, so as to achieve the effect of improving the shooting quality.

[0073] Optionally, the color filter layer is composed of color filter units, and the plurality of color filter layers include a first color filter layer, a second color filter layer and a third color filter layer. Figure 3 , Figure 3 FIG. 1 shows one of the schematic diagrams of the color filter layer provided in an embodiment of the present invention. Figure 3 As shown, the plurality of color filter layers may include a first color filter layer 21 , a second color filter layer 22 and a third color filter layer 23 .

[0074] In an embodiment of the present invention, each color filter layer may include only two types of color filter pixels, one of which is a color filter pixel with a specific color, which only allows light of a specific color to pass through, such as red, color filter, blue, etc., and the other color filter pixel is a fully transparent color filter pixel, which allows light of all colors to pass through. It should be noted that the fully transparent color filter pixel can be realized by not filling any material, or by filling with transparent material. In the case of filling with transparent material, the transparent material can allow light of all bands to pass through, or only allow light of the visible light band to pass through, so as to eliminate the influence of invisible light such as infrared light and ultraviolet light on imaging.

[0075] like Figure 3As shown, the first color filter unit of the first color filter layer 21 includes a red color filter pixel 61 and a full-transparent color filter pixel 64, the second color filter unit of the second color filter layer 22 includes a green color filter pixel 62 and a full-transparent color filter pixel 64, and the third color filter unit of the third color filter layer 23 includes a blue color filter pixel 63 and a full-transparent color filter pixel 64. Specifically, the first color filter layer includes a plurality of 4×4 first color filter units, the first color filter unit includes four 2×2 first color filter groups 81, and the first color filter group includes one red color filter pixel and three full-transparent color filter pixels; the second color filter layer includes a plurality of 4×4 second color filter units, the second color filter unit includes four 2×2 second color filter groups 82, and the second color filter group includes two diagonally arranged red color filter pixels and two diagonally arranged full-transparent color filter pixels; the third color filter layer includes a plurality of 4×4 third color filter units, the third color filter unit includes four 2×2 third color filter groups 83, and the first color filter group includes one blue color filter pixel and three full-transparent color filter pixels. in, Figure 3 The vertical lines in the figure represent red filter pixels, the oblique lines represent green filter pixels, and the horizontal lines represent blue filter pixels.

[0076] Since the human eye is naturally more sensitive to green, the number of red filter pixels and blue filter pixels can be set equal, and the number of green filter pixels can be set equal to the sum of the number of red filter pixels and blue filter pixels. This is to obtain better green light and better suit the viewing habits of the human eye. In addition, the number of red filter pixels can be set equal to the sum of the number of green filter pixels and blue filter pixels, or the number of blue filter pixels can be set equal to the sum of the number of red filter pixels and green filter pixels. For example, Figure 3 As shown, in an embodiment of the present invention, the color filter unit of the first color filter layer may include 4 red color filter pixels and 12 fully transparent color filter pixels, the color filter unit of the second color filter layer may include 8 green color filter pixels and 8 fully transparent color filter pixels, and the color filter unit of the third color filter layer may include 4 blue color filter pixels and 12 fully transparent color filter pixels.

[0077] It should be noted that Figure 3 The color filter unit of the color filter layer shown in FIG. 1 is the smallest color filter unit of the color filter layer. The complete color filter pixel array of each color filter layer can be composed of Figure 3 The color filter unit is expanded as shown in FIG. Figure 3 The color filter unit in the embodiment is a 4X4 array, with a total of 16 color filter pixels. In the actual production process, the complete color filter pixel array of each color filter layer can be a 1600X1200 array composed of 120,000 4X4 color filter units, with a total of 1.92 million color filter pixels.

[0078] Reference Figure 4 , Figure 4 FIG. 1 shows one of the schematic diagrams of the moving direction of the color filter layer provided in an embodiment of the present invention. Figure 4As shown, the first color filter layer 21, the second color filter layer 22 and the third color filter layer 23 are overlapped with each other.

[0079] like Figure 4 As shown in the figure, in the case where the first color filter layer, the second color filter layer and the third color filter layer overlap each other and the edges are aligned, the first color filter layer can be moved in direction a by a distance of a color filter pixel by the driving unit, and then the third color filter layer can be moved in direction b by a distance of a color filter pixel by the driving unit, so that the first color filter layer, the second color filter layer and the third color filter layer overlap each other to form a first color filter array; the first color filter layer can also be moved in the opposite direction of direction b by a distance of a color filter pixel by the driving unit, and then moved in direction a by a distance of a color filter pixel by the driving unit, and then Then, the second color filter layer is moved in the opposite direction of direction b by a distance of a color filter pixel through the driving unit, so that the first color filter layer, the second color filter layer and the third color filter layer are overlapped to form a first color filter array; the second color filter layer can also be moved in the opposite direction of direction a by a distance of a color filter pixel through the driving unit, and then the third color filter layer is moved in the opposite direction of direction a by a distance of a color filter pixel through the driving unit, and then moved in the direction b by a distance of a color filter pixel, so that the first color filter layer, the second color filter layer and the third color filter layer are overlapped to form a first color filter array. It should be noted that the above method of moving each color filter layer to form the first color filter array is described as an example, and technicians can also use other moving methods to make the first color filter layer, the second color filter layer and the third color filter layer overlap each other to form the first color filter array, which is not specifically limited in the embodiments of the present invention.

[0080] Reference Figure 5 , Figure 5 FIG. 4 shows a top view of a first color filter array provided by an embodiment of the present invention. Figure 5 As shown, when the first color filter layer 21, the second color filter layer 22 and the third color filter layer 23 are stacked together to form a first color filter array, the red color filter pixels of the first color filter layer 21 are aligned with half of the fully transparent color filter pixels of the second color filter layer 22, and the other half of the fully transparent color filter pixels of the second color filter layer 22 are aligned with the blue color filter pixels of the third color filter layer 23, so that the three color filter layers can be superimposed to form a Bayer array (for example, an RGGB array), so that the sensor can obtain more accurate color information.

[0081] Reference Figure 6 , Figure 6 FIG. 2 shows a second schematic diagram of the movement direction of the color filter layer provided in an embodiment of the present invention. Figure 6As shown, in the case where multiple color filter layers are overlapped to form a first color filter array, the first color filter layer can be moved in direction b by a distance of one color filter pixel by the driving unit, and then the third color filter layer can be moved in the opposite direction of direction b by a distance of one color filter pixel by the driving unit, so that the first color filter layer, the second color filter layer and the third color filter layer are overlapped to form a second color filter array; the first color filter layer can also be moved in direction b by a distance of 2 color filter pixels by the driving unit, and then the second color filter layer can be moved in direction b by a distance of one color filter pixel by the driving unit, so that the first color filter layer, the second color filter layer and the third color filter layer are overlapped to form a second color filter array; the second color filter layer can also be moved in the opposite direction of direction b by a distance of one color filter pixel by the driving unit, and then the third color filter layer can be moved in the opposite direction of direction b by a distance of 2 color filter pixels by the driving unit, so that the first color filter layer, the second color filter layer and the third color filter layer are overlapped to form a second color filter array. It should be noted that the above-mentioned method of moving each color filter layer to form the second color filter array is described as an example. The technician can also use other moving methods to make the first color filter layer, the second color filter layer and the third color filter layer overlap with each other to form the second color filter array. The embodiment of the present invention does not make specific limitations.

[0082] Reference Figure 7 , Figure 7 FIG. 4 shows a top view of a second color filter array provided by an embodiment of the present invention. Figure 7 As shown, when the first color filter layer 21, the second color filter layer 22 and the third color filter layer 23 are stacked together to form a second color filter array, the red color filter pixels of the first color filter layer 21 and half of the green color filter pixels of the second color filter layer 22 are aligned, the blue color filter pixels of the third color filter layer 23 and the other half of the green color filter pixels of the second color filter layer 22 are aligned, and the fully transparent color filter pixels of different color filter layers are aligned with each other, so that half of the pixels in the second color filter array formed by the superposition of the three color filter layers are fully transparent color filter pixels, which improves the transmittance compared to the first color filter array, so that the sensor can obtain more accurate brightness information.

[0083] In an embodiment of the present invention, if the transmittance of each green pixel of red, green, and blue is 40%, and the transmittance of the fully transparent filter pixel is 100%, then by calculation, the transmittance of each 4X4 first color filter array is 40%, and the transmittance of each 4X4 second color filter array is 40%*40%*0.5+100%*0.5=58%, which is nearly 50% higher than the conventional first color filter array. Therefore, by adopting the multiple color filter layer structures and moving methods proposed in the present invention, multiple color filter layers can be moved to form a second color filter array with high light projection rate. When taking photos, the photosensitive layer of the image sensor can receive more sufficient light, thereby improving the effect of taking photos.

[0084] Optionally, generally speaking, the image sensor is small in size. In order to arrange the driving unit in a smaller space, the driving unit may include a micro-electromechanical system (MEMS), such as an electrostatic comb drive. The micro-electromechanical system is a micron-scale mechanical driving system.

[0085] refer to Figure 8 , Figure 8 FIG. 4 shows a structural diagram of an electrostatic comb drive provided by an embodiment of the present invention. Figure 8 As shown, the electrostatic comb tooth drive may include a power supply 34, a fixed frame 35, a first comb tooth portion and a second comb tooth portion, wherein the first comb tooth portion is connected to the fixed frame 35, the second comb tooth portion is connected to one of the plurality of color filter layers, the power supply 34 is connected to the first comb tooth portion and the second comb tooth portion, and under the drive of the power supply 34, the first comb tooth portion and the second comb tooth portion move relative to each other to change the relative position relationship between different color filter layers. The first comb tooth portion is composed of a first capacitor plate 36 and a plurality of first comb teeth 37 fixed on the first capacitor plate 36, and the second comb tooth portion is composed of a second capacitor plate 38 and a plurality of second comb teeth 39 fixed on the second capacitor plate 38. The first capacitor plate 36 and the second capacitor plate 38 are charged with charges of the same polarity or charges of different polarities under the action of the power supply 34, so that a repulsive force or an attractive force is generated between the first capacitor plate 36 and the second capacitor plate 38 to change the distance between the first capacitor plate 36 and the second capacitor plate 38. The first comb tooth 37 and the second comb tooth 39 fit each other, acting as a slide rail and playing a limiting role.

[0086] Furthermore, when the first capacitor plate 36 and the second capacitor plate 38 are powered by a pair of positive and negative electrodes, positive charges can only be charged in one capacitor plate and negative charges can be charged in the other capacitor plate, so that only electrostatic attraction can be generated between the two capacitor plates. Therefore, an elastic device can also be arranged between the first capacitor plate 36 and the second capacitor plate 38, and an elastic force is generated between the two capacitor plates through the elastic device to drive the two capacitor plates to move away from each other. In this way, when the electrostatic attraction between the two capacitor plates is greater than the elastic force, the two capacitor plates move closer to each other, and when the electrostatic attraction between the two capacitor plates is less than the elastic force, the two capacitor plates move away from each other. In this way, the electrostatic comb drive can be conveniently driven by a relatively simple control circuit, thereby reducing R&D and manufacturing costs.

[0087] In summary, an image sensor provided by an embodiment of the present invention includes: a photosensitive layer, multiple color filter layers, and a driving unit connected to the multiple color filter layers; the multiple color filter layers are superimposed and arranged on the light receiving surface of the photosensitive layer; the driving unit is used to drive the relative positions of the multiple color filter layers to change. The image sensor can work under different color filter layer positions, the color filter array of the image sensor can be changed, and the electronic device can capture multiple images based on different color filter arrays through the image sensor, and then perform image synthesis. The color filter array of the image sensor can be mechanically switched to improve the sensitivity, so that an image sensor can take into account the advantages of different color filter arrays. In addition, since the synthesized image is taken by the same image sensor, the pixel alignment process is avoided, and the accuracy and efficiency of image synthesis are improved.

[0088] Fig. 9 is one of the flow charts of the steps of the image generation method provided by an embodiment of the present invention. The method is applied to an electronic device, the electronic device including the above-mentioned image sensor, such as Fig. 9 As shown, the method may include:

[0089] Step 101: Acquire a first image captured by a photosensitive layer of the image sensor.

[0090] In an embodiment of the present invention, a plurality of color filter layers may have a first relative position relationship and a second relative position relationship, wherein, in the first relative position relationship, the plurality of color filter layers in the image sensor are all at their corresponding first positions, and in the second relative position relationship, the plurality of color filter layers in the image sensor are all at their corresponding second positions. It should be noted that each color filter layer corresponds to its own first position and second position, and the first position and second position corresponding to each color filter layer may be different positions or the same positions. Different relative position relationships may correspond to different superimposed color filter arrays. When the plurality of color filter layers are all in the first relative position relationship, the plurality of color filter layers are superimposed on each other to form a first color filter array; when the plurality of color filter layers are all in the second relative position relationship, the plurality of color filter layers are superimposed on each other to form a second color filter array; wherein the transmittance of the second color filter array is greater than the transmittance of the first color filter array.

[0091] When taking a photo through an image sensor, the positions of multiple color filter layers can be adjusted through a driving unit first, so that the multiple color filter layers are in a corresponding first position, so that the multiple color filter layers are in a first relative position relationship, and the light receiving surfaces of the photosensitive layers are overlapped to form a first color filter array. After the first color filter array is formed, the first image is collected through the photosensitive layer. The positions of each color filter layer in the multiple color filter layers can also be adjusted through a driving unit first, so that the multiple color filter layers are in a corresponding second position, so that the multiple color filter layers are in a second relative position relationship, and the light receiving surfaces of the photosensitive layers are overlapped to form a second color filter array. After the second color filter array is formed, the first image is collected through the photosensitive layer.

[0092] It should be noted that multiple color filter layers may also have default positions. When the driving unit is not controlled, the multiple color filter layers all remain in the default positions, wherein the default positions corresponding to the multiple color filter layers may be the first positions corresponding to the multiple color filter layers, or may be the second positions corresponding to the multiple color filter layers. Therefore, when taking photos through the image sensor, the first image can be directly captured through the photosensitive layer without adjusting the positions of the multiple color filter layers through the driving unit, thereby improving the shooting speed.

[0093] Step 102: Control a driving unit of the image sensor to drive the relative positions of the plurality of color filter layers to change at least once.

[0094] In an embodiment of the present invention, the plurality of color filter layers include a first color filter layer composed of red color filter pixels and fully transparent color filter pixels, a second color filter layer composed of green color filter pixels and fully transparent color filter pixels, and a third color filter layer composed of blue color filter pixels and fully transparent color filter pixels. When the color filter pixel arrays of the plurality of color filter layers are overlapped to form a first color filter array, all red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap all blue color filter pixels of the third color filter layer.

[0095] In the case where the color filter pixel arrays of the multiple color filter layers are overlapped to form a first color filter array, the driving unit of the image sensor is controlled to drive the multiple color filter layers to move so that the relative positions of the multiple color filter layers change at least once, and the color filter pixel arrays of the multiple color filter layers are overlapped to form at least one second color filter array; in the first color filter array, all red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap with each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with all blue color filter pixels of the third color filter layer; wherein the transmittance of the second color filter array is greater than the transmittance of the first color filter array.

[0096] The driving unit may correspond to a first driving control operation and a second driving control operation, wherein the first driving control operation may be an operation of adjusting multiple color filter layers to their corresponding first positions so as to adjust the relative position relationship of the multiple color filter layers to the first relative position relationship, and the second driving control operation may be an operation of adjusting multiple color filter layers to their corresponding second positions so as to adjust the relative position relationship of the multiple color filter layers to the second relative position relationship.

[0097] Specifically, if the first image is captured when multiple color filter layers form a first color filter array, after obtaining the first image, each color filter layer remains in the first color filter array. Therefore, when the color filter pixel arrays of different color filter layers are overlapped to form the first color filter array, the multiple color filter layers can be driven to move by a driving unit, thereby changing the relative position relationship between the different color filter layers, so that the color filter pixel arrays of different color filter layers are overlapped to form a second color filter array.

[0098] If the first image is captured when multiple color filter layers form a second color filter array, after obtaining the first image, each color filter layer remains in the second color filter array. Therefore, when the color filter pixel arrays of different color filter layers are overlapped to form the second color filter array, the multiple color filter layers can be driven to move by a driving unit, thereby changing the relative position relationship between the different color filter layers, so that the color filter pixel arrays of different color filter layers are overlapped to form the first color filter array; wherein the transmittance of the second color filter array is greater than the transmittance of the first color filter array.

[0099] Optionally, the method of controlling the driving unit of the image sensor to drive the multiple color filter layers to move may include: controlling the driving unit to drive the first color filter layer, the second color filter layer and the third color filter layer to move, so that all red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other, all blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of the multiple color filter layers overlap each other. That is, in the embodiment of the present invention, when the color filter pixel arrays of the multiple color filter layers are superimposed on each other to form a second color filter array, all red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other, all blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of the multiple color filter layers overlap each other.

[0100] In an embodiment of the present invention, the relative position relationship between multiple color filter layers can be adjusted so that the same image sensor can be sensitive under different color filter arrays, so that images with different color information and light intensity can be obtained for the same shooting scene, effectively improving the shooting capability of a single-camera electronic device.

[0101] Step 103, after each change in the relative position between the plurality of color filter layers, acquiring the second image captured by the photosensitive layer to obtain at least one second image.

[0102] If the first image is captured when multiple color filter layers form a first color filter array, after the multiple color filter layers change their relative position relationship once, a second color filter array covering the photosensitive layer is formed between the multiple color filter layers, and at this time, a second image can be collected through the photosensitive layer. It should be noted that if the multiple color filter layers change their relative position relationship multiple times, after each change in relative position relationship, a second color filter array covering the photosensitive layer is formed between the multiple color filter layers, and multiple second images can be collected.

[0103] Step 104: Generate a target image based on the first image and the at least one second image.

[0104] In the case where at least one second image contains only one second image, the target image can be generated directly based on the first image and the second image. In the case where at least one second image contains multiple second images, the first image can be first synthesized with one of the second images to obtain an intermediate image, and then the intermediate image can be synthesized with the remaining second image to generate a new intermediate image. This process can be repeated until all second images participate in the synthesis process to obtain the target image.

[0105] In the case where the at least one second image includes only one second image, the process of generating the target image based on the first image and the second image is as follows:

[0106] In the embodiment of the present invention, since the second color filter array can include half of the fully transparent filter pixels, the types and transmittances of the color filter pixels of the first color filter array and the second color filter array are different. The color filter array with lower transmittance can usually provide more accurate color information for the photosensitive layer, while the color filter array with higher transmittance can usually provide more accurate brightness information for the photosensitive layer. Therefore, in the first image and the second image, the color of one image is more accurate, and the brightness of the other image is more accurate. Therefore, the first image and the second image can be used to synthesize a target image with accurate color and brightness, so as to achieve the effect of improving the captured image quality.

[0107] It should be noted that, since there are multiple color filter layers, there may be fused color filter pixels in the color filter array formed by overlapping the color filter layers, for example Figure 6The blue filter pixels and green filter pixels in the image are overlapped to form a blue-green fused filter pixel, and the red filter pixels and green filter pixels are overlapped to form a red-green fused filter pixel. Since the transmittance of the fused filter pixel is lower than that of the single-layer filter pixel, for the fused filter pixel in the first image or the second image, the signal strength of the fused filter pixel can be corrected by the fused pixel gain, and then participate in the subsequent image fusion operation process. Among them, the fused pixel gain can be determined by the technicians through calibration during the research and development stage.

[0108] For example, the image sensor can be placed in a flat field correction (flatfield) light source environment, and an image can be captured in the state of the first color filter array to obtain a first signal intensity corresponding to the photosensitive pixel when covering the green color filter pixel, and an image can be captured in the state of the second color filter array to obtain a second signal intensity corresponding to the photosensitive pixel when covering the blue-green fusion filter pixel, and a third signal intensity corresponding to the photosensitive pixel when covering the red-green fusion pixel. The ratio of the second signal intensity to the first signal intensity is used as the fusion pixel gain of the blue-green fusion filter pixel, and the ratio of the third signal intensity to the first signal intensity is used as the fusion pixel gain of the red-green fusion filter pixel.

[0109] In the embodiment of the present invention, the fusion of the first image and the second image may be to synthesize the first image into a new target image, or to use one image to enhance the other image to obtain the target image.

[0110] Specifically, the first image and the second image may be fused based on a multi-scale decomposition method. Fig.10 , Fig.10 FIG. 4 shows a flowchart of an image fusion based on multi-scale decomposition provided by an embodiment of the present invention. Fig.10 As shown, firstly, the first image 141 and the second image 142 are multi-scaled decomposed to obtain a multi-scale representation 143 of the first image and a multi-scale representation 144 of the second image, and then, according to a specific fusion rule, the multi-scale representation 143 of the first image and the multi-scale representation 144 of the second image are fused to obtain a multi-scale representation 145 of the fused image, and then the multi-scale representation 145 of the fused image is multi-scaled inversely transformed to obtain a target image 146. Image fusion based on the multi-scale decomposition method can better retain the detail information in the first image and the second image, so that the fused target image can better retain the detail information of the original image, and the method can better retain the brightness characteristics of the original image, so that the fused target image can achieve a better low-light shooting effect.

[0111] In an embodiment of the present application, the multi-scale decomposition method may include Laplace pyramid transform, contrast pyramid transform, dual-tree complex wavelet transform, curvelet transform, non-subsampled contourlet transform, non-subsampled shearlet transform, etc. Among them, the Laplace pyramid transform can include four steps: low-pass filtering, downsampling, interpolation and band-pass filtering of the image; the wavelet transform can decompose the image at two mutually perpendicular spatial frequencies, and extract the high-frequency edge details of the image in three directions while extracting the image scanning information; the curvelet transform can well represent the anisotropic edge features in the image, and combines the advantages of the ridgelet transform that can well describe the straight line features of the image and the wavelet transform that is suitable for expressing point features. The edge information of the image can accurately and precisely represent the curve singular features of the image with fewer non-zero coefficients; the non-subsampled contourlet transform is an improved transformation method proposed on the basis of the contourlet transform. Compared with the contourlet transform, the non-subsampled contourlet transform does not undergo a downsampling operation during the decomposition and reconstruction of the image, so the translation invariance of the transformation is guaranteed and the pseudo-Gibbs phenomenon is solved; the non-subsampled shearlet transform is improved on the basis of the shearlet transform. Compared with the shearlet transform, the non-subsampled shearlet transform does not undergo a downsampling operation during the decomposition and reconstruction of the image, so the non-subsampled shearlet transform has translation invariance. It should be noted that when the technician uses the multi-scale decomposition method to fuse the first image and the second image, the technician can flexibly adjust the specific transformation method of the multi-scale decomposition according to actual needs, and the embodiments of the present application do not specifically limit this.

[0112] In addition, since the first image and the second image are images captured by the same sensor, when the electronic device remains still, the viewing angles for capturing the first image and the second image are exactly the same. In the synthesis of the first image and the second image, there is no need to align the pixels of the first image and the second image one by one, which not only improves the speed of image synthesis, but also avoids the occurrence of poor synthesis effects due to alignment errors, thereby further improving the shooting quality of the target image.

[0113] In summary, an image generation method provided by an embodiment of the present invention includes: acquiring a first image captured by a photosensitive layer of an image sensor; controlling a driving unit of the image sensor to drive the relative positions of multiple color filter layers to change at least once; acquiring a second image captured by the photosensitive layer after each change in the relative positions of the multiple color filter layers, and obtaining at least one second image; and generating a target image based on the first image and at least one second image. When the above-mentioned scheme of the present invention is used to shoot the first image and at least one second image and perform image synthesis, there is no need to align the pixels of the first image and the second image one by one, which not only improves the speed of image synthesis, but also avoids the occurrence of poor synthesis effects due to alignment errors, and further improves the shooting quality of the target image.

[0114] Fig.11 FIG. 2 is a second structural diagram of an image sensor provided by an embodiment of the present invention. Fig.11 As shown, the image sensor may include: a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer.

[0115] The photosensitive layer is also commonly referred to as a photoelectric sensor, which can sense the intensity of light and output a corresponding electrical signal according to the intensity of light. In the embodiment of the present invention, the form of the photosensitive layer included in the image sensor is not limited. For example, the photosensitive layer can be a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), or other forms. Among them, one side of the photosensitive layer is the light receiving surface, which is used to receive light signals.

[0116] Since the photosensitive layer can only sense the intensity of light but cannot distinguish the color of light (i.e. the wavelength of light), the image sensor also needs to set a target color filter layer on the side of the photosensitive layer that receives light (the light receiving surface). The color filter pixels of the target color filter layer correspond to the photosensitive pixels of the photosensitive layer. The color filter pixels only allow light of a specific color (wavelength) to pass through, so that the photosensitive pixels of the photosensitive layer receive light of a specific color. Through the correspondence between the color filter pixels and the photosensitive pixels, the image sensor can obtain the color information of the light and thus capture a color image.

[0117] In the embodiment of the present invention, Fig.11 As shown, the target color filter layer is arranged on the light-receiving surface of the photosensitive layer. In order to ensure a good color filtering effect, the target color filter layer 24 can completely cover the photosensitive layer 10 and be arranged parallel to the photosensitive layer 10. Fig.11 As shown, the image sensor includes a target color filter layer 24, a driving unit 30 connected to the target color filter layer 24, and a photosensitive layer 10 covered by the target color filter layer 24. In addition, the target color filter layer 24 can also be covered with a light-gathering layer 40, which is composed of microlenses, each of which covers a color filter pixel of the target color filter layer 24. The microlenses of the light-gathering layer 40 can converge light, so that the light passes through the color filter pixel more concentratedly, thereby improving the utilization rate of light. The backlight surface of the photosensitive layer 10 can also be connected to the wiring layer 50, and the wiring layer 50 is arranged with signal lines connecting the photosensitive layer 10 and the processor.

[0118] The driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change. The driving unit is connected to the target color filter layer, and under the driving of the driving unit, the relative position relationship between the target color filter layer and the photosensitive layer is changed.

[0119] The target color filter layer includes a color filter pixel array, the color filter pixel array includes at least three types of color filter pixels, and the photosensitive layer includes a photosensitive pixel array. For example, when the number of types of color filter pixels is 3, the color filter pixels may include red color filter pixels, green color filter pixels, and blue-green color filter pixels.

[0120] The color filter pixels of the color filter pixel array overlap with the photosensitive pixels of the photosensitive pixel array one by one. The number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array.

[0121] Reference Fig.12 , Fig.12 FIG. 2 shows a second schematic diagram of a color filter layer provided in an embodiment of the present invention. Fig.12 As shown, the color filter pixel array of the target color filter layer may include red, green and blue color filter pixels, the target color filter layer includes a plurality of 4×4 color filter units; the color filter unit includes four 2×2 target color filter groups 84, and each target color filter group 84 includes red, green and blue color filter pixels. Fig.12 The vertical lines in the figure represent red filter pixels, the oblique lines represent green filter pixels, and the horizontal lines represent blue filter pixels.

[0122] In an embodiment of the present invention, the color filter pixels of the color filter pixel array overlap one-to-one with the photosensitive pixels of the photosensitive pixel array; the number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array. For example, in the case where the color filter pixels include three types of pixels, red, green, and blue, the color filter pixel array has a first positional relationship, a second positional relationship, and a third positional relationship relative to the photosensitive layer. In the case of different positional relationships, the types of color filter pixels corresponding to each photosensitive pixel on the photosensitive layer are different.

[0123] Reference Fig.13 , Fig.13 A first position relationship diagram provided by an embodiment of the present invention is shown. Fig.13 As shown, for the target color filter layer 24 including three color filter pixels of red, green and blue, when the color filter pixel array 70 of the target color filter layer 24 is in a first position relationship relative to the photosensitive layer 10, the photosensitive pixel 11 of the photosensitive layer is covered by the blue color filter pixel, which receives blue light information.

[0124] Reference Fig.14 , Fig.14 FIG. 1 shows one of the second position relationship diagrams provided in an embodiment of the present invention, such as Fig.14 As shown, the target color filter layer 24 is along Fig.13 Move in the direction of arrow e in the figure, so that the color filter pixel array 70 of the target color filter layer 24 is in a second position relationship relative to the photosensitive layer. At this time, the photosensitive pixel 11 of the photosensitive layer is covered by the red color filter pixel, and it receives red light information.

[0125] Reference Fig.15 , Fig.15 FIG. 2 shows a second schematic diagram of a second position relationship provided by an embodiment of the present invention. Fig.15 As shown, the target color filter layer 24 is along Fig.14 Continue to move in the direction of arrow e in the figure, so that the color filter pixel array 70 of the target color filter layer 24 is in another second position relationship relative to the photosensitive layer. At this time, the photosensitive pixel 11 of the photosensitive layer is covered by the green color filter pixel, which receives green light information.

[0126] Optionally, when the number of target color filter layers is equal to 1, the color filter pixel array of the target color filter layer is composed of an equal number of red color filter pixels, green color filter pixels and blue color filter pixels, so that when the target color filter layer is translated horizontally or vertically relative to the photosensitive layer, the color of the color filter pixels corresponding to all the photosensitive pixels on the photosensitive layer changes once for each movement of one color filter pixel.

[0127] In summary, an image sensor provided by an embodiment of the present invention includes: a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer; the target color filter layer is arranged on the light receiving surface of the photosensitive layer; and the driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change. The image sensor can work under different color filter layer positions, and the color filter array of the image sensor can be changed, so that the electronic device can capture multiple images based on different color filter arrays through the image sensor, and then perform image synthesis. The color filter array of the image sensor can be mechanically switched to improve the sensitivity, so that an image sensor can take into account the advantages of different color filter arrays. In addition, since the synthesized image is captured by the same image sensor, the pixel alignment process is avoided, and the accuracy and efficiency of image synthesis are improved.

[0128] Fig.16 is a second flow chart of the steps of the image generation method provided by an embodiment of the present invention. The method is applied to an electronic device, which includes the above-mentioned image sensor, such as Fig.16 As shown, the method may include:

[0129] Step 201: Acquire a first image captured by a photosensitive layer of the image sensor.

[0130] In the embodiment of the present invention, the target color filter layer covered on the photosensitive layer of the image sensor can move parallel to the photosensitive layer to change the color of the light received by each photosensitive pixel in the photosensitive layer. Under the drive of the driving unit, the target color filter layer can move parallel to the photosensitive layer so that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have multiple corresponding relationships. Under different corresponding relationships, the same photosensitive pixel on the photosensitive layer is covered by color filter pixels of different colors.

[0131] For example, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer are in a first corresponding relationship, the photosensitive pixel A of the photosensitive layer is covered by the red color filter pixels of the target color filter layer, and in a second corresponding relationship, the photosensitive pixel A of the photosensitive layer is covered by the blue color filter pixels of the target color filter layer.

[0132] In the embodiment of the present invention, when taking a photo through an image sensor, the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer are first placed in a first corresponding relationship, and then a first image is collected through the photosensitive layer.

[0133] Step 202: Control a driving unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once.

[0134] After obtaining the first image, the target color filter layer can be controlled to move parallel to the photosensitive layer through the driving unit. Specifically, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first corresponding relationship, the target color filter layer is driven to move at least once by the driving unit, thereby changing the relative position relationship between the target color filter layer and the photosensitive layer at least once, so that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one second corresponding relationship.

[0135] When the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first relative position relationship, the driving unit controlling the image sensor drives the target color filter layer to move so that the relative position between the target color filter layer and the photosensitive layer changes at least once, and the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one second corresponding relationship; wherein, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the first relative position relationship, the photosensitive pixels are aligned with one color filter pixel; when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the second relative position relationship, the photosensitive pixels are aligned with another color filter pixel.

[0136] For example, if the color filter layer contains three kinds of color filter pixels, red, green and blue, the driving unit controlling the image sensor drives the relative position between the target color filter layer and the photosensitive layer to change twice, so that each pixel on the photosensitive layer can correspond to a color filter pixel of a different color before and after each change of the relative position.

[0137] Step 203, after each change in the relative position between the target color filter layer and the photosensitive layer, acquiring a second image captured by the photosensitive layer to obtain at least one second image.

[0138] For example, the target color filter layer is located relative to the photosensitive layer. Figure 8 The first corresponding relationship shown in FIG. 1 is a first image obtained through the photosensitive layer, and then the photosensitive layer is moved to Fig. 9 The distance of one color filter pixel is moved in the direction e shown, so that the target color filter layer is in a position relative to the photosensitive layer. Fig. 9 A second corresponding relationship is shown, in which the first second image is obtained through the photosensitive layer, and then the photosensitive layer is moved to Fig. 9 Move the color filter layer in the direction e shown in the figure by the distance of one color filter pixel, so that the target color filter layer is at a position relative to the photosensitive layer. Fig.10 Another second corresponding relationship is shown, in which a second image is obtained through the photosensitive layer.

[0139] Step 204: Generate a target image based on the first image and the at least one second image.

[0140] Generally speaking, during a shooting process, a photosensitive pixel can only receive light of a specific color, while each pixel in the captured image requires at least red, green and blue color information, so that the pixels in the image can display rich colors. Therefore, it is necessary to use a color restoration algorithm (such as demosaic, remosaic, etc.) to calculate the other two color information of the photosensitive pixel. The color restoration algorithm can estimate the intensity of the other two colors of a photosensitive pixel based on the color of light received by a photosensitive pixel and the color of light received by several photosensitive pixels around the photosensitive pixel.

[0141] In the embodiment of the present invention, since each pixel of the first image and the second image corresponds one to one, and the corresponding pixels receive different colors of light, for one shot, each photosensitive pixel actually obtains signals of two colors through the first image and the second image, and each photosensitive pixel only needs to restore another color signal to generate the final target image, which greatly improves the accuracy of the color restoration process and can significantly improve the image quality of the final output target image.

[0142] Furthermore, in an embodiment of the present invention, if the target color filter layer includes three color filter pixels of red, green, and blue, there may be another second correspondence between the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer, thereby obtaining two second images. For example, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer are in a first correspondence, the photosensitive pixel A of the photosensitive layer is covered by the red color filter pixel of the target color filter layer, and in a second correspondence, the photosensitive pixel A of the photosensitive layer is covered by the blue color filter pixel of the target color filter layer, and in another second correspondence, the photosensitive pixel A of the photosensitive layer is covered by the green color filter pixel of the target color filter layer.

[0143] Correspondingly, after capturing a second image under a second corresponding relationship, the target color filter layer can be further driven to move by a driving unit, thereby changing the relative position relationship between the target color filter layer and the photosensitive layer, so that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer are in another second corresponding relationship, and another second image is captured.

[0144] In an embodiment of the present invention, for one shooting, through the first image and the two second images, each photosensitive pixel actually obtains signals of three colors (for example, red, green and blue), and can directly generate a color target image pixel according to the three color signals corresponding to each photosensitive pixel. The color restoration process is omitted, and there is no need to estimate the color of the pixel. Color pixels can be directly generated according to the real color information, which further improves the image quality and effect of the final output target image.

[0145] In summary, another image generation method provided by an embodiment of the present invention includes: acquiring a first image captured by the photosensitive layer of the image sensor; controlling the driving unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once; acquiring a second image captured by the photosensitive layer after each change in the relative position between the target color filter layer and the photosensitive layer, and obtaining at least one second image; generating a target image based on the first image and the at least one second image. When the above scheme of the present invention is used to shoot multiple images and perform image synthesis, each photosensitive pixel can obtain multiple color signals, and color restoration can be performed based on the multiple color information obtained by the photosensitive pixels, or color restoration is not required at all, thereby improving the color accuracy of the target image output in the end.

[0146] Fig.17 is one of the block diagrams of the image generating device provided by the embodiment of the present invention, such as Fig.17 As shown, the image generating device comprises:

[0147] A first acquisition module 301 is used to acquire a first image captured by a photosensitive layer of an image sensor;

[0148] A control module 302, configured to control a driving unit of the image sensor to drive the relative positions of the plurality of color filter layers to change at least once;

[0149] A second acquisition module 303 is used to acquire a second image captured by the photosensitive layer after each change in relative position between the plurality of color filter layers, to obtain at least one second image;

[0150] The generating module 304 is configured to generate a target image based on the first image and at least one second image.

[0151] Optionally, the plurality of color filter layers include a first color filter layer composed of red color filter pixels and fully transparent color filter pixels, a second color filter layer composed of green color filter pixels and fully transparent color filter pixels, and a third color filter layer composed of blue color filter pixels and fully transparent color filter pixels, and the control module includes:

[0152] A control submodule is used to control a driving unit of an image sensor to drive the multiple color filter layers to move when the color filter pixel arrays of the multiple color filter layers overlap to form a first color filter array, so that the relative positions of the multiple color filter layers change at least once, and the color filter pixel arrays of the multiple color filter layers overlap to form at least one second color filter array; in the first color filter array, all red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with all blue color filter pixels of the third color filter layer; wherein the transmittance of the second color filter array is greater than the transmittance of the first color filter array.

[0153] Optionally, the control submodule is also used to control the driving unit to drive the first color filter layer, the second color filter layer and the third color filter layer to move, so that all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap with each other, all the blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of multiple color filter layers overlap with each other.

[0154] In summary, an image generation device provided by an embodiment of the present invention includes a first acquisition module for acquiring a first image captured by a photosensitive layer of an image sensor; a control module for controlling a driving unit of the image sensor to drive the relative positions between multiple color filter layers to change at least once; a second acquisition module for acquiring a second image captured by the photosensitive layer after each change in the relative positions between the multiple color filter layers to obtain at least one second image; and a generation module for generating a target image based on the first image and at least one second image. When the above-mentioned device of the present invention is used to shoot the first image and at least one second image and perform image synthesis, there is no need to align the pixels of the first image and at least one second image one by one, which not only improves the speed of image synthesis, but also avoids the occurrence of poor synthesis effects due to alignment errors, further improving the shooting quality of the target image.

[0155] Fig.18 is a block diagram of the second image generating device provided by an embodiment of the present invention, such as Fig.18 As shown, the image generating device comprises:

[0156] A first acquisition module 401 is used to acquire a first image captured by a photosensitive layer of an image sensor;

[0157] A control module 402, configured to control a driving unit of the image sensor to drive a relative position between a target color filter layer and a photosensitive layer to change at least once;

[0158] A second acquisition module 403 is used to acquire a second image captured by the photosensitive layer after each change in the relative position between the target color filter layer and the photosensitive layer, to obtain at least one second image;

[0159] The generating module 404 is configured to generate a target image based on the first image and at least one second image.

[0160] Optionally, the control module further includes:

[0161] A control submodule is used to control a driving unit of an image sensor to drive a target color filter layer to move when the color filter pixels of the target color filter layer have a first relative position relationship with the photosensitive pixels of the photosensitive layer, so that the relative position between the target color filter layer and the photosensitive layer changes at least once, and the color filter pixels of the target color filter layer have at least one second corresponding relationship with the photosensitive pixels of the photosensitive layer; wherein, when the color filter pixels of the target color filter layer have the first relative position relationship with the photosensitive pixels of the photosensitive layer, the photosensitive pixels are aligned with one color filter pixel; and when the color filter pixels of the target color filter layer have the second relative position relationship with the photosensitive pixels of the photosensitive layer, the photosensitive pixels are aligned with another color filter pixel.

[0162] In summary, an image generation device provided by an embodiment of the present invention includes a first acquisition module for acquiring a first image captured by a photosensitive layer of an image sensor; a control module for controlling a driving unit of the image sensor to drive the relative position between a target color filter layer and the photosensitive layer to change at least once; a second acquisition module for acquiring a second image captured by the photosensitive layer after each change in the relative position between the target color filter layer and the photosensitive layer, and obtaining at least one second image; and a generation module for generating a target image based on the first image and at least one second image. When the above-mentioned device of the present invention is used to capture the first image and at least one second image and perform image synthesis, each photosensitive pixel acquires a plurality of color signals, and color restoration can be performed based on the plurality of color information acquired by the photosensitive pixels, or color restoration may not be performed at all, thereby improving the color accuracy of the target image output in the end.

[0163] The embodiment of the present invention further provides an electronic device, referring to Fig.19 , Fig.19 A schematic diagram of the hardware structure of an electronic device provided by the present invention is shown.

[0164] The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will appreciate that Fig.17 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0165] The processor 510 is configured to execute the following process:

[0166] Acquire a first image captured by a photosensitive layer of an image sensor; control a driving unit of the image sensor to drive the relative positions of a plurality of color filter layers to change at least once; acquire a second image captured by the photosensitive layer after each change in the relative positions of the plurality of color filter layers to obtain at least one second image; generate a target image based on the first image and the at least one second image.

[0167] In the case where the color filter pixel arrays of multiple color filter layers are overlapped to form a first color filter array, the driving unit controlling the image sensor drives the multiple color filter layers to move so that the relative positions between the multiple color filter layers change at least once, and the color filter pixel arrays of the multiple color filter layers are overlapped to form at least one second color filter array; in the first color filter array, all the red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap with each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with all the blue color filter pixels of the third color filter layer; wherein the transmittance of the second color filter array is greater than the transmittance of the first color filter array.

[0168] The control driving unit drives the first color filter layer, the second color filter layer and the third color filter layer to move so that all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap with each other, all the blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of multiple color filter layers overlap with each other.

[0169] Acquire a first image captured by a photosensitive layer of an image sensor; control a driving unit of the image sensor to drive the relative positions of a plurality of color filter layers to change at least once; acquire a second image captured by the photosensitive layer after each change in the relative positions of the plurality of color filter layers to obtain at least one second image; generate a target image based on the first image and the at least one second image.

[0170] When the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first relative position relationship, the driving unit of the image sensor is controlled to drive the target color filter layer to move so that the relative position between the target color filter layer and the photosensitive layer changes at least once, and the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one second corresponding relationship; wherein, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the first relative position relationship, the photosensitive pixels are aligned with one color filter pixel; when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the second relative position relationship, the photosensitive pixels are aligned with another color filter pixel.

[0171] In an embodiment of the present invention, when the above-mentioned device of the present invention is used to capture a first image and at least one second image and perform image synthesis, there is no need to align the pixels of the first image and the at least one second image one by one, which not only improves the speed of image synthesis, but also avoids the occurrence of poor synthesis effects due to alignment errors, further improves the shooting quality of the target image, or enables each photosensitive pixel to obtain multiple color signals, and color restoration can be performed based on the multiple color information obtained by the photosensitive pixels, or there is no need to perform color restoration at all, thereby improving the color accuracy of the target image finally output.

[0172] It should be understood that in the embodiment of the present invention, the radio frequency unit 501 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 510 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 501 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 501 can also communicate with the network and other devices through a wireless communication system.

[0173] The electronic device provides users with wireless broadband Internet access through the network module 502, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0174] The audio output unit 503 can convert the audio data received by the RF unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output it as sound. Moreover, the audio output unit 503 can also provide audio output related to a specific function performed by the electronic device 500 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, etc.

[0175] The input unit 504 is used to receive audio or video signals. The input unit 504 may include a graphics processor (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 506. The image frame processed by the graphics processor 5041 can be stored in the memory 509 (or other storage medium) or sent via the radio frequency unit 501 or the network module 502. The microphone 5042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 501 in the case of a telephone call mode.

[0176] The electronic device 500 also includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 5061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 5061 and / or the backlight when the electronic device 500 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 505 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0177] The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0178] The user input unit 507 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. The touch panel 5071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 5071 or near the touch panel 5071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 5071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the contact point coordinates, and then sends it to the processor 510, receives the command sent by the processor 510 and executes it. In addition, the touch panel 5071 can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch panel 5071, the user input unit 507 may also include other input devices 5072. Specifically, other input devices 5072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail here.

[0179] Furthermore, the touch panel 5071 may be overlaid on the display panel 5061. When the touch panel 5071 detects a touch operation on or near it, it transmits the information to the processor 510 to determine the type of the touch event. Then, the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. Fig.17 In the figure, the touch panel 5071 and the display panel 5061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0180] The interface unit 508 is an interface for connecting an external device to the electronic device 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 500 or may be used to transmit data between the electronic device 500 and an external device.

[0181] The memory 509 can be used to store software programs and various data. The memory 509 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 509 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0182] The processor 510 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 509 and calling data stored in the memory 509, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 510.

[0183] The electronic device 500 may also include a power supply 511 (such as a battery) for supplying power to each component. Preferably, the power supply 511 may be logically connected to the processor 510 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0184] The electronic device 500 may further include a camera module, which may be logically connected to the processor 510 and transmit captured images to the processor 510 .

[0185] The camera module may also include an image sensor, which can receive light and generate an image signal.

[0186] The image sensor may include a photosensitive layer, multiple color filter layers, and a driving unit connected to the multiple color filter layers; the multiple color filter layers are superimposed on the light receiving surface of the photosensitive layer; the driving unit is used to drive the relative positions of the multiple color filter layers to change.

[0187] The camera module may also include a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer; the target color filter layer is arranged on the light receiving surface of the photosensitive layer; the driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change.

[0188] Optionally, the driving unit drives the plurality of color filter layers to switch between a first state and a second state;

[0189] In the first state, the plurality of color filter layers have a first relative position relationship, and the plurality of color filter layers overlap each other and form a first color filter array;

[0190] In the second state, the plurality of color filter layers have a second relative position relationship, and the plurality of color filter layers overlap each other and form a second color filter array;

[0191] Wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array.

[0192] Optionally, the plurality of color filter layers include a first color filter layer, a second color filter layer and a third color filter layer;

[0193] The first color filter layer includes a plurality of 4×4 first color filter units, the first color filter unit includes four 2×2 first color filter groups, the first color filter group includes one red color filter pixel and three full-transmittance color filter pixels;

[0194] The second color filter layer includes a plurality of 4×4 second color filter units, the second color filter units include four 2×2 second color filter groups, the second color filter groups include two diagonally arranged red color filter pixels and two diagonally arranged full-transmittance color filter pixels;

[0195] The third color filter layer includes a plurality of 4×4 third color filter units, the third color filter unit includes four 2×2 third color filter groups, and the first color filter group includes one blue color filter pixel and three full-transmission color filter pixels.

[0196] Optionally, when the plurality of color filter layers are stacked on each other to form a first color filter array, all red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap each other with all blue color filter pixels of the third color filter layer;

[0197] When the multiple color filter layers are stacked on each other and form a second color filter array, all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap with each other, all the blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of the multiple color filter layers overlap with each other.

[0198] Optionally, the driving unit is connected to side edges of the plurality of color filter layers, and the driving unit is used to drive the plurality of color filter layers to move in a direction perpendicular to the side edges.

[0199] Optionally, the driving unit includes a first driving unit connected to the first color filter layer, a second driving unit connected to the second color filter layer, and a third driving unit connected to the third color filter layer;

[0200] The first driving unit is used to drive the first color filter layer to move in a first direction;

[0201] The second driving unit is used for driving the second color filter layer to move in the first direction;

[0202] The third driving unit is used to drive the third color filter layer to move in a second direction, and the first direction is opposite to the second direction.

[0203] Optionally, the driving unit comprises an electrostatic comb-tooth driver, and the electrostatic comb-tooth driver comprises a power supply, a first comb-tooth portion, and a second comb-tooth portion;

[0204] The second comb-tooth portion is connected to the plurality of color filter layers;

[0205] The power source is connected to the first comb-tooth portion and the second comb-tooth portion, and the power source is used to drive the first comb-tooth portion and the second comb-tooth portion to move relative to each other.

[0206] The image sensor may further include a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer;

[0207] The target color filter layer is arranged on the light receiving surface of the photosensitive layer;

[0208] The driving unit is used for driving the relative position between the target color filter layer and the photosensitive layer to change.

[0209] Optionally, the target color filter layer includes a color filter pixel array, the color filter pixel array includes at least three types of color filter pixels, and the photosensitive layer includes a photosensitive pixel array;

[0210] The color filter pixels of the color filter pixel array overlap one-to-one with the photosensitive pixels of the photosensitive pixel array;

[0211] The number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array.

[0212] Optionally, the target color filter layer includes a plurality of 4×4 color filter units; the color filter unit includes four 2×2 target color filter groups, and each of the target color filter groups includes three color filter pixels: red, green and blue.

[0213] Optionally, the driving unit comprises an electrostatic comb-tooth driver, and the electrostatic comb-tooth driver comprises a power supply, a first comb-tooth portion, and a second comb-tooth portion;

[0214] The second comb-tooth portion is connected to the target color filter layer;

[0215] The power source is connected to the first comb-tooth portion and the second comb-tooth portion, and the power source is used to drive the first comb-tooth portion and the second comb-tooth portion to move relative to each other.

[0216] In addition, the electronic device 500 includes some functional modules not shown, which will not be described in detail here.

[0217] Preferably, an embodiment of the present invention further provides a mobile terminal, including a processor 510, a memory 509, and a computer program stored in the memory 509 and executable on the processor 510. When the computer program is executed by the processor 510, each process of the above-mentioned image generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0218] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned image generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0219] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0220] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0221] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. An image sensor, characterized in that: include: A photosensitive layer, a plurality of color filter layers, and a driving unit connected to the plurality of color filter layers; The plurality of color filter layers are stacked on the light receiving surface of the photosensitive layer; the color filter layer further comprises at least two types of color filter pixels; the color filter pixels allow light of a specific wavelength to pass through; the color filter layer comprises fully transparent color filter pixels; The driving unit is used to drive the relative positions of the multiple color filter layers to change, and the multiple color filter layers are superimposed to generate at least two color filter arrays with different transmittances. The sensor is used to synthesize the at least two images captured by the color filter arrays into a target image.

2. The image sensor according to claim 1, characterized in that The driving unit drives the plurality of color filter layers to switch between a first state and a second state; In the first state, the plurality of color filter layers have a first relative position relationship, and the plurality of color filter layers overlap each other and form a first color filter array; In the second state, the plurality of color filter layers have a second relative position relationship, and the plurality of color filter layers overlap each other and form a second color filter array; Wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array.

3. The image sensor according to claim 2, characterized in that The plurality of color filter layers include a first color filter layer, a second color filter layer and a third color filter layer; The first color filter layer includes a plurality of 4×4 first color filter units, the first color filter unit includes four 2×2 first color filter groups, the first color filter group includes one red color filter pixel and three full-transmittance color filter pixels; The second color filter layer includes a plurality of 4×4 second color filter units, the second color filter units include four 2×2 second color filter groups, the second color filter groups include two diagonally arranged red color filter pixels and two diagonally arranged full-transmittance color filter pixels; The third color filter layer includes a plurality of 4×4 third color filter units, the third color filter unit includes four 2×2 third color filter groups, and the first color filter group includes one blue color filter pixel and three full-transmission color filter pixels.

4. The image sensor according to claim 3, characterized in that: When the plurality of color filter layers are stacked on each other to form a first color filter array, all red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap each other with all blue color filter pixels of the third color filter layer; When the multiple color filter layers are stacked on each other and form a second color filter array, all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap with each other, all the blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of the multiple color filter layers overlap with each other.

5. The image sensor according to claim 3, characterized in that: The driving unit is connected to the side edges of the plurality of color filter layers, and the driving unit is used to drive the plurality of color filter layers to move in a direction perpendicular to the side edges.

6. The image sensor according to claim 3, characterized in that: The driving unit comprises a first driving unit connected to the first color filter layer, a second driving unit connected to the second color filter layer, and a third driving unit connected to the third color filter layer; The first driving unit is used to drive the first color filter layer to move in a first direction; The second driving unit is used for driving the second color filter layer to move in the first direction; The third driving unit is used to drive the third color filter layer to move in a second direction, and the first direction is opposite to the second direction.

7. The image sensor according to claim 1, characterized in that: The driving unit includes an electrostatic comb drive, and the electrostatic comb drive includes a power supply, a first comb portion and a second comb portion; The second comb-tooth portion is connected to the plurality of color filter layers; The power source is connected to the first comb-tooth portion and the second comb-tooth portion, and the power source is used to drive the first comb-tooth portion and the second comb-tooth portion to move relative to each other.

8. A camera module, characterized in that: Comprising the image sensor as claimed in any one of claims 1 to 7.

9. An electronic device, characterized in that: Comprising the camera module as claimed in claim 8.

10. An image generation method, applied to an electronic device, characterized in that: The electronic device comprises the image sensor according to any one of claims 1 to 7, and the method comprises: Acquire a first image captured by a photosensitive layer of the image sensor; Controlling a driving unit of the image sensor to drive the relative positions of the plurality of color filter layers to change at least once; After each change in the relative positions between the plurality of color filter layers, acquiring a second image captured by the photosensitive layer to obtain at least one second image; A target image is generated based on the first image and the at least one second image.

11. The method according to claim 10, characterized in that The plurality of color filter layers include a first color filter layer composed of red color filter pixels and full-transmittance color filter pixels, a second color filter layer composed of green color filter pixels and full-transmittance color filter pixels, and a third color filter layer composed of blue color filter pixels and full-transmittance color filter pixels; The driving unit controlling the image sensor drives the relative positions of the plurality of color filter layers to change at least once, comprising: In a case where the color filter pixel arrays of the plurality of color filter layers are overlapped to form a first color filter array, controlling the driving unit of the image sensor to drive the plurality of color filter layers to move so that the relative positions between the plurality of color filter layers are changed at least once, and the color filter pixel arrays of the plurality of color filter layers are overlapped to form at least one second color filter array; Among them, in the first color filter array, all red color filter pixels of the first color filter layer and half of the fully transparent color filter pixels of the second color filter layer overlap with each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with all blue color filter pixels of the third color filter layer; the transmittance of the second color filter array is greater than the transmittance of the first color filter array.

12. The method according to claim 11, characterized in that The driving unit controlling the image sensor drives the plurality of color filter layers to move, comprising: The driving unit is controlled to drive the first color filter layer, the second color filter layer and the third color filter layer to move, so that all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other, all the blue color filter pixels of the second color filter layer and the other half of the green color filter pixels of the second color filter layer are aligned, and the fully transparent color filter pixels of the multiple color filter layers overlap each other.

13. An image sensor, characterized in that: include: A photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer; The target color filter layer is arranged on the light receiving surface of the photosensitive layer; The driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change. The image sensor is made to work at different color filter layer positions, and the color filter array of the image sensor is changed, so that the electronic device can capture multiple images based on different color filter arrays through the image sensor, and synthesize the multiple images to obtain a target image; Wherein, the target color filter layer includes a color filter pixel array, the color filter pixel array includes at least three types of color filter pixels, and the photosensitive layer includes a photosensitive pixel array; The color filter pixels of the color filter pixel array overlap one-to-one with the photosensitive pixels of the photosensitive pixel array; The number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array; When the number of the target color filter layer is equal to 1, the color filter array of the target color filter layer is composed of an equal number of red color filter pixels, green color filter pixels and blue color filter pixels, so that when the relative position of the target color filter layer with respect to the photosensitive layer changes, the color of the color filter pixels corresponding to all the photosensitive pixels on the photosensitive layer changes.

14. The image sensor according to claim 13, characterized in that: The target color filter layer includes a plurality of 4×4 color filter units; the color filter unit includes four 2×2 target color filter groups, and each of the target color filter groups includes three color filter pixels: red, green and blue.

15. The image sensor according to claim 13, characterized in that: The driving unit includes an electrostatic comb drive, and the electrostatic comb drive includes a power supply, a first comb portion and a second comb portion; The second comb-tooth portion is connected to the target color filter layer; The power source is connected to the first comb-tooth portion and the second comb-tooth portion, and the power source is used to drive the first comb-tooth portion and the second comb-tooth portion to move relative to each other.

16. A camera module, characterized in that: Comprising the image sensor as claimed in any one of claims 13 to 15.

17. An electronic device, characterized in that: Comprising the camera module as described in claim 16.

18. An image generation method, applied to an electronic device, characterized in that: The electronic device comprises the image sensor according to any one of claims 13 to 15, and the method comprises: Acquire a first image captured by a photosensitive layer of the image sensor; Controlling a driving unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once; After each change in the relative position between the target color filter layer and the photosensitive layer, a second image captured by the photosensitive layer is acquired to obtain at least one second image; A target image is generated based on the first image and the at least one second image.

19. The method according to claim 18, characterized in that The driving unit controlling the image sensor drives the relative position between the target color filter layer and the photosensitive layer to change at least once, including: In a case where the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first relative position relationship, controlling the driving unit of the image sensor to drive the target color filter layer to move, so that the relative position between the target color filter layer and the photosensitive layer changes at least once, and the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one second corresponding relationship; Wherein, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the first relative position relationship, the photosensitive pixels are aligned with one color filter pixel; when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the second relative position relationship, the photosensitive pixels are aligned with another color filter pixel.

Images