Image sensor, camera module, shooting method, shooting device and electronic equipment
By using moving color filter array combination technology in the image sensor, the problem of insufficient color information in the image sensor when there is insufficient light or poor environment is solved, and image quality and color diversity are improved.
Patent Information
- Application Number
- CN202210731345.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-24
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-06-24
AI Technical Summary
When the image sensors in existing electronic devices lack sufficient light or poor environment, they obtain insufficient color information, resulting in poor quality of the captured images.
Using an image sensor including a photosensitive array, a first color filter array, a second color filter array and a driver, the target color filter array is controlled to move in its plane through the driver to form a different combination of color filter arrays, thereby improving image sensor acquisition Color information richness.
Improves the quality of images taken by electronic devices and enhances the diversity and accuracy of color information, especially in the absence of insufficient light or poor environment.
Smart Images

Figure CN115278002B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of camera technology, and specifically relates to an image sensor, a camera module, a shooting method, a shooting device and an electronic device. Background Art
[0002] With the development of electronic devices, the functions of electronic devices are becoming more and more abundant. For example, images can be captured by electronic devices.
[0003] Specifically, when an electronic device is used to capture an image, if the type of filters included in the image sensor of the electronic device is insufficient or the current shooting environment is poor (e.g., insufficient light, etc.), the image sensor may not obtain sufficient color information, which may result in poor quality of the image captured by the electronic device. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide an image sensor, a camera module, a shooting method, a shooting device and an electronic device, which can improve the quality of images taken by the electronic device.
[0005] In a first aspect, an embodiment of the present application provides an image sensor, comprising: a photosensitive array, a first color filter array, a second color filter array, and at least one driving element; the first color filter array and the second color filter array are arranged in a superimposed manner, and the first color filter array and the second color filter array are both arranged on the light incident side of the photosensitive array; at least one driving element is connected to a target color filter array, and the target color filter array is at least one of the first color filter array and the second color filter array; wherein at least one driving element is used to control the target color filter array to move within the plane where the target color filter array is located; before and after the target color filter array moves, the combined color filter array composed of the two color filter arrays is different.
[0006] In a second aspect, an embodiment of the present application provides a camera module, which includes the image sensor as described in the first aspect.
[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes the camera module as described in the second aspect.
[0008] In a fourth aspect, an embodiment of the present application provides a shooting method, which is applied to an electronic device such as the third aspect, the method comprising: when a first color filter array and a second color filter array in a camera module of the electronic device are in an initial state, collecting a first original image through the camera module; controlling at least one driving member of the camera module to move at least once to drive a target color filter array to move at least once within a plane where the target color filter array is located, and after each movement of the target color filter array is completed, controlling the camera module to collect a group of original images, each group of original images including at least one second original image; outputting a target image based on the first original image and at least one second original image in each group of original images; wherein the target color filter array is at least one item of the first color filter array and the second color filter array, and the color parameters of the second original images in different groups are different.
[0009] In a fifth aspect, an embodiment of the present application provides a shooting device, which includes: an acquisition module, a control module and a processing module; the acquisition module is used to acquire a first original image through the camera module when the first color filter array and the second color filter array in the camera module of the electronic device are in an initial state; the control module is used to control at least one driving member of the camera module to move at least once, so as to drive the target color filter array to move at least once in the plane where the target color filter array is located, and after each movement of the target color filter array is completed, control the camera module to acquire a group of original images, each group of original images including at least one second original image; the processing module is used to output a target image based on the first original image acquired by the acquisition module and at least one second original image in each group of original images; wherein the target color filter array is at least one item of the first color filter array and the second color filter array, and the color parameters of the second original images in different groups are different.
[0010] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method in the fourth aspect.
[0011] In a seventh aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method in the fourth aspect are implemented.
[0012] In an eighth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method of the fourth aspect.
[0013] In a ninth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the fourth aspect.
[0014] In an embodiment of the present application, the image sensor includes: a photosensitive array, a first color filter array, a second color filter array, and at least one driving member; and at least one driving member can be used to control the target color filter array to move within the plane where the target color filter array is located, wherein the target color filter array is at least one of the first color filter array and the second color filter array; before and after the target color filter array moves, the combined color filter array composed of the first color filter array and the second color filter array is different. This can improve the richness of color information obtained by the image sensor, and thus improve the quality of the captured image. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 One of the working schematic diagrams of the piezoelectric material provided in the embodiment of the present application;
[0016] Figure 2 The second working schematic diagram of the piezoelectric material provided in the embodiment of the present application;
[0017] Figure 3 One of the structural schematic diagrams of the image sensor provided in the embodiment of the present application;
[0018] Figure 4 A schematic diagram of the color filter array direction provided in an embodiment of the present application;
[0019] Figure 5 One of the structural schematic diagrams of the image sensor provided in the embodiment of the present application before at least one driving member drives the target color filter array to move;
[0020] Figure 6 One of the structural schematic diagrams after at least one driving element in the image sensor provided by the embodiment of the present application drives the target color filter array to move a pixel distance along the row direction of the target color filter array;
[0021] Figure 7 The second structural diagram of the image sensor provided in the embodiment of the present application;
[0022] Figure 8 The third structural diagram of the image sensor provided in the embodiment of the present application;
[0023] Fig. 9 The fourth structural diagram of the image sensor provided in the embodiment of the present application;
[0024] Fig.10The fifth structural diagram of the image sensor provided in the embodiment of the present application;
[0025] Fig.11 The sixth structural diagram of the image sensor provided in the embodiment of the present application;
[0026] Fig.12 The second structural schematic diagram of the image sensor provided in the embodiment of the present application before at least one driving member drives the target color filter array to move;
[0027] Fig.13 The second structural schematic diagram of the image sensor provided in the embodiment of the present application after at least one driving element drives the target color filter array to move a pixel distance along the row direction of the target color filter array;
[0028] Fig.14 A schematic diagram of the structure after at least one driving element in the image sensor provided by an embodiment of the present application drives the target color filter array to move a pixel distance along the column direction of the target color filter array;
[0029] Fig.15 The seventh structural diagram of the image sensor provided in the embodiment of the present application;
[0030] Fig.16 The eighth structural diagram of the image sensor provided in the embodiment of the present application;
[0031] Fig.17 A ninth structural diagram of an image sensor provided in an embodiment of the present application;
[0032] Fig.18 The tenth structural diagram of the image sensor provided in the embodiment of the present application;
[0033] Fig.19 The eleventh structural diagram of the image sensor provided in the embodiment of the present application;
[0034] Fig. 20 The twelfth structural diagram of the image sensor provided in the embodiment of the present application;
[0035] Fig.21 A flowchart of a shooting method provided in an embodiment of the present application;
[0036] Fig. 22 This is one of the example schematic diagrams of the shooting method provided in the embodiment of the present application;
[0037] Fig.23 This is the second example schematic diagram of the shooting method provided in the embodiment of the present application;
[0038] Fig.24 A schematic diagram of a photographing device provided in an embodiment of the present application;
[0039] Fig.25 A schematic diagram of an electronic device provided in an embodiment of the present application;
[0040] Fig.26 A hardware schematic diagram of an electronic device provided in an embodiment of the present application.
[0041] in, Figures 3 to 20 The reference numerals in the figures are as follows:
[0042] 200: image sensor; 201: photosensitive array; 202: first color filter array; 203: second color filter array; 204: driving element; 204-1: first driving element; 204-2: second driving element; 205: light. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0044] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0045] First, some nouns or terms involved in the claims and description of this application are explained below.
[0046] The image sensor is the core device of the camera. Sensors can be divided into two types: one is the widely used charge-coupled device (CCD) element; the other is the complementary metal oxide semiconductor (CMOS) device. Compared with traditional cameras, traditional cameras use "film" as the carrier of their information recording, while the "film" of digital cameras is their imaging photosensitive element. The photosensitive element is the "film" of digital cameras that does not need to be replaced and is integrated with the camera.
[0047] CMOS devices, like CCD, are semiconductors that can record light changes in digital cameras. The CMOS workflow includes sensing light signals through a large number of photodiodes (pixels), then converting them into electrical signals, passing through amplification circuits and digital / analog conversion circuits to form a digital signal matrix (i.e., image), which is then processed by an image signal processor and compressed and stored.
[0048] CMOS Camera Module is a camera module widely used in mobile phones. It is mainly composed of lens, voice coil motor, infrared filter, image sensor (CMOS), digital signal processor (DSP) and flexible printed circuit board (FPC).
[0049] The working process of CCD camera module includes the voice coil motor driving the lens to move to the accurate focus position, the external light passes through the lens, is filtered by the infrared filter, and shines on the photosensitive diode (pixel) of the image sensor. The photosensitive diode converts the sensed light signal into an electrical signal, which is then passed through the amplification circuit and the digital / analog conversion circuit to form a digital signal matrix (i.e., image), which is then processed by the DSP and compressed and stored.
[0050] The image sensor includes a color filter array (CFA). General photoelectric sensors can only sense the intensity of light but cannot distinguish the wavelength (color) of light. Therefore, the image sensor needs to pass through a color filter to obtain the color information of the pixel. CFA filters light according to wavelength, and a specific CFA only allows light of a specific wavelength to pass through.
[0051] When an electric field or voltage is applied to the surface of a piezoelectric material, the electric dipole moment will be stretched due to the electric field. To resist the change, the piezoelectric material will stretch along the direction of the electric field. This process of mechanical deformation caused by the electric field is called the "inverse piezoelectric effect". The inverse piezoelectric effect is essentially the process of converting electrical energy into mechanical energy; for example, Figure 1 As shown in the figure, this is the state of the piezoelectric material when it is not subjected to an electric field. Figure 2 As shown, this is the state of the piezoelectric material when it is subjected to an electric field; the piezoelectric material can be a single crystal, ceramic, thin film, etc.
[0052] Filters are a type of optical element that selectively selects spectral wavelengths for transmission, reflection, cutoff, and attenuation. According to their functions, they can be divided into bandpass filters, long-wave pass filters, short-wave pass filters, dichroic filters, neutral density filters, notch filters, cold mirrors, hot mirrors, color filters, etc. They are divided into interference filters and color absorption filters. These filters are widely used in spectral analysis, biomedicine, machine vision, fluorescence microscopy, industrial automation and other industries and product fields. Bandpass filters selectively allow certain spectral wavelengths to pass through and prevent certain spectral wavelengths from passing through within their spectral range.
[0053] Image sensors usually have fixed-pattern noise (FPN). This FPN is a fixed noise that appears as fixed vertical stripes (Column FPN) or horizontal stripes (Row FPN) in severe cases. Sensor suppliers generally call it VFPN and HFPN. Generally speaking, FPN depends on factors such as the mismatch of column comparators leading to differences in outputs of different columns, or the coupling between different rows leading to horizontal stripes; and poor power supply systems, mismatched clock source signals, and module layer design may also cause FPN problems. The severity of this noise often varies with the temperature of the entire electronic device and the sensor analog gain, and it cannot be processed by noise reduction algorithms. When an electronic device is taking pictures in a polar night environment, FPN will bring a very bad look and feel due to the weak image signal.
[0054] The image sensor, camera module, shooting method, shooting device and electronic device provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0055] The image sensor provided in the embodiment of the present application can be applied to scenes of capturing images.
[0056] Exemplarily, when an image is captured by an electronic device, since the image sensor includes a first color filter array, a second color filter array and at least one driving member; the at least one driving member can be used to control the target color filter array to move within the plane where the target color filter array is located, wherein the target color filter array is at least one of the first color filter array and the second color filter array; and before and after the target color filter array moves, the combined color filter array composed of the first color filter array and the second color filter array is different. This can improve the richness of color information acquired by the image sensor, and thus improve the quality of the image captured by the electronic device.
[0057] The present application embodiment provides an image sensor, Figure 3A possible structural diagram of the image sensor 200 provided in an embodiment of the present application is shown. Figure 3 As shown, the image sensor 200 provided in the embodiment of the present application may include: a photosensitive array 201 , a first color filter array 202 , a second color filter array 203 , and at least one driving element 204 .
[0058] In the embodiment of the present application, the first color filter array and the second color filter array are superimposed, and the first color filter array and the second color filter array are both arranged on the light incident side of the photosensitive array, thereby ensuring that the photosensitive array can obtain the color information filtered by the first color filter array and the second color filter array.
[0059] It should be noted that the number of the photosensitive array, the first color filter array and the second color filter array shown in the embodiment of the present application is only for reference, and the specific number of the photosensitive array, the first color filter array and the second color filter array can be determined according to actual needs.
[0060] Alternatively, if Figure 3 As shown, the at least one driving element 204 is connected to a target color filter array, and the target color filter array is at least one of the first color filter array 202 and the second color filter array 203 .
[0061] Among them, at least one driving member 204 can be used to control the target color filter array to move within the plane where the target color filter array is located; before and after the target color filter array moves, the combined color filter array composed of the first color filter array 202 and the second color filter array 203 can be different.
[0062] Optionally, the at least one driving member may be used to control the target color filter array to move in any direction within the plane where the target color filter array is located, such as horizontally, i.e., in the row direction, vertically, i.e., in the column direction, or in an irregular direction, i.e., diagonally.
[0063] For example, Figure 4 As shown, the direction of the X-axis of the rectangular coordinate system can be used as the row direction of the target color filter array, and the direction of the Y-axis of the rectangular coordinate system can be used as the column direction of the target color filter array, wherein the direction of the X-axis can be to the left or to the right; the direction of the Y-axis can be to the upward or downward direction.
[0064] Optionally, when the at least one driving member is connected to both the first color filter array and the second color filter array, the at least one driving member can control the first color filter array and the second color filter array to move together, and can respectively control the first color filter array and the second color filter array to move in different directions.
[0065] Optionally, when the number of the at least one driving element is 1, one driving element may be connected to any one of the first color filter array and the second color filter array; or, one driving element may be connected to both the first color filter array and the second color filter array.
[0066] Optionally, when the number of the at least one driving element is at least two, different driving elements may be connected to different color filter arrays in the first color filter array and the second color filter array.
[0067] In the embodiment of the present application, the combined color filter array is a color filter array formed by superimposing the first color filter array and the second color filter array. It can be understood that different combined color filter arrays have different color information through the combined color filter arrays, and thus the color information obtained by the photosensitive array is different.
[0068] Optionally, in the embodiment of the present application, the at least one driving element may be specifically used to control the target color filter array to move an odd number of pixels within the plane where the target color filter array is located.
[0069] Specifically, the at least one driving element can be used to control the target color filter array to move an odd number of pixels along the row direction or column direction of the target color filter array, so that after the target color filter array moves, the first color filter array and the second color filter array can be combined into a new color filter array.
[0070] In the embodiment of the present application, since at least one driving member can be used to control the target color filter array to move an odd number of pixels within the plane where the target color filter array is located, after the target color filter array moves, the first color filter array and the second color filter array can be combined into a new color filter array, thereby making the color information of external light passing through the first color filter array and the second color filter array different.
[0071] Optionally, in an embodiment of the present application, after the target color filter array moves, the effective photosensitive area of the photosensitive array is located within the projection area of the combined color filter array on the photosensitive array.
[0072] It can be understood that the effective photosensitive area of the photosensitive array in the embodiment of the present application is the photosensitive area where the image sensor uses the color information obtained by the photosensitive array as the subsequent image processing.
[0073] In the embodiment of the present application, the projection area of the combined color filter array on the photosensitive array may be the projection area of the combined color filter array on the photosensitive array along the optical axis direction of the image sensor.
[0074] In the embodiment of the present application, since the effective photosensitive area of the photosensitive array is located within the projection area of the combined color filter array on the photosensitive array after the target color filter array moves, it can be ensured that the color information acquired by the effective photosensitive area of the photosensitive array is the color information filtered by the combined color filter array.
[0075] Optionally, the at least one driving member may be fixedly connected to the target color filter array; or may be movably connected, which may be determined according to actual use requirements and is not limited in the embodiments of the present application. For example, the at least one driving member may be fixedly connected to the target color filter array by welding.
[0076] Optionally, the driving member may be a device capable of generating deformation; or may be a device capable of generating movement. For example, the driving member may be a movable device made of an electrical structure or a mechanical structure. The driving member may also be a deformable material made of a deformable material.
[0077] Optionally, in the embodiment of the present application, at least one driving member may be made of piezoelectric material.
[0078] When the driving element applies voltage, the driving element may deform along the plane where the target color filter array is located, and may drive the target color filter array to move an odd number of pixels on the plane where the target color filter array is located.
[0079] In the embodiments of the present application, for the relevant description of the piezoelectric material, please refer to the relevant description in the above-mentioned glossary. In order to avoid repetition, it will not be repeated here.
[0080] Specifically, the driving element may drive the target color filter array to move an odd number of pixels in the column direction or the row direction of the plane where the target color filter array is located.
[0081] In the embodiment of the present application, since at least one driving element is made of piezoelectric material, when a voltage is applied to the driving element, the driving element deforms along the plane where the target color filter array is located, and drives the target color filter array to move an odd number of pixels in the plane where the target color filter array is located, thereby ensuring that after the target color filter array is moved, the combined color filter array composed of the first color filter array and the second color filter array can obtain different color information.
[0082] In an image sensor provided by an embodiment of the present application, the image sensor includes: a photosensitive array, a first color filter array, a second color filter array, and at least one driving member; and at least one driving member can be used to control the target color filter array to move within the plane where the target color filter array is located, wherein the target color filter array is at least one of the first color filter array and the second color filter array; before and after the target color filter array moves, the combined color filter array composed of the first color filter array and the second color filter array is different. This can improve the richness of color information obtained by the image sensor, and thus improve the quality of the captured image.
[0083] The image sensor provided by the embodiment of the present application is further described below in combination with two possible implementations, wherein one possible implementation is: one of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYYM array, and the other of the first color filter array and the second color filter array is a blue-red-green-blue BRGB array. Another possible implementation is: one of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array, and the other of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array.
[0084] Optionally, in some embodiments of the present application, one of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYYM array, and the other color filter array in the first color filter array and the second color filter array is a blue-red-green-blue BRGB array.
[0085] Among them, at least one driving member may include a driving member, and the driving member can be used to control the target color filter array to move a first preset distance along the row direction of the target color filter array, and the first preset distance can be an odd number of pixel distances; after the driving member controls the target color filter array to move a first preset distance along the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is a (Y+B)(C+R)(M+G)(Y+B) array.
[0086] It can be understood that when the first color filter array is a CYYM array, the second color filter array is a BRGB array; when the first color filter array is a BRGB array, the second color filter array is a CYYM array.
[0087] For ease of explanation, the following embodiments are described by taking the first color filter array being a CYYM array and the second color filter array being a BRGB array as an example. The same is true when the first color filter array is a BRGB array and the second color filter array is a CYYM array. To avoid repetition, they are not described here.
[0088] For example, Figure 5 As shown, assuming that the target color filter array is the second color filter array 203, a driving element 204 can be connected to the second color filter array 203, then Figure 6 As shown, the driving element 204 can control the second color filter array 203 to move an odd number of pixels along the row direction of the first color filter array 203 .
[0089] For example, Figure 7 As shown, assuming that the target color filter array is the first color filter array 202, a driver 204 can be connected to the first color filter array 202, and the driver 204 can control the first color filter array 202 to move an odd number of pixels along the row direction of the first color filter array 202.
[0090] In the embodiment of the present application, in the initial state, that is, when neither the first color filter array nor the second color filter array moves, the C filter in the first color filter array corresponds to the B filter in the second color filter array, the Y filter in the first color filter array corresponds to the R filter in the second color filter array, the Y filter in the first color filter array corresponds to the G filter in the second color filter array, and the M filter in the first color filter array corresponds to the B filter in the second color filter array. Therefore, in the initial state, the combined color filter array composed of the first color filter array and the second color filter array is a (C+B)(Y+R)(Y+G)(M+B) array. Since in the optical field, the B filter can only pass blue (B), the R filter can only pass red (R), the G filter can only pass green (G), the C filter can only pass green (G) and blue (B), the Y filter can only pass green (G) and red (R), and the M filter can only pass blue (B) and red (R). Therefore, the combined color filter array can pass three kinds of color information, RGB.
[0091] After a driver controls the target color filter array to move an odd number of pixels along the row direction of the target color filter array, the C filter in the first color filter array corresponds to the R filter in the second color filter array, the Y filter in the first color filter array corresponds to the B filter in the second color filter array, the Y filter in the first color filter array corresponds to the B filter in the second color filter array, and the M filter in the first color filter array corresponds to the G filter in the second color filter array. Therefore, after a driver controls the target color filter array to move an odd number of pixels along the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is a (Y+B)(C+R)(M+G)(Y+B) array. In the optical field, the B filter can only pass blue (B), the R filter can only pass red (R), the G filter can only pass green (G), the C filter can only pass green (G) and blue (B), the Y filter can only pass green (G) and red (R), and the M filter can only pass blue (B) and red (R). Therefore, the combined color filter array cannot pass any color information.
[0092] For example, Figure 8 As shown, the first color filter array 202 is a CYYM array. Fig. 9 As shown, the second color filter array 203 is a BRGB array. In the initial state, as shown in FIG. Fig.10 As shown, the combined color filter array composed of the first color filter array and the second color filter array is a (C+B)(Y+R)(Y+G)(M+B) array. After a driving element controls the target color filter array to move an odd number of pixels along the row direction of the target color filter array, as shown in FIG. Fig.11 As shown, the combined color filter array composed of the first color filter array and the second color filter array is a (Y+B)(C+R)(M+G)(Y+B) array.
[0093] The at least one driving element in the embodiment of the present invention may include a plurality of driving elements, and under the action of the plurality of driving elements, the target color filter array may be controlled to move a first preset distance along a row direction of the target color filter array.
[0094] In the embodiment of the present application, after the target color filter array is controlled by the driving component to move an odd number of pixels in the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is: (Y+B)(C+R)(M+G)(Y+B) array, so that the combined color filter array cannot pass any external color information.
[0095] Optionally, in some embodiments of the present application, one of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array, and the other color filter array in the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array.
[0096] Wherein, at least one driver may include at least one first driver and at least one second driver, and each first driver may be arranged adjacent to a second driver. At least one first driver may be used to control the target color filter array to move a second preset distance along the row direction of the target color filter array, and the second preset distance may be an odd number of pixel distances; after at least one first driver controls the target color filter array to move a second preset distance along the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is: (Y+C)(C+Y)(Y+M)(M+Y) array; at least one second driver may be used to control the target color filter array to move a third preset distance along the column direction of the target color filter array, and the third preset distance may be an odd number of pixel distances; after at least one second driver controls the target color filter array to move a third preset distance along the column direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is (M+C)(Y+Y)(C+M)(Y+Y) array.
[0097] For example, Fig.12 As shown, at least one driving member 204 includes a first driving member 204-1 and a second driving member 204-2, and the first driving member 204-1 and the second driving member 204-2 are arranged adjacent to each other. Assuming that the target color filter array is the second color filter array 203, then Fig.13 As shown, the first driving element 204-1 can be used to control the second color filter array 203 to move a distance of 1 pixel along the row direction of the second color filter array 203; Fig.14 As shown, the second driving element 204 - 2 can be used to control the second color filter array 203 to move a distance of 1 pixel along the column direction of the second color filter array 203 .
[0098] For example, Fig.15 As shown, at least one driver 204 includes a first driver 204-1 and a second driver 204-2, and the first driver 204-1 and the second driver 204-2 are arranged adjacent to each other. The first driver 204-1 can be used to control the target color filter array to move an odd number of pixels along the row direction of the target color filter array; the second driver 204-2 can be used to control the target color filter array to move an odd number of pixels along the column direction of the target color filter array.
[0099] In the embodiment of the present application, in the initial state, that is, when neither the first color filter array nor the second color filter array moves, the C filter in the first color filter array corresponds to the C filter in the second color filter array, the Y filter in the first color filter array corresponds to the Y filter in the second color filter array, the M filter in the first color filter array corresponds to the M filter in the second color filter array, and the Y filter in the first color filter array corresponds to the Y filter in the second color filter array. Therefore, in the initial state, the combined color filter array composed of the first color filter array and the second color filter array is a (C+C)(Y+Y)(M+M)(Y+Y) array, that is, a CYMY array. In the optical field, the B filter can only pass blue (B), the R filter can only pass red (R), the G filter can only pass green (G), the C filter can only pass green (G) and blue (B), the Y filter can only pass green (G) and red (R), and the M filter can only pass blue (B) and red (R). Therefore, the combined color filter array can pass three kinds of color information, CMY.
[0100] After at least one first driving member controls the target color filter array to move an odd number of pixel distances along the row direction of the target color filter array, the C filter in the first color filter array corresponds to the Y filter in the second color filter array, the Y filter in the first color filter array corresponds to the C filter in the second color filter array, the M filter in the first color filter array corresponds to the Y filter in the second color filter array, and the Y filter in the first color filter array corresponds to the M filter in the second color filter array. Therefore, after at least one first driving member controls the target color filter array to move an odd number of pixel distances along the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is a (C+Y)(Y+C)(M+Y)(Y+M) array, i.e., a GGRR array. In the optical field, the B filter can only pass blue (B), the R filter can only pass red (R), the G filter can only pass green (G), the C filter can only pass green (G) and blue (B), the Y filter can only pass green (G) and red (R), and the M filter can only pass blue (B) and red (R). Thus, the combined color filter array can pass two types of color information, GR.
[0101] After at least one second driving member controls the target color filter array to move an odd number of pixel distances along the column direction of the target color filter array, the C filter in the first color filter array corresponds to the M filter in the second color filter array, the Y filter in the first color filter array corresponds to the Y filter in the second color filter array, the M filter in the first color filter array corresponds to the C filter in the second color filter array, and the Y filter in the first color filter array corresponds to the Y filter in the second color filter array. Therefore, after at least one second driving member controls the target color filter array to move an odd number of pixel distances along the column direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is a (C+M)(Y+Y)(M+C)(Y+Y) array, that is, a BYBY array. In the optical field, the B filter can only pass blue (B), the R filter can only pass red (R), the G filter can only pass green (G), the C filter can only pass green (G) and blue (B), the Y filter can only pass green (G) and red (R), and the M filter can only pass blue (B) and red (R). Thus, the combined color filter array can pass BY two kinds of color information.
[0102] In the embodiment of the present application, at least one first driving element and at least one second driving element both control the target color filter array to move in an initial state.
[0103] For example, Fig.16 As shown, the first color filter array 202 is a CYMY array, such as Fig.17 As shown, the second color filter array 203 is a CYMY array. In the initial state, as shown in FIG. Fig.18 As shown, the combined color filter array composed of the first color filter array and the second color filter array is a (C+C)(Y+Y)(M+M)(Y+Y) array, that is, a CYMY array; after at least one first driving element controls the target color filter array to move an odd number of pixels along the row direction of the target color filter array, as shown in FIG. Fig.19 As shown in FIG. 1 , the combined color filter array composed of the first color filter array and the second color filter array is a (C+Y)(Y+C)(M+Y)(Y+M) array, i.e., a GGRR array; after at least one second driving element controls the target color filter array to move an odd number of pixels along the row direction of the target color filter array, as shown in FIG. Fig. 20 As shown, the combined color filter array composed of the first color filter array and the second color filter array is a (C+M)(Y+Y)(M+C)(Y+Y) array, that is, a BYBY array.
[0104] In the embodiment of the present application, after the target color filter array is controlled to move by at least one driving member, a GGRR array and a BYBY array can be obtained respectively, thereby improving the richness of the acquired color information.
[0105] The embodiment of the present application also provides a camera module, which includes the image sensor in the above embodiment. For other descriptions of the camera module, please refer to the relevant descriptions in the above image sensor embodiment.
[0106] The present application also provides an electronic device, which includes the image sensor in the above embodiment. For other descriptions of the electronic device, please refer to the relevant descriptions in the above image sensor embodiment.
[0107] An embodiment of the present application also provides a photographing method, which is applied to the electronic device provided in the above embodiment. Fig.21 FIG. 1 is a flow chart of a shooting method provided in an embodiment of the present application. Fig.21 As shown, the shooting method provided in the embodiment of the present application may include the following steps 101 to 103.
[0108] Step 101: When a first color filter array and a second color filter array in a camera module of an electronic device are in an initial state, the electronic device collects a first original image through the camera module.
[0109] It can be understood that the initial state of the first color filter array and the second color filter array is a state when the first color filter array and the second color filter array completely overlap.
[0110] In this embodiment of the present application, the number of the first original image may be 1.
[0111] Step 102: The electronic device controls at least one driving member of the camera module to move at least once to drive the target color filter array to move at least once within the plane where the target color filter array is located, and controls the camera module to capture a group of original images after each movement of the target color filter array is completed, each group of original images including at least one second original image.
[0112] In the embodiment of the present application, each movement of at least one driving member of the camera module controlled by the electronic device is controlled when the first color filter array and the second color filter array are in the initial state.
[0113] Step 103: The electronic device outputs a target image based on the first original image and at least one second original image in each group of original images.
[0114] The target color filter array is at least one of the first color filter array and the second color filter array, and the color parameters of the second original images in different groups are different.
[0115] In an embodiment of the present application, the electronic device outputs a target image based on the first original image and at least one second original image in each group of original images. It can be understood that when the electronic device obtains a group of original images, the electronic device outputs the target image based on the first original image and at least one second original image in the group of original images; when the electronic device obtains multiple groups of original images, the electronic device outputs the target image based on the first original image and at least one second original image in each group of original images in the multiple groups of original images.
[0116] Optionally, the color parameter included in the original image may be at least one of R, G, B, C, Y, and M. In actual implementation, the original image may not include any of the above color parameters.
[0117] In the embodiment of the present application, each time at least one driving member drives the target color filter array to move, the combined color filter array formed by the first color filter array and the second color filter array is different, so the color parameters of the second original images in different groups are different.
[0118] In the shooting method provided in the embodiment of the present application, since the electronic device can obtain a first original image in an initial state, and obtain a second original image containing different color parameters after controlling the movement of at least one driving member each time, and can output a target image based on the first original image and the second original image containing different color parameters, the quality of the captured image can be improved.
[0119] The shooting method provided by the embodiment of the present application is further described below in combination with two possible implementations, wherein one possible implementation is: one of the first color filter array and the second color filter array is a CYYM array, and the other of the first color filter array and the second color filter array is a BRGB array. Another possible implementation is: one of the first color filter array and the second color filter array is a CYMY array, and the other of the first color filter array and the second color filter array is a CYMY array.
[0120] Optionally, in some embodiments of the present application, one of the first color filter array and the second color filter array is a CYYM array, and the other of the first color filter array and the second color filter array is a BRGB array; at least one driving element includes a driving element; the above step 102 can be specifically implemented by the following steps 102a and 102b.
[0121] It can be understood that when the first color filter array is a CYYM array, the second color filter array is a BRGB array; when the first color filter array is a BRGB array, the second color filter array is a CYYM array.
[0122] Step 102a: The electronic device controls a driving element to move along the row direction of the target color filter array to drive the target color filter array to move a first preset distance along the row direction.
[0123] The first preset distance may be an odd number of pixel distances, for example, the first preset distance is the width of one pixel.
[0124] In the embodiment of the present application, after the target color filter array moves a first preset distance along the row direction, the relevant array features of the combined color filter array composed of the first color filter array and the second color filter array can refer to the relevant description in the above embodiment, and in order to avoid repetition, they will not be repeated here.
[0125] Step 102b: After the target color filter array has completed moving along the row direction, the electronic device controls the camera module to collect a first set of original images.
[0126] The first group of original images may include at least two second original images.
[0127] In the embodiments of the present application, for the relevant description of the "row direction", please refer to the relevant description in the above embodiments. In order to avoid repetition, it will not be repeated here.
[0128] In the embodiment of the present application, after the target color filter array is moved along the row direction, the combined color filter array composed of the first color filter array and the second color filter array is a (Y+B)(C+R)(M+G)(Y+B) array, thereby completely blocking the light from the external environment and obtaining the original image without any external environment color parameters.
[0129] In the embodiment of the present application, when one of the first color filter array and the second color filter array is a CYYM array, and the other of the first color filter array and the second color filter array is a BRGB array, since the target color filter array can be driven by a driving member to move an odd number of pixels in the row direction, a second original image that does not contain external environment color information can be obtained, thereby improving the diversity of the original image obtained.
[0130] Optionally, in the embodiment of the present application, the above step 103 can be specifically implemented by the following steps 103a and 103b.
[0131] Step 103a: The electronic device obtains noise information of the camera module based on at least one second original image in each group of original images.
[0132] The noise information includes at least one of background noise information and fixed noise information of the camera module.
[0133] In the embodiment of the present application, the electronic device can obtain the noise model and the fitting curve corresponding to the background noise information according to the background noise information.
[0134] For example, Fig. 22 and Fig.23 As shown, after obtaining accurate background noise information in the current environment, the electronic device can obtain a noise model based on the background noise information. Fig. 22 The horizontal axis in represents the sensitivity of the first original image, Fig. 22 The ordinate axis in represents the noise model of the first original image. In this way, the electronic device can obtain the noise model of the first original image at different sensitivities. Fig. 22 includes actual noise data and initial noise data, where Fig. 22 and Fig.23 The circles in the figure represent the actual noise data. Fig. 22 The squares in the figure represent the initial noise data; Fig.23 The horizontal axis in represents the sensitivity of the first original image, Fig.23 The abscissa axis in represents the fitting curve of the first original image. In this way, the electronic device can obtain the fitting curve of the first original image at different sensitivities.
[0135] Step 103b: The electronic device performs denoising processing on the first original image based on the noise information and outputs a target image.
[0136] Optionally, when the noise information includes both background noise information and fixed noise information of the camera module, there is no limitation on the order of denoising the first original image based on the noise information. For example, the process may be based on the background noise information first and then on the fixed noise information; or the process may be based on the fixed noise information first and then on the background noise information; or the process may be based on the fixed noise information at the same time and the first original image may be denoised based on both the background noise information and the fixed noise information.
[0137] In the embodiment of the present application, since the external shooting environment of the at least one second original image is the same as the external shooting environment of the first original image, it can be considered that the noise information of the at least one second original image is the same as the noise information of the first original image, wherein the noise information includes background noise information and fixed noise information. Therefore, the noise information obtained through the at least one second original image can be used to denoise the first original image.
[0138] In an embodiment of the present application, after the electronic device obtains the noise model and fitting curve corresponding to the background noise information according to the background noise information, it can use artificial intelligence (AI) noise reduction technology to denoise the first original image according to the obtained noise model and fitting curve.
[0139] In the embodiment of the present application, the electronic device can directly perform denoising on the first original image according to the fixed noise information.
[0140] In the embodiment of the present application, since the first original image can be denoised based on at least one of the acquired background noise information and fixed noise information, the noise in the first original image can be reduced, and thus the quality of the captured image can be improved.
[0141] Optionally, in some embodiments of the present application, one of the first color filter array and the second color filter array is a CYMY array, and the other of the first color filter array and the second color filter array is a CYMY array; at least one driving member includes at least one first driving member and at least one second driving member, and each first driving member is arranged adjacent to a second driving member; the above-mentioned step 102 can be specifically implemented by the following steps 102c to 102f.
[0142] Step 102c: The electronic device controls at least one first driving element to move along the row direction of the target color filter array, so as to drive the target color filter array to move a second preset distance along the row direction.
[0143] The second preset distance may be an odd number of pixel distances, for example, the second preset distance is the width of one pixel.
[0144] Step 102d: After the target color filter array has completed moving along the row direction, the electronic device controls the camera module to capture a second set of original images.
[0145] In the embodiment of the present application, after the target color filter array is moved along the row direction, the first color filter array and the second color filter array can form a new combined color filter array, so that the camera module captures a second set of original images through the new combined color filter array.
[0146] In the embodiment of the present application, for the relevant description of the combined color filter array composed of the first color filter array and the second color filter array after the target color filter array moves along the row direction by a second preset distance, reference can be made to the relevant description in the above embodiment, which will not be repeated here to avoid repetition.
[0147] Step 102e: The electronic device controls at least one second driving element to move along the column direction of the target color filter array, so as to drive the target color filter array to move a third preset distance along the column direction.
[0148] The third preset distance may be an odd number of pixel distances, for example, the third preset distance is the width of one pixel.
[0149] In the embodiments of the present application, for the relevant description of the "column direction", please refer to the relevant description in the above embodiments. In order to avoid repetition, it will not be repeated here.
[0150] Step 102f: After the target color filter array has completed moving along the column direction, the electronic device controls the camera module to capture the third set of original images.
[0151] The second group of original images and the third group of original images may each include one second original image.
[0152] In the embodiment of the present application, after the target color filter array is moved along the column direction, the first color filter array and the second color filter array can form a new combined color filter array, so that the camera module captures a third group of original images through the new combined color filter array.
[0153] In the embodiment of the present application, after the target color filter array moves a third preset distance along the column direction, the relevant array features of the combined color filter array composed of the first color filter array and the second color filter array can refer to the relevant description in the above embodiment. In order to avoid repetition, they will not be repeated here.
[0154] In the embodiment of the present application, when the first color filter array and the second color filter array are both CYMY arrays, since the target color filter array can be driven by the driving member to move a second preset distance along the row direction of the target color filter array, and the target color filter array can be driven by the driving member to move a third preset distance along the column direction of the target color filter array, different combined color filter arrays can be formed, thereby improving the diversity of color parameters contained in the acquired original image.
[0155] Optionally, in the embodiment of the present application, the above step 103 can be specifically implemented by the following step 103c.
[0156] Step 103c: The electronic device fuses the first original image with at least one second original image in each group of original images to output a target image.
[0157] The target image includes RGB color information and CMY color information.
[0158] Specifically, the electronic device can fuse the first original image and at least one second original image in each group of original images through a color fusion algorithm to obtain a target image. For related technologies of the color fusion algorithm, please refer to relevant technologies. In order to avoid repetition, it will not be described here.
[0159] In the embodiment of the present application, since the color parameters included in the first original image and at least one second original image are different, the first original image and at least one second original image are fused, and the obtained target image can simultaneously include the color parameters included in the first original image and at least one second original image, thereby improving the richness of the color parameters included in the target image.
[0160] Optionally, the shooting method provided in the embodiment of the present application may further include the following step 104.
[0161] Step 104: The electronic device processes the target image signal to obtain a final image.
[0162] The final image may be an image displayed in the shooting preview interface; or may be an image saved in the electronic device under the triggering of the user.
[0163] The following is an illustrative description of the shooting method provided in the embodiment of the present application in conjunction with specific scenarios.
[0164] For application scenarios where users need to obtain images with relatively rich color information, exemplarily, the first color filter array and the second color filter array in the camera module of the electronic device are CYMY arrays respectively; the electronic device can capture an original image 1 without moving the color filter array, wherein the original image 1 contains CMY color information; then, the electronic device first controls a driver to control the first color filter array, i.e., the target color filter array, to move along the row direction of the first color filter array by a distance of 1 pixel, wherein the first color filter array is the target color filter array, and after the movement, captures an original image 2, i.e., the original image 2 contains GR color information, and controls the driver to control the first color filter array to move to the original state; then controls another driver to control the first color filter array to move along the column direction of the first color filter array by a distance of 1 pixel, and after the movement, captures an original image 3, i.e., the original image 3 contains BY color information; the electronic device obtains a fused image through image fusion processing, wherein the fused image contains RGB color information and CMY color information, thereby improving the richness of color information contained in the target image.
[0165] For application scenarios where the shooting environment is poor, the user needs to remove noise information from the captured images, wherein the noise information includes background noise information and fixed noise information. For example, the first color filter array in the camera module of the electronic device is a CYYM array, and the second color filter array is a BRGB array; the electronic device can capture an original image 1 without moving the color filter array; then, the electronic device controls a driving component to control the first color filter array to move a distance of 1 pixel along the row direction of the first color filter array, wherein the first color filter array is a target color filter array, and captures two original images 2 after the movement, that is, the original images 2 do not contain any color information; the electronic device obtains background noise information and fixed noise information based on the two original images 2, and obtains a noise model and a fitting curve corresponding to the background noise information based on the background noise information; finally, the electronic device removes the background noise of the first original image according to the obtained noise model and fitting curve through artificial intelligence noise reduction technology; the first original image is removed by fixed noise information, so that the noise information in the original image 1 can be removed.
[0166] It should be noted that the shooting method provided in the embodiment of the present application can be executed by a shooting device or a control module in the shooting device for executing the shooting method. In the embodiment of the present application, the shooting device provided in the embodiment of the present application is described by taking the shooting method executed by the shooting device as an example.
[0167] Combination Fig.24 The embodiment of the present application provides a shooting device 900, which may include: an acquisition module 901, a control module 902 and a processing module 903; the acquisition module 901 is used to acquire a first original image through the camera module when the first color filter array and the second color filter array in the camera module of the electronic device are in an initial state; the control module 902 is used to control at least one driving member of the camera module to move at least once, so as to drive the target color filter array to move at least once in the plane where the target color filter array is located, and after each movement of the target color filter array, control the camera module to acquire a group of original images, each group of original images including at least one second original image; the processing module 903 is used to output a target image based on the first original image acquired by the acquisition module 901 and at least one second original image in each group of original images; wherein the target color filter array is at least one item of the first color filter array and the second color filter array, and the color parameters of the second original images in different groups are different.
[0168] Optionally, in an embodiment of the present application, one of the first color filter array and the second color filter array is a CYYM array, and the other of the first color filter array and the second color filter array is a BRGB array; at least one driving member includes a driving member; a control module 902 is specifically used to control a driving member to move along the row direction of the target color filter array to drive the target color filter array to move a first preset distance along the row direction, and the first preset distance is an odd number of pixel distances; and after the target color filter array completes the movement along the row direction, the camera module is controlled to capture a first group of original images, wherein the first group of original images includes at least two second original images.
[0169] Optionally, in an embodiment of the present application, the processing module 903 is specifically used to obtain noise information of the camera module based on at least one second original image in each group of original images; based on the noise information, denoising the first original image, and outputting a target image; wherein the noise information includes at least one of the background noise information and fixed noise information of the camera module.
[0170] Optionally, in an embodiment of the present application, one of the first color filter array and the second color filter array is a CYMY array, and the other of the first color filter array and the second color filter array is a CYMY array; the at least one driving member includes at least one first driving member and at least one second driving member, and each first driving member is arranged adjacent to a second driving member; the control module 902 is specifically used to control the at least one first driving member to move along the row direction of the target color filter array to drive the target color filter array to move along the row direction by a second preset distance, and the second preset distance is an odd number of pixel distances; after the target color filter array is moved along the row direction, the camera module is controlled to capture a second group of original images; the at least one second driving member is controlled to move along the column direction of the target color filter array to drive the target color filter array to move along the column direction by a third preset distance, and the third preset distance is an odd number of pixel distances; after the target color filter array is moved along the column direction, the camera module is controlled to capture a third group of original images; wherein, the second group of original images and the third group of original images each include a second original image.
[0171] Optionally, in an embodiment of the present application, the processing module 903 is specifically used to merge the first original image with at least one second original image in each group of original images to output a target image; wherein the target image includes RGB color information and CMY color information.
[0172] In the shooting device provided in the embodiment of the present application, since the shooting device can obtain a first original image in an initial state, and obtain a second original image containing different color parameters after controlling the movement of at least one driving member each time, and can output a target image based on the first original image and the second original image containing different color parameters, the quality of the captured image can be improved.
[0173] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.
[0174] The shooting device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not make specific limitations.
[0175] The shooting device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0176] The shooting device provided in the embodiment of the present application can achieve Fig.21 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0177] like Fig.25As shown, an embodiment of the present application also provides an electronic device 200, including a processor 202, a memory 201, and a program or instruction stored in the memory 201 and executable on the processor 202. When the program or instruction is executed by the processor 202, each process of the above-mentioned shooting method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0178] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
[0179] Fig.26 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0180] The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and other components.
[0181] Optionally, the sensor 1005 includes: an image sensor. Figure 3 As shown, the image sensor 200 includes: a photosensitive array 201, a first color filter array 202, a second color filter array 203 and at least one driving element 204; the first color filter array 202 and the second color filter array 203 are arranged in a superimposed manner, and the first color filter array 202 and the second color filter array 203 are both arranged on the light incident side of the photosensitive array 201; the at least one driving element 204 is connected to a target color filter array, and the target color filter array is at least one of the first color filter array 202 and the second color filter array 203; wherein the at least one driving element 204 is used to control the target color filter array to move within the plane where the target color filter array is located; before and after the target color filter array moves, the combined color filter array composed of the first color filter array 202 and the second color filter array 203 is different.
[0182] Optionally, in the embodiment of the present application, at least one driving element is specifically used to control the target color filter array to move an odd number of pixels within the plane where the target color filter array is located.
[0183] Optionally, in an embodiment of the present application, after the target color filter array moves, the effective photosensitive area of the photosensitive array is located within the projection area of the combined color filter array on the photosensitive array.
[0184] Optionally, in an embodiment of the present application, one of the first color filter array (202) and the second color filter array (203) is a cyan-yellow-magenta or magenta-yellow CYYM array, and the other of the first color filter array (202) and the second color filter array (203) is a blue-red-green-blue BRGB array; at least one driving element (204) includes a driving element (204), and the driving element (204) is used to control the target color filter array to move a first preset distance along the row direction of the target color filter array, and the first preset distance is an odd number of pixel distances; after the driving element (204) controls the target color filter array to move a first preset distance along the row direction of the target color filter array, the combined color filter array formed by the first color filter array (202) and the second color filter array (203) is: (Y+B)(C+R)(M+G)(Y+B) array.
[0185] Optionally, in the embodiment of the present application, one of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array, and the other of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array; the at least one driving element includes at least one first driving element and at least one second driving element, and each first driving element is arranged adjacent to a second driving element; wherein the at least one first driving element is used to control the target color filter array to move a second preset distance along the row direction of the target color filter array, and the second preset distance is an odd number of pixel distances; when the at least one first driving element controls the target color filter array to move a second preset distance along the row direction of the target color filter array, the second preset distance is an odd number of pixel distances; After the wave array moves a second preset distance in the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is: (Y+C)(C+Y)(Y+M)(M+Y) array; at least one second driving element is used to control the target color filter array to move a third preset distance in the column direction of the target color filter array, and the third preset distance is an odd number of pixels; after at least one second driving element controls the target color filter array to move a third preset distance in the column direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is: (M+C)(Y+Y)(C+M)(Y+Y) array.
[0186] Optionally, in an embodiment of the present application, at least one driving element is made of piezoelectric material; wherein, when a voltage is applied to the driving element, the driving element deforms along the plane where the target color filter array is located, and drives the target color filter array to move an odd number of pixels in the plane where the target color filter array is located.
[0187] For other descriptions of the image sensor, please refer to the relevant descriptions in the above-mentioned image sensor embodiments, which will not be repeated here to avoid repetition.
[0188] In an image sensor provided by an embodiment of the present application, the image sensor includes: a photosensitive array, a first color filter array, a second color filter array, and at least one driving member; and at least one driving member can be used to control the target color filter array to move within the plane where the target color filter array is located, wherein the target color filter array is at least one of the first color filter array and the second color filter array; before and after the target color filter array moves, the combined color filter array composed of the first color filter array and the second color filter array is different. This can improve the richness of color information obtained by the image sensor, and thus improve the quality of the captured image.
[0189] Those skilled in the art will appreciate that the electronic device 1000 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1010 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.26 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0190] Among them, the processor 1010 is used to collect a first original image through the camera module when the first color filter array and the second color filter array in the camera module of the electronic device are in an initial state; the processor 1010 is also used to control at least one driving member of the camera module to move at least once, so as to drive the target color filter array to move at least once in the plane where the target color filter array is located, and after each movement of the target color filter array is completed, control the camera module to collect a group of original images, each group of original images includes at least one second original image; the processor 1010 is also used to output a target image based on the collected first original image and at least one second original image in each group of original images; wherein the target color filter array is at least one item of the first color filter array and the second color filter array, and the color parameters of the second original images in different groups are different.
[0191] Optionally, in an embodiment of the present application, one of the first color filter array and the second color filter array is a CYYM array, and the other of the first color filter array and the second color filter array is a BRGB array; at least one driving member includes a driving member; the processor 1010 is specifically used to control a driving member to move along the row direction of the target color filter array to drive the target color filter array to move a first preset distance along the row direction, and the first preset distance is an odd number of pixel distances; and after the target color filter array completes the movement along the row direction, the camera module is controlled to capture a first group of original images, and the first group of original images includes at least two second original images.
[0192] Optionally, in an embodiment of the present application, the processor 1010 is specifically used to obtain noise information of the camera module based on at least one second original image in each group of original images; based on the noise information, denoising the first original image, and outputting a target image; wherein the noise information includes at least one of background noise information and fixed noise information of the camera module.
[0193] Optionally, in an embodiment of the present application, one of the first color filter array and the second color filter array is a CYMY array, and the other color filter array in the first color filter array and the second color filter array is a CYMY array; at least one driving member includes at least one first driving member and at least one second driving member, and each first driving member is arranged adjacent to a second driving member; the processor 1010 is specifically used to control at least one first driving member to move along the row direction of the target color filter array to drive the target color filter array to move along the row direction by a second preset distance, and the second preset distance is an odd number of pixel distances; after the target color filter array is moved along the row direction, the camera module is controlled to capture a second group of original images; the at least one second driving member is controlled to move along the column direction of the target color filter array to drive the target color filter array to move along the column direction by a third preset distance, and the third preset distance is an odd number of pixel distances; after the target color filter array is moved along the column direction, the camera module is controlled to capture a third group of original images; wherein the second group of original images and the third group of original images each include a second original image.
[0194] Optionally, in an embodiment of the present application, the processor 1010 is specifically configured to fuse the first original image with at least one second original image in each group of original images, and output a target image; wherein the target image includes RGB color information and CMY color information.
[0195] In the electronic device provided in the embodiment of the present application, since the electronic device can obtain a first original image in an initial state, and obtain a second original image containing different color parameters after controlling the movement of at least one driving member each time, and can output a target image based on the first original image and the second original image containing different color parameters, the quality of the captured image can be improved.
[0196] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.
[0197] It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1007 includes a touch panel 10071 and at least one of other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0198] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0199] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.
[0200] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned shooting method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0201] The processor is a processor in the electronic device in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk. The present application embodiment further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a program or instruction to implement each process of the above shooting method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0202] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0203] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned shooting method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0204] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0205] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.
[0206] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image sensor, characterized in that: The image sensor comprises: a photosensitive array, a first color filter array, a second color filter array and at least one driving element; The first color filter array and the second color filter array are arranged in a superimposed manner, and the first color filter array and the second color filter array are both arranged on the light incident side of the photosensitive array; The at least one driving member is connected to a target color filter array, and the target color filter array is at least one of the first color filter array and the second color filter array; Wherein, the at least one driving member is used to control the target color filter array to move within the plane where the target color filter array is located; Before and after the target color filter array moves, a combined color filter array formed by the first color filter array and the second color filter array is different.
2. The image sensor according to claim 1, characterized in that The at least one driving element is specifically used to control the target color filter array to move an odd number of pixel distances within the plane where the target color filter array is located.
3. The image sensor according to claim 1, characterized in that After the target color filter array moves, the effective photosensitive area of the photosensitive array is located within the projection area of the combined color filter array on the photosensitive array.
4. The image sensor according to any one of claims 1 to 3, characterized in that: One of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYYM array, and the other of the first color filter array and the second color filter array is a blue-red-green-blue BRGB array; Wherein, the at least one driving element comprises one driving element, and the one driving element is used to control the target color filter array to move a first preset distance along a row direction of the target color filter array, and the first preset distance is an odd number of pixel distances; After the one driving component controls the target color filter array to move the first preset distance along the row direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is: (Y+B)(C+R)(M+G)(Y+B) array.
5. The image sensor according to any one of claims 1 to 3, characterized in that: One of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array, and the other of the first color filter array and the second color filter array is a cyan-yellow-magenta or magenta-yellow CYMY array; The at least one driving member comprises at least one first driving member and at least one second driving member, and each first driving member is disposed adjacent to a second driving member; Wherein, the at least one first driving member is used to control the target color filter array to move a second preset distance along the row direction of the target color filter array, and the second preset distance is an odd number of pixel distances; after the at least one first driving member controls the target color filter array to move the second preset distance along the row direction of the target color filter array, a combined color filter array composed of the first color filter array and the second color filter array is: (Y+C)(C+Y)(Y+M)(M+Y) array; The at least one second driving member is used to control the target color filter array to move a third preset distance along the column direction of the target color filter array, and the third preset distance is an odd number of pixel distances; after the at least one second driving member controls the target color filter array to move the third preset distance along the column direction of the target color filter array, the combined color filter array composed of the first color filter array and the second color filter array is: (M+C)(Y+Y)(C+M)(Y+Y) array.
6. The image sensor according to any one of claims 1 to 3, characterized in that: The at least one driving member is made of piezoelectric material; Wherein, when the driving element applies voltage, the driving element deforms along the plane where the target color filter array is located, and drives the target color filter array to move an odd number of pixels on the plane where the target color filter array is located.
7. A camera module, characterized in that: Comprising the image sensor as claimed in any one of claims 1 to 6.
8. An electronic device, characterized in that: The electronic device comprises the camera module as claimed in claim 7.
9. A shooting method, applied to the electronic device as claimed in claim 8, characterized in that: The method comprises: When the first color filter array and the second color filter array in the camera module of the electronic device are in an initial state, collecting a first original image through the camera module; Controlling at least one driving member of the camera module to move at least once, so as to drive the target color filter array to move at least once in the plane where the target color filter array is located, and controlling the camera module to capture a group of original images after each movement of the target color filter array, wherein each group of original images includes at least one second original image; Outputting a target image based on the first original image and at least one second original image in each group of original images; The target color filter array is at least one of the first color filter array and the second color filter array, and the color parameters of the second original images in different groups are different.
10. The method according to claim 9, characterized in that One of the first color filter array and the second color filter array is a CYYM array, and the other of the first color filter array and the second color filter array is a BRGB array; the at least one driving element includes a driving element; The at least one driving member of the camera module is controlled to move at least once to drive the target color filter array to move at least once in the plane where the target color filter array is located, and after each movement of the target color filter array, the camera module is controlled to collect a group of original images, each group of original images including at least one second original image, including: Controlling the one driving member to move along the row direction of the target color filter array to drive the target color filter array to move a first preset distance along the row direction, wherein the first preset distance is an odd number of pixel distances; After the target color filter array has completed moving along the row direction, the camera module is controlled to collect a first group of original images, wherein the first group of original images includes at least two second original images.
11. The method according to claim 10, characterized in that The step of outputting a target image based on the first original image and at least one second original image in each group of original images comprises: Based on at least one second original image in each group of original images, obtaining noise information of the camera module; Based on the noise information, denoising the first original image is performed to output the target image; The noise information includes at least one of background noise information and fixed noise information of the camera module.
12. The method according to claim 9, characterized in that One of the first color filter array and the second color filter array is a CYMY array, and the other of the first color filter array and the second color filter array is a CYMY array; the at least one driving member includes at least one first driving member and at least one second driving member, and each first driving member is disposed adjacent to a second driving member; The at least one driving member of the camera module is controlled to move at least once to drive the target color filter array to move at least once in the plane where the target color filter array is located, and after each movement of the target color filter array, the camera module is controlled to collect a group of original images, each group of original images including at least one second original image, including: Controlling the at least one first driving member to move along the row direction of the target color filter array to drive the target color filter array to move a second preset distance along the row direction, where the second preset distance is an odd number of pixel distances; After the target color filter array has completed moving along the row direction, controlling the camera module to capture a second set of original images; Controlling the at least one second driving member to move along the column direction of the target color filter array to drive the target color filter array to move a third preset distance along the column direction, wherein the third preset distance is an odd number of pixel distances; After the target color filter array has completed moving along the column direction, controlling the camera module to capture a third set of original images; The second group of original images and the third group of original images each include a second original image.
13. The method according to claim 12, characterized in that The step of outputting a target image based on the first original image and at least one second original image in each group of original images comprises: Merging the first original image with at least one second original image in each group of original images to output the target image; Wherein, the target image includes RGB color information and CMY color information.
14. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the shooting method as claimed in any one of claims 9 to 13.
15. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the shooting method according to any one of claims 9 to 13 are implemented.
